Missing Samples

8 posts / 0 new
Last post
Offline
Last seen: 6 years 1 month ago
Joined: 07/12/2012
Posts: 51
Missing Samples

We miss samples on a configuration topic. Reliability QoS is RELIABLE and durability is TRANSIENT LOCAL. The topic is published once on startup

and there are 1600 samples. However, I noticed that only 1024 is consistenly read. I have read some articles that mention this behavior:

http://community.rti.com/kb/why-does-my-dds-datareader-miss-first-few-samples, but the consistency of the magic number 1024 makes me wonder if

there is maybe a configuration setiing that needs to be changed. Both writer and reader have the QoS mentioned - if was verified with Analyzer.

Thank you

Nico.

 

Organization:
Gerardo Pardo's picture
Offline
Last seen: 16 hours 29 min ago
Joined: 06/02/2010
Posts: 601

Hi Nico,

What is your HISTORY Qos policy? Is it set to KEEP_LAST (this is the default). If so this maybe the problem.

Take a look at this posting as well: http://community.rti.com/kb/why-isnt-my-reliable-data-reader-receiving-all-data-samples

Gerardo

 

Offline
Last seen: 7 months 5 days ago
Joined: 05/23/2013
Posts: 64

Hi Nico,

You can try to change the "max_samples_per_read " and "max_fragmented_samples" configurations to 1600 as the default values of them are 1024.
I am suggesting this approach because you always see that magic number 1024.

Thanks,
Kyoungho

Offline
Last seen: 5 years 8 months ago
Joined: 01/31/2011
Posts: 37

Are you using read() or take() to access the data?

As Kyoungho pointed out, 1024 is the default value for the DataReaderQos.reader_resource_limits.max_samples_per_read (link to html docs for max_samples_per_read).  This limit controls the number of samples that the application can receive from the middleware in a single call to read() or take(). If more data exists, the application will need to issue multiple read/take calls.

If you are using take(), the first call to take() will remove the first 1024 samples from the reader's local cache so that subsequent calls will return additional data samples (1025-1600, in your case).  If you are using read(), then the call to read() will "keep" the samples in the reader's local cache.  Subsequent calls to read() will return the same 1024 samples, unless you specify the DDS_NOT_READ_SAMPLE_STATE flag in the call to read().

As Kyoungho suggested, increasing the the max_samples_per_read can make sure you get all of the data in a single call to take() or read().

Offline
Last seen: 6 years 1 month ago
Joined: 07/12/2012
Posts: 51

Hi Kyoungho and Sumeet,

As both of you recommended, increasing max_samples_per_read to 1600 resolves the issue. Thanks for the help

Nico.

 

Offline
Last seen: 6 years 1 month ago
Joined: 07/12/2012
Posts: 51

Sumeet,

I am doing a read() on the data with SampleStateKind/NOT_READ_SAMPLE_STATE and still only get 1024 samples with multiple reads.

The other states used are 

ResourceLimitsQosPolicy/LENGTH_UNLIMITED
ViewStateKind/ANY_VIEW_STATE
InstanceStateKind/ANY_INSTANCE_STATE

Anyway increasing the value of max_samples_per_read allows me to get all samples with one read

Thanks

Nico.

 

Offline
Last seen: 6 years 1 month ago
Joined: 07/12/2012
Posts: 51

Here are some more information on the issue: I am using a StatusCondition to read new samples with StatusKind/DATA_AVAILABLE_STATUS enabled.

From the behaviour I saw, one can assume that the StatusCondition will only trigger once when all 1600 samples are in the reader cache and with

max_samples_per_read set to the default of 1024, only 1024 samples will be read(). Is it possible to trigger a StatusCondition for samples not read so that

it triggers multiple times or is a ReadCondition the only way to go ?

Thanks

Nico.

 

Offline
Last seen: 5 years 8 months ago
Joined: 01/31/2011
Posts: 37

Hi Nico,

Your observation is correct - the StatusCondition (DATA_AVAILABLE_STATUS) triggers once the data arrives, but is cleared as soon as you call read()/take().  It will not be triggered again until new data arrives, even if you don't read everything that's available.

You have two options:

  1. Use a ReadCondition
  2. Change how you access the data relative to the WaitSet.  Instead of blocking on the WaitSet and then calling read() when the WaitSet unblocks, instead try to read data before blocking on the WaitSet.

The logic goes something like this:

  1. Call DataReader::read()
  2. If DataReader::read() returns data, process.  After you finish processing, return to step (1)
  3. If the DataReader::read() returns DDS_RETCODE_NO_DATA, then call WaitSet::wait() with the StatusCondition.

In this way, you're first checking if data is available before blocking in the WaitSet. This gets around the situation where there is unread data in the queue, but the WaitSet does not get woken up because the StatusCondition has not been re-triggered.

-sumeet