Copy of samples from sequence in Java API

4 posts / 0 new
Last post
Offline
Last seen: 6 years 1 month ago
Joined: 07/12/2012
Posts: 51
Copy of samples from sequence in Java API

This question has to do with general programming and concurrency, but since the Java API is an exception

from the other languages in the RTI API, it is a valid question for this  forum I think. According to a post (https://community.rti.com/examples/using-sequences), the Java API does not support deep copy of objects from

sequences. For example the C++ API support copy of object without using Loanable sequences.

Consider this Java code for generic read() of objects:

public synchronized <T, R> List read_io(jDaProcessDDS<T, R> p) {
		SampleInfoSeq infoSeq = new SampleInfoSeq();
		TS dataSeq = (TS) new LoanableSequence(Class.class);
		List<R> results = new ArrayList<>();

		try {
			reader.read_untyped(dataSeq, infoSeq,
					ResourceLimitsQosPolicy.LENGTH_UNLIMITED,
					SampleStateKind.ANY_SAMPLE_STATE,
					ViewStateKind.ANY_VIEW_STATE,
					InstanceStateKind.ALIVE_INSTANCE_STATE);

			for (int i = 0; i < dataSeq.size(); ++i) {
				if (true == ((SampleInfo) infoSeq.get(i)).valid_data) {
					results.add(p.runit((T) dataSeq.get(i)));
				}
			}
		} catch (RETCODE_NO_DATA noDataEx) {
			// Do nothing
		} finally {
			reader.return_loan_untyped(dataSeq, infoSeq);
		}

		return results;
	}

This will work most of the time, but will be problematic if another caller calls this function while the previous caller is still processing its returned data (The next call may change data that the previous caller still has references to). it seems a deep copy of data must be returned to fix this function. I would like to know what the most idiomatic and efficient way data can be copied in Java ?

Thanks

Organization:
Keywords:
Gerardo Pardo's picture
Offline
Last seen: 18 hours 7 min ago
Joined: 06/02/2010
Posts: 601

Hi,

I am a bit confused by the sentence in the post  you reference (https://community.rti.com/examples/using-sequences) that says:

In all APIs except Java, FooSeq contains deep copies of Foo elements; in Java, which does not provide direct support for deep copy semantics, FooSeq contains references to Foo objects. In Java, sequences implement the java.util.List interface, and thus support all of the collection APIs and idioms familiar to Java programmers.

I think that sentence may be a bit misleading. In all Languages APIs (including Java, C, C++, C#) when you do a "read/take" to get a sequence back from the DataReader, the samples are returned "by reference" meaning each element points to a buffer managed by the DataReader that needs to be returned by using the  return_loan operation. During the time the application holds those references (i.e. up until it calls the return_loan) those samples are "locked" so the DataReader cannot modify them even if new data arrives...

Given this it seems perfectly OK to have another thread call the read_io operation  while another thread is processing that data because nobody will be modifying the content of the samples received. The DataReader will have two outstanding loans, so it will not modify them until both threads call return_loan and the threads processing the data sholuld only read, not write into those data objects.

A different question is whether you do want two threads to concurrently process the sama data or rather you want each sample to be processed just once. If you wanted the data to be processed once you could call "read" woth the SampleStateKind.NOT_READ_SAMPLE_STATE that will make sure that you will not read the same sample twice. Alternatively you could use take instead of read.

Gerardo

Offline
Last seen: 6 years 1 month ago
Joined: 07/12/2012
Posts: 51

Hi Gerardo,

Thanks, your post does clear up the issues. The behavior of the sequences is worth understanding well, so I have a follow-up :

 The intention of the read_io function was to return data to be processed by the calling thread. That is the reason I asked about a deep copy of the data since is apparent from what you said that the reference to the data will be invalid after return_loan() returns. The Java API documentation actually also warns that reference to data should not be used after return_loan() returns.

 Since making a copy of the data is  not ideal, what are the other options ? Is it a good option to let the application allocate a buffer and loan it to the reader instead ?

 

Thanks

Nico

Gerardo Pardo's picture
Offline
Last seen: 18 hours 7 min ago
Joined: 06/02/2010
Posts: 601

Hi Nico,

The alternative I see is to not call return_loan until you have completed the processing of the samples. This was the intent behind having that "loan" concept/API.

The easiest woud be to keep them in the original LoanableSequence container which already extends  java.util.List rather than copy them to a new List<R>.

If you really need to, you can copy the references to another container, but this adds extra complication, see later. Either way the sample remains "loaned" inside the DataReader cache. Once you have processed them you can return the loan.

Thereturn_loan function expects a LoanableSequence, so even if you copied the references to a List<R> I think you should also keep around the original LoanableSequence and SampleInfoSeq so you can use it to call return_loan.

If you were to copy the references to a different container instead of passing the LoanableSequence to the processing logic, then I would need to double check whether  in Java you need to return the original LoanableSequence and SampleInfoSeq. I know that in the case of C/C++ you would need to do that because they not only "borrow" from the DataReader cache the data, but also the memory to hold the sequence of pointers to the data which needs to be also returned (or it would create a memory leak). If that was not the case in Java, then it would make it possible for you to construct the LoanedSequence on the fly from the List<R> once you are done processing, and even return them in different "groups"  that do not exactly corespond to the original ones you got on each sequence, as long as you eventually return all the samples. 

Gerardo