Dynamic data sequence. Delete element. Java

{third times the charm, the first answer was wrong, the second answer was lost when my laptop crashed}

Dynamic Data elements are views on an octet sequence.  The reflection built into the DynamicData object allows it to determine where it is looking in the buffer and present to you, based on knowledge of the Type, what the data is. 

What it isn't, is a(n abstract) sequence of pojo objects. 

We don't provide a conversion method to go back and forth between a pojo (for example, one generated by rtiddsgen) and DynamicData.  If this were available, you could convert the DynamicData into a pojo, use the java abstract sequence api remove() call on the element to be removed, and then convert it back. 

Autogenerating a converter based on IDL is achievable, now that we have rtiddsgen2 to play with.  But I don't think that this is the right use case, as the use of DynamicData in tools presupposes that you don't have access to the IDL that defined the Type, in the first place.

Another option is to recursively copy one DynamicData object into another, skipping the to-be-deleted object, taking into account the reduced number of objects in the sequence.  That might be faster than dumping XML, editing it and then loading it back in.

A third option would be to instead of generating XML, instead convert the DynamicData into a W3C DOM object and manipulate that, then convert back into DynamicData. 

All of these options require that the original octet buffer be converted into some other form first, sorry, except for the DD -> DD copy, which should be the fastest (and probably easiest to implement, since it requires knowledge of only one art form, that being DynamicData).

Now, hopefully this one successfully posts.

You're welcome!

Let us know how you get on.

rip

I'm not quite sure I get the setup.  The edge cases I see are

1) Delete the first element of N|N>1 element sequence

2) Delete the last element of an N|N>1 element sequence

3) Delete a non-first, non-last element of an N|N > 2 element sequence

4) Delete the single (sole) element in a 1 element sequence

your code works for which cases, and is failing for which cases?

If you are deleting the sole element in a one-element list... wouldn't that be also expressed as 'I don't need this sequence at all'?

What happens if instead you simply don't set the dynamicDataCopy sequence complex member at all?   I'm trying to remember if this what the DynamicData reflection expects.

Shot in the dark.

See, now that's just weird.

Any chance you can supply a reproducer to support@rti.com?  If you don't have a support account, you can send it to me directly and I'll look at it.  My email is com.rti@rip (a human should figure that out :) )

You have a comment: 

  //They haven't been set and 3 and 4 are beyond the sequence bounds - should these raise an exception? 

Yep:

Exception in thread "main" com.rti.dds.infrastructure.RETCODE_NO_DATA
    at com.rti.dds.util.Utilities.rethrow(Utilities.java:92)
    at com.rti.dds.infrastructure.RETCODE_ERROR.check_return_codeI(RETCODE_ERROR.java:142)
    at com.rti.dds.dynamicdata.DynamicData.get_int(DynamicData.java:464)
    at DynamicDataTest.main(DynamicDataTest.java:53)
DDS_DynamicData_get_long:!get_long

(disregard the 'at DynamicDataTest.main(DynamicDataTest.java:53' line number.  I'm adding commentary to the file so that number no longer has shifted some).

If that is different from what you are getting, that's problematic.  What platform are you on?

(Same as ''//Again, returns zero, but as count is only one shouldn't this raise an exception?'', it does on my system)

Now, for 'because, reasons'.

Try this methodolgy.  Instead of trying to get/set the individual items, get the sequence, manipulate the sequence, set the sequence.  Note this rework uses bind/unbind and typecode peeking to determine that I need to use get_int_seq/set_int_seq...even though the code sets it up that way earlier.  Because, reasons.

import com.rti.dds.dynamicdata.DynamicData;
import com.rti.dds.infrastructure.*;
import com.rti.dds.typecode.*;
 
public class DynamicDataTest 
{
    public static void main(String[] args) 
    {
        
        //Create a type of sequence of TC_LONG (int) of max 2 elements
        TypeCode intSeqTC = TypeCodeFactory.TheTypeCodeFactory.create_sequence_tc(2, TypeCode.TC_LONG);
        // typedef sequence<long, 2> intSeqTC;
        
        //Create a structure containing one of the above type
        StructMember[] sms=null;
        sms = new StructMember[1];
        sms[0] = new StructMember("seq",false,TypeCode.NOT_BITFIELD,false,intSeqTC);
         
        //Create a type of the above structure
        TypeCode complexTC = TypeCodeFactory.TheTypeCodeFactory.create_struct_tc("intSeq", sms);
        
        //Dynamic data for the above complex type
        DynamicData complexDynamicData = new DynamicData(complexTC,DynamicData.PROPERTY_DEFAULT);
       
        //Dynamic data for the complex type's content
        DynamicData dynamicDataOfSeq = new DynamicData(intSeqTC,DynamicData.PROPERTY_DEFAULT);
        DynamicData scratchDynamicData = new DynamicData();
        
        int count;
        count = dynamicDataOfSeq.get_member_count();
        
        //Set two successive elements in the empty sequence to 10 and 20
        dynamicDataOfSeq.set_int(null, 1, 10);
        dynamicDataOfSeq.set_int(null, 2, 20);
        int val;
        val = dynamicDataOfSeq.get_int(null, 1);
 
        //Set the sequence on the complex member
        complexDynamicData.set_complex_member(null, 1, dynamicDataOfSeq);
        
        //Get the sequence from the complex member via a previously unused DynamicData Object
	// note that the object has not been configured with a type!
        complexDynamicData.bind_complex_member(scratchDynamicData, null, 1);
        count = scratchDynamicData.get_member_count(); // 2
        val = scratchDynamicData.get_int(null, 1);     // 10
        val = scratchDynamicData.get_int(null, 2);     // 20
        complexDynamicData.unbind_complex_member(scratchDynamicData);

        // Drop the item in slot 1, in favor of slot 2
        TypeCode tc = complexDynamicData.get_member_type(null, 1);
        try {
            if (tc.kind() == TCKind.TK_SEQUENCE) {
                if (tc.content_type() == TypeCode.TC_LONG) {
                    IntSeq intSeq = new IntSeq();
                    complexDynamicData.get_int_seq(intSeq, null, 1);
                    count = intSeq.size(); // 2
		    // note these are now 0-based indices
                    intSeq.set(0, intSeq.get(1));     // 20
                    intSeq.remove(1);
                    complexDynamicData.set_int_seq(null, 1, intSeq);
                }
            }
        } catch (BadKind bk) {
            System.out.println("BadKind");
        }
        
        complexDynamicData.bind_complex_member(scratchDynamicData, null, 1);
        count = scratchDynamicData.get_member_count(); // 1
        val = scratchDynamicData.get_int(null, 1);
        try {
            val = scratchDynamicData.get_int(null, 2);
        } catch (Exception ee) {
            val = -1;
        }  
        complexDynamicData.unbind_complex_member(scratchDynamicData);
    }
}

To delete everything in the sequence, I would simply

    IntSeq intSeq = new IntSeq(); 
    complexDynamicData.set_int_seq(null, 1, intSeq);

Hiya,

Yeah, the insight is "you need to write the entire app framework to wrap the 10 or so lines of code that demonstrates the algorithm that you want to show" :)

I've got the framework which I assembled over the holidays (family notwithstanding), I will get the rest of it this week (no more family around).

So the below is the "quick" algorithm.  It assumes that there is an element in the sequence that you want to drop.  The copy uses the fast get_/set_ complex member methods to copy the existing into a buffer, and then copy the buffer into the target sequence.  It correctly handles 

01(2)3 -> 013
(0)13 -> 13
1(3) -> 1
(1)3 -> 3
(3) -> 

but fails on

1234 -> 1234

go figure.  It's something internal:  DDS_DynamicData_unbind_complex_member:internal error 1 trying to to stream

If I know that the element I want to delete is not in the current sequence (either in the sequence or within a child of the sequence) I would do the get_/set_ at the higher level, ie get(seq), set(seq) rather than for_elements_in_seq { get(ele), set(ele) } and hopefully bypass that problem.

    System.out.print("is a TK_SEQUENCE<");
    TypeCode outertc = src.get_member_type(null, memberId);
    TypeCode tc = outertc.content_type();

    // do we want to remove one of these elements?
    boolean check = (memberName.compareTo(field) == 0);

    // src is the container holder, so get the container
    DynamicData srcSeq = new DynamicData(null, DynamicData.PROPERTY_DEFAULT);
    src.bind_complex_member(srcSeq, memberName, DynamicData.MEMBER_ID_UNSPECIFIED);
    DynamicDataInfo eInfo = new DynamicDataInfo();
    srcSeq.get_info(eInfo);
    int count = eInfo.member_count;
    System.out.print(tc.name() + ", " + count + "> \n");
    // Element (innards) ndx (0) memberId (1) is a TK_SEQUENCE<InnerStruct, 4> 

    // tgt is the container holder, so get the container
    DynamicData tgtSeq = new DynamicData(null, DynamicData.PROPERTY_DEFAULT);
    tgt.bind_complex_member(tgtSeq, memberName, DynamicData.MEMBER_ID_UNSPECIFIED);
                
    DynamicData dde = new DynamicData(tc, DynamicData.PROPERTY_DEFAULT);
    int p = 0; // memberId base 1 so will ++p on use
    for (int elm = 1; elm <= count; elm++) 
    {
        if (check && (elm == (toDrop + 1))) 
        { 
            System.out.println("elm | " + elm + " (dropped)");
            continue; 
        }
        srcSeq.get_complex_member(dde, null, elm);
        tgtSeq.set_complex_member(null, ++p, dde);
        System.out.println("elm > " + elm + " into " + p + " (ok)");
    }
    src.unbind_complex_member(srcSeq);
    tgt.unbind_complex_member(tgtSeq);

The framework recursively enters a function which decides what to do based on member_info kind and TCKind, until it hits a sequence or an array.  This is the IDL I used:

struct InnerStruct {
    string<8> aString;
    long aLong;
    sequence<long, 8> aLongSeq;
};

struct MiddleStruct {
    sequence<InnerStruct, 8> innards;
};

struct Outer {
    long primitive;
    MiddleStruct thorax;
    InnerStruct anArray[8];
};

The framework is not fully recursive -- it does not try to descend into sequence or array elements.  It will recurse into structs and copy primitives and non-sequence/array elements, recursively, but once it hits a sequence or array that's as far as it will go on that branch (instance.thorax.innards are processed as atomic elements). 

Obligatory trailer:  DynamicData is going to change with a future release (in line with the XTypes standard), the above is using the 5.1.0 and prior API.

Hi John,

Yeah, sounds like a lot of proprietary stuff, alas.   I would have to recreate your entire application internals, including recursive descent, and ...

On the other hand, I'm available for contract work at the moment.  I live in Edinburgh, how close is that to you :D .

clear_all_members will "Clear the contents of all data members of this object, including key members." 

What I'm wondering is -- given the way DynamicData is structured, a sequence is internally:

[A] DynamicData Object (TK_SEQUENCE)
[B] -> DynamicData Object (the sequence values, as an anonymous construct)
[C] -> -> DynamicData Object (a value in the sequence, a TK_STRUCT)

(read that as given [A], you bind a DynamidData object to the TK_SEQUENCE, and now you have a sequence.  You bind a DynamicData object then to each of the values in the sequence, one at a time)

I believe clear_all_members will work only on [C], ie it will clear a TK_STRUCT's member values to 0, 0.0, '', NULL, but I'm unsure what it will do with [B], and I'm guessing it will have no effect at all on [A].  You could call clear_all_members() on the TK_SEQUENCE's parent, but that would blow away all the values from the other fields in that struct.

I might be able to help you optimize the XML handling, though.  Does the java tree class have an "export XML" function? or are you using a W3C DOM object? There are tools that can generate java classes that will handle either the DOM or the raw XML (by converting it into a DOM), process it to remove the unwanted nodes, then hand it back (as DOM or as new serialized XML).  The one I'm thinking of is MapForce (from Altova, they also do XMLSpy). 

but yeah, maybe we've beaten this one to death.

rip