annapurna: gerardo$  rtiddsgen -example x64Darwin10gcc4.2.1 -language C++ -namespace -replace SequenceWithRuntimeMaxLen.idl
Running rtiddsgen version 5.0.0, please wait ...
Done

The reason why you cannot change the maximum length at run-time can be seen by examining the  SequenceWithRuntimeMaxLen.h  On line 29 you can see there the declaration:

static const DDS_Long myAdjustableSeq_MAX_LEN = 4;

This is the variable that is used to size the maximum size of the sequence. Since this variable is declared static const in a header file it gets "compiled in" and cannot be changed at run-time.

To delay de setting of this variable all we need to do it replace that declaration in the header file with an external declaration as in:

// SequenceWithRuntimeMaxLen.h  line 29:
extern DDS_Long myAdjustableSeq_MAX_LEN;

Now the storage for the variable must be declared somewhere else. So we should add it to the SequenceWithRuntimeMaxLen.cxx

I edited SequenceWithRuntimeMaxLen.cxx  and added the declaration on line 40, right after the inclusion of the SequenceWithRuntimeMaxLen.h

// SequenceWithRuntimeMaxLen.c  lines 39-40:
#include "SequenceWithRuntimeMaxLen.h"
DDS_Long myAdjustableSeq_MAX_LEN = 4;

Now it is possible to set the myAdjustableSeq_MAX_LEN at run-time. You must do it before the call to SequenceWithRuntimeMaxLenTypeSupport::register_type() and you must to it consistently both in the publishing and subscribing sides.

I modified the generated examples SequenceWithRuntimeMaxLen_publication.cxx and SequenceWithRuntimeMaxLen_subscription.cxx  and set myAdjustableSeq_MAX_LEN to a larger value prior to calling SequenceWithRuntimeMaxLenTypeSupport::register_type()

// SequenceWithRuntimeMaxLen_publication.cxx lines 136-140

    /* Register type before creating topic */
    myAdjustableSeq_MAX_LEN = 10;

    type_name = SequenceWithRuntimeMaxLenTypeSupport::get_type_name();
    retcode = SequenceWithRuntimeMaxLenTypeSupport::register_type(
        participant, type_name);

// SequenceWithRuntimeMaxLen_publication.cxx lines 196-200

    /* Register the type before creating the topic */
    myAdjustableSeq_MAX_LEN = 10;

    type_name = SequenceWithRuntimeMaxLenTypeSupport::get_type_name();
    retcode = SequenceWithRuntimeMaxLenTypeSupport::register_type(
          participant, type_name);

To show it all works I modified the length of the sequence on the SequenceWithRuntimeMaxLen_publication.cxx to something greater than the original 4. 

// SequenceWithRuntimeMaxLen_publication.cxx lines 194-195

        /* Modify the data to be sent here */
        instance->myAdjustableSeq.length(8);

Now when I run the example applications I can see I get a sequence with a length of 8.

Output from the publisher:

annapurna: gerardo$ ./objs/x64Darwin10gcc4.2.1/SequenceWithRuntimeMaxLen_publisher 
Writing SequenceWithRuntimeMaxLen, count 0
Writing SequenceWithRuntimeMaxLen, count 1

Output from the subscriber:

annapurna: gerardo$ ./objs/x64Darwin10gcc4.2.1/SequenceWithRuntimeMaxLen_subscriber 

SequenceWithRuntimeMaxLen subscriber sleeping for 4 sec...
   key: 0
   myAdjustableSeq:
      myAdjustableSeq[0]: 0
      myAdjustableSeq[1]: 0
      myAdjustableSeq[2]: 0
      myAdjustableSeq[3]: 0
      myAdjustableSeq[4]: 0
      myAdjustableSeq[5]: 0
      myAdjustableSeq[6]: 0
      myAdjustableSeq[7]: 0
SequenceWithRuntimeMaxLen subscriber sleeping for 4 sec...
   key: 0
   myAdjustableSeq:
      myAdjustableSeq[0]: 0
      myAdjustableSeq[1]: 0
      myAdjustableSeq[2]: 0
      myAdjustableSeq[3]: 0
      myAdjustableSeq[4]: 0
      myAdjustableSeq[5]: 0
      myAdjustableSeq[6]: 0
      myAdjustableSeq[7]: 0

Approach 2: Modify the type-plugin and configure the DataWriter and DataReader to use dynamic memory allocation for the sequence.

This approach also modifies the type plugin code to do dynamic memory allocation for the sequence. It also takes advantage of a new feature introduced by RTI Connext DDS 5.0.0 that allows configuring the DataWriter cache to use dynamic memory for buffers that exceed a given size.

With this approach the maximum-length of the sequence becomes a property of each specific data-object and not a global property on the data-type.  The example can be found attached to the end of this posting.

To achieve this we need the following 5 steps:

STEP 2.1) Set the maximum length to some very large value, larger than it will be needed, but not so large that it overflows integers when computing the size on bytes for the arrays.

This does not waste space because with the changes we are making we never allocate anything to the maximum declared size. However there are checks in the serialize/deserialize functions that check that the length does not exceed this maximum so we need to set it to something large.

I used the value 1000000 so my test IDL looks like:

const long myAdjustableSeq_MAX_LEN=10000000;

struct SequenceWithRuntimeMaxLen {
  long key; //@Key
  sequence<long, myadjustableseq_max_len="" >="" myadjustableseq;="" };="">

STEP 2.2) Modify the type plugin to initialize the sequence with a zero maximum size, rather than the maximum:

This is done modifying the SequenceWithRuntimeMaxLen_initialize_ex() function found in SequenceWithRuntimeMaxLen.c

Replace:

        if (!DDS_LongSeq_set_maximum(&sample->myAdjustableSeq,
                ((myAdjustableSeq_MAX_LEN)))) {

            return RTI_FALSE;
        }

With:

        if (!DDS_LongSeq_set_maximum(&sample->myAdjustableSeq, 0)) {
            return RTI_FALSE;
        }

STEP 2.3) Modify the type plugin deserialization to set the size of the sequence to fit what it needs to accomodate the sample received.

This is done modifying the SequenceWithRuntimeMaxLenPlugin_deserialize_sample() function found in SequenceWithRuntimeMaxLenPlugin.cxx

Code added to SequenceWithRuntimeMaxLenPlugin.cxx line 454 after the declaration of

RTICdrUnsignedLong sequence_length;

// Peek into the stream to get the length of the sequence
RTICdrStream_deserializLong(stream,  &sequence_length);

// Rewind the stream 4 bytes to leave it as it was before we peeked
RTICdrStream_incrementCurrentPosition(stream, -4);

// Set the maximum length of the sequence to the actual length
DDS_LongSeq_set_maximum(&sample->myAdjustableSeq, sequence_length);

STEP 2.4) Modify the type plugin to set the maximum to zero again when is no longer needed. This releases the memory for the sequence elements

This is done modifying the SequenceWithRuntimeMaxLenPlugin_return_sample() which is a macro defined in SequenceWithRuntimeMaxLenPlugin.h  

I commented out that definition SequenceWithRuntimeMaxLenPlugin_return_sample in SequenceWithRuntimeMaxLenPlugin.h  line 53:

And added the declaration of this function in SequenceWithRuntimeMaxLenPlugin.c to first set the maximum length of the sequence to zero:

void SequenceWithRuntimeMaxLenPlugin_return_sample(
    PRESTypePluginEndpointData endpointData,
    void *sample, void *handle);
{
    SequenceWithRuntimeMaxLen * sampleData = sample;
    DDS_LongSeq_set_maximum(&sample->myAdjustableSeq, 0);
    PRESTypePluginDefaultEndpointData_returnSample(endpointData, sample, handle);
}

STEP 2.5) Configure the data-writer to use dynamic memory

By default a DataWriter allocates buffers to hold serialized samples of the maximum size. This can be wasteful for data-types where the average sample is significantly smaller that the largest possible sample.  Specifically if a data-type contains a sequences the DataWriter by default allocates buffers sized to hold samples where the sequence has the maximum declared size.

RTI Connext DDS version 5.0.0 introduced a new feature that can configure a DataWriter to use dynamic memory allocation for large samples.  With this feature the DataWriter allocates buffers to only hold a configured 'pool_buffer_max_size' and dynamically allocates the buffer if it needs to serialize a sample that does not fit on that size.  More details on this feature see Chapter 20 of The RTI Connext DDS Users's Guide http://community.rti.com/rti-doc/500/ndds.5.0.0/doc/pdf/RTI_CoreLibrariesAndUtilities_UsersManual.pdf

The configuration of the DataWriter to use dynamic allocation for large samples is done using the PropertyQoSPolicy in the DataWriterQoS. The property that configures it is called "dds.data_writer.history.memory_manager.fast_pool.pool_buffer_max_size"

I configured this by editing the USER_QOS_PROFILE.xml located in the working directory of the application. The  USER_QOS_PROFILE.xml  was generated by rtiddsgen and already contains a default configuration for the DataWriter QoS. I modified the configuration and added:

	        <property>
		    <value>
		        <element>
			  <name> dds.data_writer.history.memory_manager.fast_pool.pool_buffer_max_size</name>
			  <value>1000</value>
			</element>
		    </value>
		</property>

Directly inside the <datawriter_qos>  element.

VERIFICATION STEP) 

To show it all works I modified both the maximum-length and the length of the sequence on the SequenceWithRuntimeMaxLen_publication.cxx each time I publish a sample. Note that this does not really prove that the memory is not being allocated to the maximum size, but with the changes we made it should not happen.

// SequenceWithRuntimeMaxLen_publication.cxx lines 194-195

        /* Modify the data to be sent here */
	instance->myAdjustableSeq.maximum(1 + 3*count);
        instance->myAdjustableSeq.length(1 + 3*count);

Now when I run the example applications I can see I get a sequence with a varying length

Output from the publisher:

annapurna: gerardo$ ./objs/x64Darwin10gcc4.2.1/SequenceWithRuntimeMaxLen_publisher 
Writing SequenceWithRuntimeMaxLen, count 0
Writing SequenceWithRuntimeMaxLen, count 1

Output from the subscriber:

annapurna: gerardo$ ./objs/x64Darwin10gcc4.2.1/SequenceWithRuntimeMaxLen_subscriber 
SequenceWithRuntimeMaxLen subscriber sleeping for 4 sec...
SequenceWithRuntimeMaxLen subscriber sleeping for 4 sec...
   key: 0
   myAdjustableSeq:
      myAdjustableSeq[0]: 0
      myAdjustableSeq[1]: 0
      myAdjustableSeq[2]: 0
      myAdjustableSeq[3]: 0
SequenceWithRuntimeMaxLen subscriber sleeping for 4 sec...

   key: 0
   myAdjustableSeq:
      myAdjustableSeq[0]: 0
      myAdjustableSeq[1]: 0
      myAdjustableSeq[2]: 0
      myAdjustableSeq[3]: 0
      myAdjustableSeq[4]: 0
      myAdjustableSeq[5]: 0
      myAdjustableSeq[6]: 0
SequenceWithRuntimeMaxLen subscriber sleeping for 4 sec...
   key: 0
   myAdjustableSeq:
      myAdjustableSeq[0]: 0
      myAdjustableSeq[1]: 0
      myAdjustableSeq[2]: 0
      myAdjustableSeq[3]: 0
      myAdjustableSeq[4]: 0
      myAdjustableSeq[5]: 0
      myAdjustableSeq[6]: 0
      myAdjustableSeq[7]: 0
      myAdjustableSeq[8]: 0
      myAdjustableSeq[9]: 0