Open topic with navigation

7.3.14 Managing Instances (Working with Keyed Data Types)

This section applies only to data types that use keys; see 2.4 DDS Samples, Instances, and Keys. Using the following operations for non-keyed types has no effect. This section describes how instances work on DataWriters. See also 6. Working with Instances.

Topics come in two flavors: those whose associated data type has specified some fields as defining the ‘key,’ and those whose associated data type has not. An example of a data-type that specifies key fields is shown in Figure 7.14: Data Type with a Key.

Figure 7.14: Data Type with a Key

typedef struct Flight {
    @key long   flightId;
    string departureAirport;
    string arrivalAirport;
    Time_t departureTime;
    Time_t estimatedArrivalTime;
    Location_t currentPosition;
};

7.3.14.1 Writing Instances

If the data type has some fields that act as a ‘key,’ the Topic contains one or more instances whose values can be independently maintained. In Figure 7.14: Data Type with a Key, the flightId is the ‘key’. Different flights will have different values for the key. Each flight is an instance of the Topic. Each write() (or write() variation such as write_w_timestamp()) will update the information about a single flight—meaning that when a DataWriter calls write(), the DataWriter is updating the instance represented by the flightId.

When a DataWriter updates an instance by calling write(), a sample of that instance is sent to matching DataReaders, and the DataReaders consider the instance to be ALIVE.

If the DataWriter’s 7.5.22 RESOURCE_LIMITS QosPolicy specifies a max_instances limit that is not infinite, the limit will apply when writing. If a DataWriter writes an instance that it has not written before, and it has already reached the max_instances limit, it may try to reclaim the memory used by an existing instance. The rules for which instances it can replace are described in 7.3.14.7 Instance Memory Management.

If the DataWriter cannot reclaim the memory used by an existing instance, the write() call will fail. For more information on the behavior of the write() call when the max_instances limit is hit, see 7.3.8.2 write() behavior with KEEP_LAST and KEEP_ALL.

7.3.14.2 Registering Instances

If your data type has a key, you may improve performance of any operation that modifies the instance, such as any variation of write() or dispose(), by providing a non-NIL instance handle. An instance handle contains the pre-calculated instance keyhash so that it does not need to be calculated again. The instance handle for an instance can be retrieved once the instance is registered.

A DataWriter can register and retrieve an instance handle for an instance in two ways:

Once you have an instance handle, you can use it while writing to avoid calculating the instance keyhash in every write call. This performance improvement may be significant if your data is relatively small or your key fields are relatively complex. (If your data itself is large or complex, the time to calculate the keyhash may be insignificant relative to the time to serialize your data.)

You can register any number of instances up to the maximum number of instances configured in the DataWriter’s 7.5.22 RESOURCE_LIMITS QosPolicy. Explicit instance registration is completely optional. Note that registration through the register_instance() call only affects the DataWriter: matching DataReaders are not notified that the instance is ALIVE when the DataWriter registers the instance. An instance is only recognized as ALIVE when a DataWriter writes the instance. When an application registers instances and uses the instance handles for increased performance, it must keep a mapping between instance handles and instances. See the Warning below.

Figure 7.15: Explicitly Registering an Instance

Flight myFlight;
// writer is a previously-created FlightDataWriter
myFlight.flightId = 265;
DDS_InstanceHandle_t fl265Handle =
writer->register_instance(myFlight);
...
// Each time we update the flight, we can pass the handle
myFlight.departureAirport     = “SJC”;
myFlight.arrivalAirport       = “LAX”;
myFlight.departureTime        = {120000, 0};
myFlight.estimatedArrivalTime = {130200, 0};
myFlight.currentPosition      = { {37, 20}, {121, 53} };
if (writer->write(myFlight, fl265Handle) != DDS_RETCODE_OK) {
// ... handle error
}
// The writer can declare that it will no longer update information about
// this flight by unregistering itself from the instance

if (writer->unregister_instance(myFlight, fl265Handle) !=
DDS_RETCODE_OK) {
// ... handle error
}

Warning: If you decide to manage instance handles using your own application logic, make sure you keep a correct mapping between the instance and instance handle. If you pass the wrong instance handle when writing data, Connext DDS will assume that you are writing the instance associated with the handle. It does not check that the key fields match that handle, because that would negate the performance improvement from passing the handle. Passing the wrong instance handle can lead to strange behavior, because Connext DDS will treat your data sample as though it belongs to the wrong instance.

For example, if you have the History QosPolicy kind set to KEEP_LAST and depth set to 1 on the DataReader, Connext DDS should keep the last sample for each instance. But if you pass the wrong instance handle when writing, the DataReader will overwrite the wrong sample (in the wrong instance). As a result, a DataReader will not get updates for the instance it expects. An incorrect instance handle will affect all QoS policies that are applied per instance; see 6.3.1 QoS Policies that are Applied per Instance.

When you are done using an instance, you can unregister it. See 7.3.14.4 Unregistering Instances.

7.3.14.3 Disposing Instances

The dispose() operation informs DataReaders that, as far as the DataWriter knows, the instance no longer exists and can be considered “not alive.” When the dispose() operation is called, the instance state stored in the DDS_SampleInfo structure, accessed through DataReaders, will change to NOT_ALIVE_DISPOSED for that particular instance.

Often, systems use the NOT_ALIVE_DISPOSED state to indicate that some object is completely gone from the system. For example, in a flight tracking system, when a flight lands, a DataWriter may dispose of the instance corresponding to the flight. In that case, all DataReaders who are monitoring the flight will see the instance state change to NOT_ALIVE_DISPOSED, indicating that the flight has landed.

By default, instances are automatically disposed of when they are unregistered. This behavior is controlled by the autodispose_unregistered_instances field in the 7.5.31 WRITER_DATA_LIFECYCLE QoS Policy. This detail becomes important during DataWriter deletion because the DataWriter unregisters, and therefore disposes, all of its instances. There are a number of known issues with automatic disposal of instances during DataWriter deletion. For example, dispose messages can be lost because the DataWriter does not wait for the dispose messages to be acknowledged before continuing to shutdown. This can lead to scenarios where a system behaves inconsistently because some DataReaders see that an instance has been disposed, and others never receive that message. It is therefore typically recommended to set this QoS setting to false and manage instance state transitions explicitly in your application.

Note: If a DataWriter calls dispose(), it does not give up ownership of the instance (unlike when it calls unregister_instance(), in which case it is declaring that it will no longer have any updates for the instance and therefore does give up ownership of the instance to other DataWriters that may still be actively updating the instance).

Attention: Disposing does not free up memory by default. For instance, when the DataWriter calls dispose() to indicate that a flight has landed, it must keep the dispose message in its queue so all matching DataReaders get notified that the flight has landed (i.e., has been disposed). Also, in terms of memory management, Connext DDS may reclaim unregistered instances before disposed ones, or not reclaim disposed instances at all, depending on your QoS settings. See 7.3.14.7 Instance Memory Management.

See also:

7.3.14.4 Unregistering Instances

The unregister_instance() operation informs DataReaders that the DataWriter is no longer updating the instance. When a DataWriter will no longer update an instance, you can unregister it. To unregister a DataWriter from an instance, use the DataWriter’s unregister_instance() operation. Unregistering tells Connext DDS that the DataWriter has no more information on this instance; thus, it does not intend to modify that instance anymore, allowing Connext DDS to recover any resources it allocated for the instance.

By default, when the DataWriter unregisters from an instance, it also first disposes the instance; see 7.3.14.3 Disposing Instances. The autodispose_unregistered_instances field in the 7.5.31 WRITER_DATA_LIFECYCLE QoS Policy controls whether instances are automatically disposed of when they are unregistered. (See 7.5.31.2 Use Cases for Unregistering without Disposing.) When this QoS is true and the DataWriter unregisters from an instance, two samples are sent to the DataReader to notify it that the instance is both unregistered and disposed. The rules about which instance memory can be reclaimed are documented in 7.3.14.7 Instance Memory Management.

unregister_instance() should only be used on instances that have been previously registered. Instances can be registered explicitly with the register_instance() operation, or implicitly with any variation of the write() or dispose() operations. See Figure 7.15: Explicitly Registering an Instance.

Once all DataWriters have unregistered from an instance, the matched DataReaders will eventually get an indication that the instance no longer has any DataWriters. This is communicated to the subscribing application by means of the DDS_SampleInfo that accompanies each DDS sample (see 8.4.6 The SampleInfo Structure). Once there are no DataWriters for the instance, the DataReader will see the value of DDS_InstanceStateKind for that instance to be NOT_ALIVE_NO_WRITERS.

Note that DataReaders can’t distinguish between a scenario where all DataWriters explicitly unregister from an instance and a scenario where all DataWriters have lost liveliness. For more information on DataWriter liveliness, see the 7.5.15 LIVELINESS QosPolicy.

The unregister_instance() operation may affect the ownership of the instance (see the 7.5.17 OWNERSHIP QosPolicy). If the DataWriter was the exclusive owner of the instance, then calling unregister_instance() relinquishes that ownership, and another DataWriter can become the exclusive owner of the instance. (In contrast, if a DataWriter calls dispose(), it does not give up ownership of the instance.)

The unregister_instance() operation indicates only that a particular DataWriter no longer has any information/data on an instance and thus no longer has anything to say about the instance. It does not indicate that anything about the instance itself has changed, such as its existence or the associated data. For example, a DataWriter that is tracking a flight may unregister from an instance when the flight goes out of range—this does not mean that the position of the flight has changed or that the flight has landed, just that the DataWriter no longer has any knowledge of the flight; other DataWriters may still update the flight’s position.

Note that this is different from the dispose() operation discussed in the previous section, which informs DataReaders that the instance is no longer “alive.” The state of an instance is stored in the DDS_SampleInfo structure that accompanies each DDS sample of data that is received by a DataReader. User code can access the instance state to see if an instance is “alive”—meaning there is at least one DataWriter that is publishing DDS samples for the instance; see 6.1 Instance States.

The unregister_instance() operation adds one sample (or two) to the DataWriter queue, so the behavior of unregister_instance() with regards to KEEP_LAST and KEEP_ALL is the same as for the write() operation. See 7.3.8.2 write() behavior with KEEP_LAST and KEEP_ALL. (Two samples are added if autodispose_unregistered_instances is set to TRUE; Connext DDS makes a dispose and an unregister sample. See autodispose_unregistered_instances in the 7.5.31 WRITER_DATA_LIFECYCLE QoS Policy.)

See also:

7.3.14.5 Looking up an Instance Handle

Some operations, such as write(), accept an instance_handle parameter. If you need to get such a handle, you can call the FooDataWriter’s lookup_instance() operation, which takes an instance as a parameter and returns a handle to that instance. This is useful only for keyed data types. 

DDS_InstanceHandle_t lookup_instance (const Foo & key_holder)

The instance must have already been registered, written, or disposed. If the instance is not known to the DataWriter, this operation returns DDS_HANDLE_NIL.

7.3.14.6 Getting the Key Value for an Instance

Once you have an instance handle (using register_instance() or lookup_instance()), you can use the DataWriter’s get_key_value() operation to retrieve the value of the key of the corresponding instance. The key fields of the data structure passed into get_key_value() will be filled out with the original values used to generate the instance handle. The key fields are defined when the data type is defined (see 2.4 DDS Samples, Instances, and Keys).

Following our example in Figure 7.15: Explicitly Registering an Instance, register_instance() returns a DDS_InstanceHandle_t that can be used in the call to the FlightDataWriter’s get_key_value() operation. The value of the key is returned in a structure of type Flight with the flightId field filled in with the integer 265.

See also: 7.5.5.5 Propagating Serialized Keys with Disposed-Instance Notifications.

7.3.14.7 Instance Memory Management

In Connext DDS, memory is primarily pre-allocated when creating entities. When data is keyed, the memory associated with each instance used for storing instance-specific metadata is allocated when the DataWriter is created. Memory is not freed at runtime, unless you delete an entity. Instead, memory is made available to be reused by the DataWriter, or 'reclaimed'.

Instance memory in the DataWriter is reclaimed two ways:

In the default case, Connext DDS has to decide which instances to replace first. This is controlled by the following QoS policies and settings.

7.3.14.7.1 DataWriterResourceLimits: replace_empty_instances

The replace_empty_instances field in the 7.5.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension) defines whether instances with no samples in the DataWriter queue be replaced first, regardless of their instance state. If there are multiple empty instances, replace_empty_instances will replace unregistered instances, then disposed instances, then alive instances. If replace_empty_instances is true, empty instances will always be replaced first before any instance that may qualify for replacement based on the instance_replacement field in the 7.5.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension).

Values: true/false

7.3.14.7.2 DataWriterResourceLimits: instance_replacement

This instance_replacement field in the 7.5.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension) defines which instance states can be replaced, and the order in which they are allowed to be replaced. This setting takes effect if all samples for an instance are fully acknowledged.

Values:

Warning: Unregistered instances are always replaced first even if you don't choose DDS_UNREGISTERED_INSTANCE_REPLACEMENT. Because unregistering an instance indicates that the DataWriter will no longer update the instance, it is assumed that reclaiming these resources first will avoid information loss in your system.

When a DataWriter disposes an instance, it cannot replace the memory related to that instance unless autopurge_disposed_instances_delay is finite or the instance_replacement field in the 7.5.6 DATA_WRITER_RESOURCE_LIMITS QosPolicy (DDS Extension) indicates that disposed instances can be replaced when instance resource limits are reached.

See also:

7.3.14.8 Consequences of Unpurged Dispose Messages

There are consequences of having many unpurged dispose messages in the DataWriter’s queue. If the DataWriter’s 7.5.9 DURABILITY QosPolicy kind is not VOLATILE, those dispose messages will be delivered to late-joining DataReaders, which may cause an unexpected spike in network traffic. In addition, the DataReaders will not notify the application about those previously-disposed instances, because by default DataReaders will not propagate dispose messages for instances that were previously unknown. (This behavior can be changed by using the propagate_dispose_of_unregistered_instances QoS setting on the DataReader.)

Failing to purge disposed instances will cause similar behavior when using TopicQueries. When the DataWriter sends the response to the TopicQuery, it will include the unpurged dispose messages, causing high network traffic. In general, all dispose and unregister messages always pass filters (associated with ContentFilteredTopics, TopicQueries, or QueryConditions). This means that even if a TopicQuery’s filter expression only specifies a specific key value or set of key values, all dispose messages for all instances in the DataWriter queue will be sent in response to the TopicQuery. To avoid this when using TopicQueries configured with HISTORY_SNAPSHOT TopicQuerySelectionKind, use the special statement at the beginning of the query: “@instance_state = ALIVE AND” followed by the rest of the expression. This prevents the DataWriter from sending not-alive samples.

See also:

7.3.14.9 Consequences of DataWriters Reclaiming Disposed Instances

If your network is subject to disconnections, and disposed instances are purged, it’s possible that a dispose message is not received by every DataReader, leading to DataReaders recognizing different instance states. This happens if your network disconnection is long enough for a reliable DataReader to be marked as inactive and the disposed instance is purged during the disconnection. If the disposed message is not purged during the disconnection, it is still possible for the dispose message to be delivered after reconnection if the 7.5.9 DURABILITY QosPolicy is not VOLATILE.

If you have one or more RTI Routing Service applications in your network, leading to multiple places where instance state gets cached and might be reclaimed, it is even more likely that a dispose message might not be received by every DataReader.

© 2020 RTI

Open topic with navigation