4.1. Usage¶

This section explains how to run Replay Service from a command line. In particular, it describes:

How to Start Replay Service (Section 4.1.1).
How to Stop Replay Service (Section 4.1.2).
Replay Service command-line parameters (Section 4.1.3).

4.1.1. Starting Replay Service¶

Replay Service runs as a separate application. The script to run the executable is in <NDDSHOME>/bin. (See Section 1.3 for the path to NDDSHOME.)

rtireplayservice [options]

To start Replay Service with a default configuration, enter:

$NDDSHOME/bin/rtireplayservice

This command will run Replay Service indefinitely until you stop it.

Replay Service is pre-loaded with a built-in configuration that has default settings.

Note

To run Replay Service on a target system (not your host development platform), you must first select the target architecture. To do so, either:

Set the environment variable CONNEXTDDS_ARCH to the name of the target architecture. (Do this for each command shell you will be using.)
Or set the variable connextdds_architecture in the file rticommon_config.[sh/bat] to the name of the target architecture. (The file is resource/scripts/rticommon_config.sh on Linux or macOS systems, resource/scripts/rticommon_config.bat on Windows systems.) If the CONNEXTDDS_ARCH environment variable is set, the architecture in this file will be ignored.

4.1.2. Stopping Replay Service¶

To stop Replay Service, press Ctrl-c. Replay Service will perform a clean shutdown.

4.1.3. Replay Service Command-Line Parameters¶

The following table describes all the command-line parameters available in Replay Service. To list the available parameters, run rtireplayservice -help.

All command-line parameters are optional; if specified, they override the values of any corresponding settings in the loaded XML configuration. See Section 4.3.1 for the XML elements that can be overridden with command-line parameters.

Table 4.1 Replay Service Command-Line Parameters¶
Parameter	Description
-appName <string>	Application name used to identify this execution for remote administration, and to name the Connext DDS participant.
-cfgFile <string>	Semicolon-separated list of configuration file paths. Default: Unspecified
-cfgName	Configuration name Used to find a `<replay_service>` matching tag in the configuration file.
-debugMode	Enables debug mode. Default: Debug mode is not enabled.
-domainIdBase <int>	This value is added to the domain IDs in the `<domain_participant>` tag in the configuration file. For example, if you set `-domainIdBase` to 50 and use domain IDs 0 and 1 in the configuration file, Replay Service will read domains 0 and 1 from the database, but will replay that data into domains 50 and 51. Default: 0
-D<name>=<value>	Defines a variable that can be used as an alternate replacement for XML environment variables, specified in the form $(VAR_NAME). Note that definitions in the environment take precedence over these definitions.
-help	Shows this help.
-heapSnapshotDir	Output directory where the heap monitoring snapshots are dumped. The filename format is: RTI_heap_<appName>_<processId>_<index>.log
-heapSnapshotPeriod <sec>	Period at which heap monitoring snapshots are dumped. Enables heap monitoring if > 0. Default: 0 (disabled)
-logFormat <format>	A mask to configure the format of the log messages for both Replay Service and Connext DDS. DEFAULT - Print message, method name, and activity context TIMESTAMPED - Print message, method name, activity context, and timestamp MINIMAL - Print only message number and method name MAXIMAL - Print all available fields Default: DEFAULT
-maxObjectsPerThread <int>	Maximum number of Thread-specific objects that can be created. Default: 1024.
-remoteAdministrationDomainId <int>	Enables remote administration and sets the domain ID for communication. Default: Remote administration is not enabled.
-remoteMonitoringDomainId <int>	Enables remote monitoring and sets the domain ID for status publication. Default: Remote monitoring is not enabled.
-verbosity <service_level>[:<dds_level>]	Controls what type of messages are logged. <service_level> is the verbosity level for the service logs and <dds_level> is the verbosity level for the DDS logs. Both can take any of the following values: `SILENT` `ERROR` `WARN` `LOCAL` `REMOTE` `ALL` Default: `ERROR:ERROR`
-version	Prints the program version and exits.

4.1.4. Replay Service Runtime Behavior¶

Replay Service currently does not delete DataWriters, even if all original DataWriters were deleted in the recorded database.

4.1.5. Working With Large Data¶

The built-in SQLite plugin implementation available in Replay Service is prepared to handle any type size and storage format. In order to improve the fidelity of the samples published with respect to the timestamps of the original samples in the database, Replay Service internally caches the next sample to be published, so that it can be accessed more quickly when it’s time to publish. This behavior is particularly useful when replaying large data.

However, when working with large data types and massive files (or filesets), Replay Service may take a while to prepare SQL statements to work with the data, resulting in a delay in publishing the first samples for any large data topics.

Because of this delay, it is recommended that you index the user data tables for those large topics before running Replay Service on them. Indexing can massively improve Replay Service’s startup time for those topics. You can create the indexes offline, after Recording Service has finished recording all the data. Index the tables on the SampleInfo_reception_timestamp field. For example, imagine a table, VeryLargeTopic@0, has been created by Recording Service; you can use the following index creation statement:

CREATE INDEX IF NOT EXISTS [VeryLargeTopic@0_idx_rt]
        ON [VeryLargeTopic@0] (SampleInfo_reception_timestamp)

4.1.6. Choosing the Sample Order for Replaying Data¶

Replay Service has the capability to replay data ordered by reception timestamp or by source timestamp. Reception timestamp represents a monotonic ascending time series and the source timestamp comes from every recorded participant’s system clock, hence it can behave in non-monotonic fashion. The source timestamp can differ between one record and another with the same information.

Furthermore, it is important to consider the DDS_DestinationOrderQosPolicy, which can create “eventual consistency” between the different Recording Service instances. For more details, please see DESTINATION_ORDER QosPolicy, in the RTI Connext DDS Core Libraries User’s Manual.

That being said, Replay Service’s built-in SQLite plugin implementation will sort the database by using the chosen sample_order before it replays the data.

4.1.7. Recreating the State of the World when Replaying (Replaying Instance History)¶

Replay Service has the ability to replay what can be called the state of the world given a starting timestamp. The state of the world is the latest value for every alive instance at a certain timestamp. That’s why we also refer to this feature as Instance History Replay. When this feature is enabled, Replay Service will read the latest value for every instance that was alive, and publish it with the first batch of samples to be published for a topic. Of course this feature relates to keyed types and topics, for unkeyed types and topics it has no effect.

As an example, imagine a keyed topic T1 was recorded, and that there are three instances for the topic, define by IDs 1, 2 and 3. The following table shows the instances and values recorded for it during a certain period of time:

Time	Instance ID	Value
1	ID=1	100
2	ID=2	200
3	ID=3	300
4	ID=1	110
5	ID=3	310
6	ID=1	disposed
7	ID=2	210
8	ID=3	320
9	ID=1	120
10	ID=2	220

If the Instance History Replay feature is enabled, Replay Service will publish the instance values that compose the state of the world at the start time provided by the user. For example, for start time T=4, Replay Service will publish, ahead of any normal replay activity, samples {ID=1,100}, {ID=2,200} and {ID=3,300}. It will then start publishing samples normally, {ID=1,110}, etc. If start time is T=7, then the state of the world will be composed of two samples, because the sample with ID=1 was disposed at time T=6. Hence, Replay Service will publish samples {ID=2,200} and {ID=3,310} as the state of the world. Of course, if the start time is not provided (T=0) then there is no history to replay.

An important note about how the instance history is replayed is that it’s published in a burst. This means that time separation between different values is not preserved. The goal of the feature is to publish a whole picture for the topic before starting with normal, time-preserving replay.

This feature can be useful in situations where very large databases are being replayed partially (this is, with a specific time range). When this is done, and this feature is not enabled, instances that were alive at the specified start time are just not present in the replay. By enabling this feature, Replay Service will publish a value for each instance that was alive at the start time provided, hence completing the whole picture for the topic before normal replay activity starts.

Under the hood, this feature uses a custom instance history index that is created by Recording Service (although the creation is disabled by default, as it can affect performance), or it can also be created offline. See Section 3.3.6.3 on how to enable this feature while recording, or Section 8 for how to index the database offline. If the index was not created while recording, or offline ahead of replaying, then Replay Service will create the index during startup, which can take some time in huge databases. So our general recommendation is to plan ahead whether this feature is going to be used when replaying your data and either use online indexing with Recording Service (by enabling the <instance_indexing tag) or save some time for the offline indexing of the database. It’s interesting to note that indexing, even when done by Replay Service, will only happen once, as it can be expected.

The searching for the instance history is quite optimized, but it takes some time. This delay can manifest during Replay Service startup, but no more searching will be needed after that during normal replay of samples. If if the looping feature is enabled, Replay Service will search for instance history again and re-publish it. This will restore the original state of the world ahead of replaying samples of the next loop.

This feature is integrated with replaying by source timestamp and other features, like jumping between timestamps. Take into account that if instance indexing was done, for example, based only for reception timestamp, but you want to Replay with instance history based on source timestamp, then Replay Service will proceed to indexing the database by source timestamp, incurring in the indexing delay. So planning ahead of time what sort of indexing and replay (source, timestamp) is going to be needed can also save time.

For more details on how to configure this feature, see Section 4.3.9.

4.1.8. Jumping in Time while Replaying¶

Replay Service has the capability of jumping in time. With this jump action, Replay Service will move the current replay position forward or backward in time.

For example, suppose you have a recording and in the middle there are some important events. During the replay, you can jump ahead to those events, skipping the unrelated events that came before.

When the jump in time is to the future, Replay Service will burst all the discovery samples between your previous position and your new position. This discovery phase can take some time depending on the amount of discovery between both positions. To improve the performance of the jump, you can enable Instance History Replay to obtain the discovery state of the world and create all the new StreamReaders. For more information, see Section 4.1.7.

In order to jump in time, you have to enable the remote administration. Remote administration exposes some commands that allow you to perform this action. See Section 4.4.3 for details.

4.1.9. Using Debug Mode while Replaying¶

Replay Service has a secondary mode to start in a debug way. This mode will allow you to add breakpoints in the replay. The replay will be paused once it hits a breakpoint and the user will have the capability to decide the behavior of the replay.

For example, suppose you have a recording and in the middle there are some important events. During the replay, you can replay that section of events adding breakpoints at the beginning and at the end of the replay section. Once, the replay hits the end breakpoint you will be able to jump to the first breakpoint and reproduce that important event again.

If the Debug mode feature is enabled, Replay Service will first hit an initial breakpoint, which is a permanent breakpoint labeled “default_breakpoint”. This initial breakpoint is set to the start timestamp of the recorded database.

In this mode, you can continue the replay until hit a breakpoint or continue the replay for a period of time and then stop the replay without hitting a real breakpoint.

Under the hood, Replay Service will create a virtual breakpoint at the end of that period of time. The virtual breakpoint will be removed internally after it was hit or we jump to another breakpoint. This period of time can be a custom time set by the user or a fixed time that we call slice period. The slice period can be set inside the XML configuration and it can’t be changed during the replay.

The debug mode use the remote administration system to receive the user’s order due to that you have to enable the remote administration to control this mode. See Section 4.4.3 for details.

4.2. Operating System Daemon¶

See generic instructions in How to Run as an Operating System Daemon.