Advanced data access using Connector (Javascript)
Introduction
RTI Connector offers a simplified API to Data Distribution Service. Different than the full DDS API, this lightweight API does not offer a one-to-one data mapping between the published or subscribed to datatype defined by the language (Javascript/Python). For example, if you use the C API, you can access a native struct that matches the type published and subscribed to (this is true unless you enable the new Flat Data feature), but with Connector you can either access individual fields (through getters and setters), or obtain a JSON representation of the sample that can be parsed back into a Javascript object (or Python dictionary).
A data writer typically works with the following methods:
clear_members(): resets all the properties of the published type to their default values
- setNumber(): sets one property as a number
- setBoolean(): sets one property as a boolean
- setString(): sets one property as a string
- set(): sets one property, automatically determining the type by looking at the parameter passed
- setFromJson()/set_dictionary(): sets one or more properties from a native (javascript object or Python dictionary) object
Because of the characteristics of the native types in the various languages supported by the Connector API, there are some cases that you need to be aware of before using the Connector API.
For example, in Javascript, numbers are all represented as double-precision floating-point. This internal format does not allow representing all the possible values of a 64-bit (signed or unsigned) integer without losing some information (if the 64-bit value requires more than 53 bits, it cannot be represented as a double-precision floating-point number).
This document will consider the various pitfalls and limitations of the Connector API related to reading and writing the sample published or subscribed to using the Javascript API.
Similar limitations might apply to other languages supported by Connector.
Simple Cases
The following table contains all IDL types supported by RTI Connext DDS that do not have issues with Javascript:
IDL Type | Example Set | Example Get |
boolean | setBoolean("myBoolean", true); | val=getBoolean(0, "myBool"); |
octet uint8 int8 uint16 int16 uint32 int32 | setNumber("a", 10); | val=getNumber(0, "a"); |
char | setString("code", "Z"); | val=getString(0, "code"); |
string | setString("name", "Joe"); | val=getString(0, "name"); val=getJSON(0).name |
For all those simple cases, the native DDS type maps naturally into a Boolean, Number or String.
64-bit Integers
In javascript, all the numbers are double-precision floating-points. Internally, javascripts use a double-precision IEE754 format to represent numbers (64-bit). An integer number larger than Number.MAX_SAFE_INTEGER (9,007,199,254,740,991) cannot be represented without data loss.
Connector API uses a dual mode to deal with 64-bit integer numbers by using a native Number (limited to Number.MAX_SAFE_INTEGER) as well as a String representation of the value (instead of a native value) that can cover any 64-bit integer.
For values smaller than Number.MAX_SAFE_INTEGER, you can use the same API as for the numbers:
output.setNumber("a", 123); |
A reader can simply access the data as a number:
let val = input.getNumber(0, "a"); // typeof(val) == "number" let val = input.getJson(0).a; // typeof(val) == "number" |
For larger numbers, the only way to set the value is through the setFromJson() and to provide the value as string:
output.setFromJson({ a: "18446744073709551615" }); |
Unfortunately, as of version 1.0.0 of Connector, the subscriber has no way to correctly read the same value since the reader always uses a Javascript number to represent the value (regardless of its value):
let val = input.getNumber(0, "a"); // typeof(val) == "number" console.log(a); // prints "18446744073709552000" let val = input.getJson(0).a; // typeof(val) == "number" console.log(a); // prints "18446744073709552000" |
Note that the value read (18446744073709552000) is different than the value published (18446744073709551615).
Single- and double-precision floating-point numbers
Single-precision floating-point numbers (32-bit float) are extended to 64-bit double-precision floating point in C before the values are transferred to the Javascript run-time. Single- and double-precision numbers are internally represented as IEEE-754. There are no known issues related to accessing float32 and float64 values, although you need to be aware that when you write a single-precision number (float32) from Javascript, you might lose some digits (as the significand of the number might not have enough bits to represent the value).
Setting and getting the value can be done through the numeric functions:
Double-precision numbers are naturally represented as Javascript Numbers. Similar to single-precision numbers, there are no known issues.
Special Values
The following values do not have a representation in JSON and must be represented through a String:
Infinity
-Infinity
-0
NaN (NaN is not supported)
Those values can still be set through the setNumber() function:
output.setNumber("myNumber", Infinity); |
and they can be assigned through JSON, but only when they are represented as Strings:
output.setFromJson({ myNumber:"Infinity"}); |
The following case will not work (although the passed object is correct):
output.setFromJson({ myNumber:Infinity}); |
this is because JSON cannot represent those values:
> JSON.stringify({a:Infinity, b:-Infinity, c:-0}) |
Different than the two Infinity values, the value negative zero can be set only using through JSON when expressed as a String:
output.setFromJson({myNumber: "-0"}); |
The following two cases will not work:
output.setFromJson({myNumber: -0}); |
To verify that those special values are read correctly, in the subscriber, use the following snippet to print the internal IEEE-754 native representation of the number:
const sample = input.samples.getJSON(i) // Will print: 0 (00000080) |
Note that Javascript will print "0" instead of "-0" when using template literals.
Quadruple-precision floating-point numbers
Quadruple-precision floating point numbers (or float128) are not supported on all the platforms, and have partial support on many platforms. Its native support is not only dependent on the processor used, but also on the OS.
Quadruple-precision floats are commonly referred also as "float128".
All the Intel architectures (i86Linux, x64Linux, i86Win, x64Win, x64Darwin), do not natively implement the IEEE-754 quadruple-precision floating point.
On the x86 architecture, most C compilers implement long double as the 80-bit extended precision type supported by x86 hardware (sometimes stored as 12 or 16 bytes to maintain data structure alignment), as specified in the C99/C11 standards (IEC 60559 floating-point arithmetic (Annex F)). An exception is Microsoft Visual C++ for x86, which makes long double a synonym for double. The Intel C++ compiler on Microsoft Windows supports extended precision, but requires the /Qlong‑double switch for long double to correspond to the hardware's extended precision format.
On Linux systems, the gcc compiler uses an extended 80-bit floating-point value to represent a long double (and uses a 16-byte storage area) and implements a GNU extension __float128 to use software-emulated support for IEEE754 128-bit floating-point numbers.
RTI Connext DDS supports quadruple-precision numbers through the type “DDS_LongDouble”. A DDS_LongDouble is internally an opaque buffer of 16 bytes that can be optionally wrapped into a native long double (by defining the macro RTI_CDR_SIZEOF_LONG_DOUBLE=16 at compile time). See this note in the RTI Connext DDS User's manual: here for the current release or here for release 6.0.1.
Since 128-bit floating-point numbers are not supported by javascript, Connector introduces a single way to get and set float128 values only on version newer than 1.0.0.
Setting a quadruple-precision floating-point field can only be achieved by using the setFromJson() function and specifing the value as a String of hex numbers representing the number in the machine-dependent format. The string must be prefixed by '0x' and followed by exactly 32 digits. For example:
output.setFromJson({ myLongDouble:"0x0000000000000080ffffa69234560000"}); |
Any attempt to set a quadruple-precision floating-point number in Connector version 1.0.0 and older produces an exception:
RTILuaJsonHelper_set_json_string:!DDS_TK_LONGDOUBLE not supported |
Because of the dependency on the internal data representation of a float128 number, the special values Infinity, -Infinity, and -0 can be set through setFromJson() only as a String of their hex number representation.
The following table shows the representation of some common numbers (the value '??' is not relevant).
Value | Native representation | Hex String |
-0 | 128-bit extended big endian | 0x80000000000000000000000000000000 |
128-bit extended little endian | 0x00000000000000000000000000000080 | |
80-bit extended little endian | 0x00000000000000800000???????????? | |
+Infinity | 128-bit extended big endian | 0x7fff8000000000000000000000000000 |
128-bit extended little endian | 0x0000000000000000000000000080ff7f | |
80-bit extended little endian | 0x0000000000000080ff7f???????????? | |
-Infinity | 128-bit extended big endian | 0xffff8000000000000000000000000000 |
128-bit extended little endian | 0x0000000000000000000000000080ffff | |
80-bit extended little endian | 0x0000000000000080ffff???????????? | |
PI | 128-bit extended big endian | 0x4000921f5b44421d8469898cc51701b8 |
128-bit extended liittle endian | 0xb80117c58c896984d14244b51f920040 | |
80-bit extended little endian | 0x35c26821a2da0fc90040???????????? |
Similarly, reading a float128 value can only be done using the getJson() function. The value associated with a float128 number is always going to be a String representing the hex bytes of the internal 16-byte buffer that is used to store the value. So, for example, reading the value of PI in a float128 will produce the following value:
const sample = input.getJson(i); // will print 0x0078931804560ec90040240300000000, |