13.4. What’s New in 7.3.0 LTS

Observability Framework is an experimental product included with Connext 7.3.0 LTS, a long-term support release that is built upon and combines all of the features in releases 7.1.0 and 7.2.0 (see Previous Releases). See the Connext Releases page on the RTI website for more information on RTI’s software release model.

Note

For what’s new in Monitoring Library 2.0, see the Connext Core Libraries Release Notes.

13.4.1. Enhanced control of entities distributed across various Collector Service instances

Observability Collector Service now adds a new label, controllability_url, to the presence metrics of the applications that it monitors; for example, controllability_url: 'https://localhost:19098'. This label contains the Host and Port that should be used to send commands to the applications and their resources using the Collector Service REST API (see the Collector Service REST API Reference). The label takes into account whether HTTPS is enabled in the Collector Service configuration.

The new label is useful when using multiple Collector Service instances to monitor the different applications. It allows you to send commands to the applications and their resources using the REST API without knowing the Collector Service instance that is monitoring the application.

For pre-packaged installations, the Host and Port information of the Collector Service can be configured using the following parameters in the configuration JSON file:

collectorConfig.controlPublicHostname
collectorConfig.controlPublicPort

For more information, see Configure the JSON File.

For standalone deployments, the Host and Port information can be configured using two new environment variables in the Collector Service Docker image:

OBSERVABILITY_CONTROL_PUBLIC_HOSTNAME
OBSERVABILITY_CONTROL_PUBLIC_PORT

For more information, see RTI’s Docker Collector Service Repository.

13.4.2. New REST API in Collector Service to control telemetry data collection and distribution

This release introduces a REST API in Collector Service that allows controlling the telemetry data collection and distribution at run-time. The REST API includes commands to:

get the current logging collection and forwarding verbosity levels for applications
dynamically change the logging collection and forwarding verbosity levels for applications
get the current metric collection configuration for observable resources
dynamically configure the set of metrics collected and forwarded for observable resources

For more details on how to use these endpoints, see Collector Service REST API Reference.

13.4.3. Support for more flexible Observability Framework deployments

The Collector Service Docker image exposes additional environment variables and configuration parameters that allow more flexible deployments of the Observability Framework components.

When running the Observability Framework components standalone, you can now specify the following environment variables:

Loki hostname to send data to a Loki server on a remote host
OTel hostname to send data to an OTel collector on a remote host
RWT hostname to specify the Collector Service public address when using Real-Time WAN Transport
Collector Service control public hostname and port to specify the public access to the Collector Service control server when it is deployed behind a NAT or load balancer

Additionally, you can provide the security artifacts independently for all HTTP servers and clients created by Collector Service. For details on the Collector Service standalone deployment, see the Docker (Separate Deployment) section.

When running the prepackaged Docker Compose installation included in the Observability Framework host package, you can now specify the RWT hostname and Collector Service control public hostname and port environment variables. The prepackaged installation deploys the Loki and OTel collectors on the same node, so there is no need to specify these as remote hosts. For details on the prepackaged installation, see the Docker Compose (Prepackaged) section.

13.4.4. Control which metrics are collected

Previously, every observable resource (DomainParticipant, Publisher, DataReader, etc.) was, by default, subscribed to all available metrics for that resource when it was registered.

Starting in release 7.3.0, all metric collection is disabled by default. You can control which metrics are collected using one or both of the following methods:

Configure the initial set of metrics a resource is subscribed to using the Monitoring QosPolicy. For details and an example, see Setting the Initial Metrics and Log Configuration.
Dynamically configure metric collection via Observability Dashboards during run time. For details and an example, see Change the Metric Configuration.

13.4.5. New Syslog facilities provide expanded log management

Previously, Observability Framework only managed log messages produced by the Connext Core and API libraries, in accordance with the Syslog Protocol facility 23 (middleware). The framework retrieved and set the collection and forwarding verbosities for this facility.

In this release, these capabilities have been implemented for three additional facilities: 10 (security_event), 22 (service), and 1 (user).

The security_event facility applies to messages generated by RTI Connext Security Plugins that are defined as security events by the DDS Secure standard.
The service facility applies to messages generated by Connext Infrastructure Services: Routing Service, Recording Service, Cloud Discovery Service, and Web Integration Service.
The user facility applies to messages generated by the User logging API. For details, see Logs.

13.4.6. New logging category and plugin class labels enable more precise third-party backend queries

Starting in release 7.3.0, every log message pushed by Observability Framework to a logging third-party backend (for example, Grafana Loki) contains a new label representing the logging category the message belongs to. If the logging category is not available for a message , its value will be N/A.

Each message belonging to the security_event facility also includes a label indicating the standard plugin class name that originated the message, as defined in the OMG ‘DDS Security’ specification. For example, DDS:Auth:PKI-DH, DDS:Access:Permissions, DDS:Crypto:AES-GCM-GMAC, or DDS:Logging:DDS_LogTopic. The logging category is just a friendly name for these standard plugin class names.

The logging categories and plugin classes (when available) are also displayed in the provided Grafana Dashboards.

For details about the new labels, see Log Labels.

13.4.7. Updated dashboards support enhanced logging and dynamic metric control

Observability Dashboards have been updated to provide a cleaner interface and support new Observability Framework features. These updates include:

Dynamic metric control
Enhanced logging capabilities
- Visualization of new new log metadata
  - facilities (USER, SERVICE, SECURITY_EVENT)
  - categories
  - plugin classes
- Ability to dynamically control log generation and forwarding for all Syslog facilities
GUI improvements
- Entity history chart
- Larger metric charts
- Updated queries
- Alerting for enabled metrics

For more information, see Observability Dashboards.

13.4.8. Name change for some observability metrics

This release changes the following metric name prefixes associated with Connext entities. For details about all available metrics, see Metrics.

Old Metric Name Prefix	New Metric Name Prefix
`dds_participant_*`	`dds_domain_particpant_*`
`dds_datareader_*`	`dds_data_reader_*`
`dds_datawriter_*`	`dds_data_writer_*`

13.4.9. Third-party software upgrades

13.4.9.1. Observability Collector Service

The following third-party software used by Observability Collector Service has been upgraded:

Table 13.2 Third-Party Software Upgrades (Collector Service)
Third-Party Software	Previous Version	Current Version
OpenTelemetry C++	1.9.1	1.13.0
OpenSSL	3.0.9	3.0.12

13.4.9.2. Docker containers for Observability Collector Service

The following third-party software used by the Docker containers created by Observability Framework has been upgraded:

Table 13.3 Third-Party Software Upgrades (Docker Containers)
Third-Party Software	Previous Version	Current Version
Prometheus	2.37.8	2.45.1
Grafana	9.5.3	10.1.4
Grafana Loki	2.8.2	2.8.5
OpenTelemetry Collector Contrib	0.80.0	0.91.0

For information on third-party software used by Connext products, see the “3rdPartySoftware” documents in your installation: <NDDSHOME>/doc/manuals/connext_dds_professional/release_notes_3rdparty.

Warning

All third-party software is subject to third-party license terms and conditions. IT IS YOUR RESPONSIBILITY TO ENSURE THAT YOUR USE OF THIRD-PARTY SOFTWARE COMPLIES WITH THE CORRESPONDING THIRD-PARTY LICENSE TERMS AND CONDITIONS.