13.4. What’s New in 7.3.0 LTS
Observability Framework is an experimental product included with Connext 7.3.0 LTS, a long-term support release that is built upon and combines all of the features in releases 7.1.0 and 7.2.0 (see Previous Releases). See the Connext Releases page on the RTI website for more information on RTI’s software release model.
Note
For what’s new in Monitoring Library 2.0, see the Connext Core Libraries Release Notes.
13.4.1. Enhanced control of entities distributed across various Collector Service instances
Observability Collector Service now adds a new label, controllability_url
,
to the presence metrics of the applications that it monitors; for example,
controllability_url: 'https://localhost:19098'
. This label contains the
Host and Port that should be used to send commands to the applications and
their resources using the Collector Service REST API
(see the Collector Service REST API Reference). The label takes
into account whether HTTPS is enabled in the Collector Service
configuration.
The new label is useful when using multiple Collector Service instances to monitor the different applications. It allows you to send commands to the applications and their resources using the REST API without knowing the Collector Service instance that is monitoring the application.
For pre-packaged installations, the Host and Port information of the Collector Service can be configured using the following parameters in the configuration JSON file:
collectorConfig.controlPublicHostname
collectorConfig.controlPublicPort
For more information, see Configure the JSON File.
For standalone deployments, the Host and Port information can be configured using two new environment variables in the Collector Service Docker image:
OBSERVABILITY_CONTROL_PUBLIC_HOSTNAME
OBSERVABILITY_CONTROL_PUBLIC_PORT
For more information, see RTI’s Docker Collector Service Repository.
13.4.2. New REST API in Collector Service to control telemetry data collection and distribution
This release introduces a REST API in Collector Service that allows controlling the telemetry data collection and distribution at run-time. The REST API includes commands to:
get the current logging collection and forwarding verbosity levels for applications
dynamically change the logging collection and forwarding verbosity levels for applications
get the current metric collection configuration for observable resources
dynamically configure the set of metrics collected and forwarded for observable resources
For more details on how to use these endpoints, see Collector Service REST API Reference.
13.4.3. Support for more flexible Observability Framework deployments
The Collector Service Docker image exposes additional environment variables and configuration parameters that allow more flexible deployments of the Observability Framework components.
When running the Observability Framework components standalone, you can now specify the following environment variables:
Loki hostname
to send data to a Loki server on a remote hostOTel hostname
to send data to an OTel collector on a remote hostRWT hostname
to specify the Collector Service public address when using Real-Time WAN TransportCollector Service control public hostname and port
to specify the public access to the Collector Service control server when it is deployed behind a NAT or load balancer
Additionally, you can provide the security artifacts independently for all HTTP servers and clients created by Collector Service. For details on the Collector Service standalone deployment, see the Docker (Separate Deployment) section.
When running the prepackaged Docker Compose installation included in the
Observability Framework host package, you can now specify the
RWT hostname
and Collector Service control public hostname and port
environment variables. The prepackaged installation deploys the Loki and
OTel collectors on the same node, so there is no need to specify these as
remote hosts. For details on the prepackaged installation, see the
Docker Compose (Prepackaged) section.
13.4.4. Control which metrics are collected
Previously, every observable resource (DomainParticipant, Publisher, DataReader, etc.) was, by default, subscribed to all available metrics for that resource when it was registered.
Starting in release 7.3.0, all metric collection is disabled by default. You can control which metrics are collected using one or both of the following methods:
Configure the initial set of metrics a resource is subscribed to using the Monitoring QosPolicy. For details and an example, see Setting the Initial Metrics and Log Configuration.
Dynamically configure metric collection via Observability Dashboards during run time. For details and an example, see Change the Metric Configuration.
13.4.5. New Syslog facilities provide expanded log management
Previously, Observability Framework only managed log messages produced
by the Connext Core and API libraries, in accordance with the Syslog
Protocol facility 23 (middleware)
. The framework retrieved and set
the collection and forwarding verbosities for this facility.
In this release, these capabilities have been implemented for three
additional facilities: 10 (security_event)
, 22 (service)
, and
1 (user)
.
The
security_event
facility applies to messages generated by RTI Connext Security Plugins that are defined as security events by the DDS Secure standard.The
service
facility applies to messages generated by Connext Infrastructure Services: Routing Service, Recording Service, Cloud Discovery Service, and Web Integration Service.The
user
facility applies to messages generated by the User logging API. For details, see Logs.
13.4.6. New logging category and plugin class labels enable more precise third-party backend queries
Starting in release 7.3.0, every log message pushed by Observability
Framework to a logging third-party backend (for example, Grafana Loki)
contains a new label representing the logging category the message
belongs to. If the logging category is not available for a message , its
value will be N/A
.
Each message belonging to the security_event
facility also includes
a label indicating the standard plugin class name that originated the
message, as defined in the OMG ‘DDS Security’
specification. For example,
DDS:Auth:PKI-DH
, DDS:Access:Permissions
,
DDS:Crypto:AES-GCM-GMAC
, or DDS:Logging:DDS_LogTopic
. The
logging category is just a friendly name for these standard plugin class
names.
The logging categories and plugin classes (when available) are also displayed in the provided Grafana Dashboards.
For details about the new labels, see Log Labels.
13.4.7. Updated dashboards support enhanced logging and dynamic metric control
Observability Dashboards have been updated to provide a cleaner interface and support new Observability Framework features. These updates include:
Dynamic metric control
Enhanced logging capabilities
Visualization of new new log metadata
facilities (USER, SERVICE, SECURITY_EVENT)
categories
plugin classes
Ability to dynamically control log generation and forwarding for all Syslog facilities
GUI improvements
Entity history chart
Larger metric charts
Updated queries
Alerting for enabled metrics
For more information, see Observability Dashboards.
13.4.8. Name change for some observability metrics
This release changes the following metric name prefixes associated with Connext entities. For details about all available metrics, see Metrics.
Old Metric Name Prefix |
New Metric Name Prefix |
---|---|
|
|
|
|
|
|
13.4.9. Third-party software upgrades
13.4.9.1. Observability Collector Service
The following third-party software used by Observability Collector Service has been upgraded:
Third-Party Software |
Previous Version |
Current Version |
---|---|---|
OpenTelemetry C++ |
1.9.1 |
1.13.0 |
OpenSSL |
3.0.9 |
3.0.12 |
13.4.9.2. Docker containers for Observability Collector Service
The following third-party software used by the Docker containers created by Observability Framework has been upgraded:
Third-Party Software |
Previous Version |
Current Version |
---|---|---|
Prometheus |
2.37.8 |
2.45.1 |
Grafana |
9.5.3 |
10.1.4 |
Grafana Loki |
2.8.2 |
2.8.5 |
OpenTelemetry Collector Contrib |
0.80.0 |
0.91.0 |
For information on third-party software used by Connext products, see the “3rdPartySoftware” documents in your installation: <NDDSHOME>/doc/manuals/connext_dds_professional/release_notes_3rdparty.
Warning
All third-party software is subject to third-party license terms and conditions. IT IS YOUR RESPONSIBILITY TO ENSURE THAT YOUR USE OF THIRD-PARTY SOFTWARE COMPLIES WITH THE CORRESPONDING THIRD-PARTY LICENSE TERMS AND CONDITIONS.