RTI Connext Observability Framework
7.3.0.2
Copyrights and Notices
1. What is Connext Observability Framework?
1.1. Telemetry Data
1.2. Distribution of Telemetry Data
1.3. Flexible Storage
1.4. Visualization of Telemetry Data
1.5. Control and Selection of Telemetry Data
1.6. Security
2. Components
2.1. Monitoring Library 2.0
2.2. Observability Collector Service
2.2.1. Storage Components
2.3. Observability Dashboards
3. Observability Framework Deployments
3.1. Current Release
3.1.1. Docker Compose (Prepackaged)
3.1.1.1. Collection, Storage, and Visualization Components
3.1.2. Docker (Separate Deployment)
3.2. Future releases
3.2.1. Collector Service
3.2.1.1. Executable
3.2.1.2. Collector Service Deployments
4. Security
4.1. Secure Communication between Connext Applications and Collector Service
4.1.1. Secure Communication between Connext Applications and Collector Service (Pre-Packaged Deployment)
4.1.2. Secure Communication between Connext Applications and Collector Service (Separate Deployment)
4.2. Secure Communication with Collector Service HTTP Servers
4.2.1. Secure Collector Service HTTP Servers (Pre-Packaged Deployment)
4.2.2. Secure Collector Service HTTP Servers (Separate Deployment)
4.3. Secure Communication with Third-Party Component HTTP Servers
4.3.1. Secure Third-Party Component HTTP Servers (Pre-Packaged Deployment)
4.3.2. Secure Third-Party Component HTTP Servers (Separate Deployment)
4.4. Generating the Observability Framework Security Artifacts
4.4.1. Generating DDS Security Artifacts
4.4.2. Generating HTTPS Security Artifacts
4.4.2.1. Preliminary Steps
4.4.2.2. Generating a New Root CA
4.4.2.3. Generating Server Certificates
4.4.2.4. BASIC-Auth Password File
5. Installing and Running Observability Framework
5.1. Installing the Host Package
5.1.1. Prerequisites
5.1.2. Install from RTI Launcher
5.1.3. Install from the Command Line
5.2. Configuring, Running, and Removing Observability Framework Components Using Docker Compose
5.2.1. Configuring the Docker Workspace for Observability Framework
5.2.1.1. Configure the JSON File
5.2.1.2. Run the Observability script to create the Observability workspace
5.2.2. Initialize and Run Docker Containers
5.2.3. Verify Docker Containers are Running
5.2.4. Configure Grafana
5.2.4.1. Initial Login
5.2.4.2. Configuration Options
5.2.5. Stop Docker Containers
5.2.6. Start Existing Docker Containers
5.2.7. Stop and Remove Docker Containers
5.2.8. Removing the Docker Workspace for Observability Framework
6. Getting Started Guide
6.1. About the Observability Example
6.1.1. Applications
6.1.2. Data Model
6.1.3. DDS Entity Mapping
6.1.4. Command-Line Parameters
6.1.4.1. Publishing Application
6.1.4.2. Subscribing Application
6.2. Before Running the Example
6.2.1. Set Up Environment Variables
6.2.2. Compile the Example
6.2.2.1. Non-Windows Systems
6.2.2.2. Windows Systems
6.2.3. Install Observability Framework
6.2.3.1. Configure Observability Framework for the Appropriate Operation Mode
6.2.4. Start the Collection, Storage, and Visualization Docker Containers
6.3. Running the Example
6.3.1. Start the Applications
6.3.2. Changing the Time Range in Dashboards
6.3.3. Simulate Sensor Failure
6.3.4. Simulate Slow Sensor Data Consumption
6.3.5. Simulate Time Synchronization Failures
6.3.6. Change the Application Logging Verbosity
6.3.7. Change the Metric Configuration
6.3.7.1. Resources used in this example
6.3.7.2. Changing metrics collected for a single DataWriter
6.3.7.3. Changing metrics collected for all DataWriters of an application
6.3.8. Close the Applications
7. Telemetry Data
7.1. Introduction
7.2. Resources
7.2.1. Resource Pattern Definitions
7.3. Metrics
7.3.1. Metric Pattern Definitions
7.3.2. Application Metrics
7.3.3. Participant Metrics
7.3.4. Topic Metrics
7.3.5. DataWriter Metrics
7.3.6. DataReader Metrics
7.3.7. Derived Metrics Generated by Prometheus Recording Rules
7.3.7.1. DDS Entity Proxy Metrics
7.3.7.2. Raw Error Metrics
7.3.7.3. Aggregated Error Metrics
7.3.7.4. Enable a Raw Error Metric
7.3.7.5. Custom Error Metrics
7.4. Logs
7.4.1. Syslog Levels and Facilities
7.4.2. Activity Context
7.4.3. Log Labels
7.4.4. Collection and Forwarding Verbosity
7.4.4.1. Changing Verbosity Levels Locally
7.4.4.2. Changing Verbosity Levels Remotely
8. Monitoring Library 2.0
8.1. Enabling Monitoring Library 2.0
8.2. Setting the Initial Metrics and Log Configuration
8.3. Setting the Application Name
8.4. Changing the Default Observability Domain ID
8.5. Configuring QoS for Monitoring Library 2.0 Entities
8.6. Setting Collector Service Initial Peers
9. Collector Service REST API Reference
9.1. Definitions
9.2. Root endpoint (base URL)
9.3. API Overview
9.4. API Reference
10. Observability Dashboards
10.1. System Status Dashboards
10.1.1. System Status Dashboard Common Elements
10.1.2. Alert Home Dashboard
10.1.3. Alert Category Dashboards
10.2. Entity List Dashboards
10.3. Entity Status List Dashboards
10.4. Entity Status Dashboards
10.5. Log Dashboards
10.5.1. Log Dashboard
10.5.2. Entity Log Dashboards
10.6. Control Dashboards
10.6.1. Log Control Dashboard
10.6.2. Metric Control Dashboards
10.6.2.1. Single Entity Metric Control Dashboards
10.6.2.2. Multiple Entity Metric Control Dashboards
11. Troubleshooting Observability Framework
11.1. Docker Container[s] Failed to Start
11.1.1. Check for Port Conflicts
11.1.2. Check that You Have the Correct File Permissions
11.2. No Data in Dashboards
11.2.1. Check that Collector Service has Discovered Your Applications
11.2.2. Check that Prometheus can Access Collector Service
11.2.3. Check that Grafana can Access Prometheus
11.2.4. Check that Grafana can Access Loki
12. Glossary
13. Release Notes
13.1. Supported Platforms
13.2. Compatibility
13.3. Supported Docker Compose Environments
13.4. What’s New in 7.3.0 LTS
13.4.1. Enhanced control of entities distributed across various Collector Service instances
13.4.2. New REST API in Collector Service to control telemetry data collection and distribution
13.4.3. Support for more flexible Observability Framework deployments
13.4.4. Control which metrics are collected
13.4.5. New Syslog facilities provide expanded log management
13.4.6. New logging category and plugin class labels enable more precise third-party backend queries
13.4.7. Updated dashboards support enhanced logging and dynamic metric control
13.4.8. Name change for some observability metrics
13.4.9. Third-party software upgrades
13.4.9.1. Observability Collector Service
13.4.9.2. Docker containers for Observability Collector Service
13.5. What’s Fixed in 7.3.0 LTS
13.5.1. Crashes
13.5.1.1.
[Critical]
Observability Collector Service could crash when an application was discovered
13.5.2. Vulnerabilities
13.5.2.1.
[Critical]
Potential out of memory error when using Curl 8.1.2
13.5.2.2.
[Critical]
Potential deletion of HSTS data when using Curl 8.1.2
13.6. Previous Releases
13.6.1. What’s New in 7.2.0
13.6.1.1. Observability Collector Service compatible with Monitoring Library 2.0
13.6.1.2. Support for most observability backends with OpenTelemetry integration
13.6.1.3. Support for Observability Collector Service security
13.6.1.4. Name change from “RTI Observability Library” to “RTI Monitoring Library 2.0”
13.6.1.5. Name change for some Observability metrics
13.6.1.6. Secured communications between Monitoring Library 2.0 and Observability Collector Service
13.6.1.7. Ability to set initial forwarding verbosity in MONITORING QoS policy
13.6.1.8. Ability to set collector initial peers in MONITORING QoS policy
13.6.1.9. Third-Party software changes
13.6.2. What’s Fixed in 7.2.0
13.6.2.1. Collector Service might have crashed on startup
13.6.2.2. Controllability issues on applications with same name
13.6.2.3. Unhandled exceptions may have caused segmentation fault
13.6.2.4. Race condition when processing remote commands led to failures and memory leaks when shutting down Collector Service
13.6.2.5. Collector Service could discard samples when monitoring large DDS applications
13.6.3. What’s New in 7.1.0
13.6.3.1. Third-Party Software
RTI Connext Observability Framework
7.
Telemetry Data
View page source
7.
Telemetry Data
7.1. Introduction
7.2. Resources
7.2.1. Resource Pattern Definitions
7.3. Metrics
7.3.1. Metric Pattern Definitions
7.3.2. Application Metrics
7.3.3. Participant Metrics
7.3.4. Topic Metrics
7.3.5. DataWriter Metrics
7.3.6. DataReader Metrics
7.3.7. Derived Metrics Generated by Prometheus Recording Rules
7.3.7.1. DDS Entity Proxy Metrics
7.3.7.2. Raw Error Metrics
7.3.7.3. Aggregated Error Metrics
7.3.7.4. Enable a Raw Error Metric
7.3.7.5. Custom Error Metrics
7.4. Logs
7.4.1. Syslog Levels and Facilities
7.4.2. Activity Context
7.4.3. Log Labels
7.4.4. Collection and Forwarding Verbosity
7.4.4.1. Changing Verbosity Levels Locally
7.4.4.2. Changing Verbosity Levels Remotely