Observability Framework

• Copyrights and Notices

• 1. About Connext Observability Framework
  • 1.1. Use Cases
  • 1.2. Components
  • 1.3. Telemetry Data
  • 1.4. How Observability Framework Works
    • 1.4.1. Distribution of Telemetry Data
    • 1.4.2. Telemetry Backends
    • 1.4.3. Remote Debugging
    • 1.4.4. Control and Selection of Telemetry Data
    • 1.4.5. Security
• 2. Components
  • 2.1. Monitoring Library 2.0
  • 2.2. Collector Service
    • 2.2.1. Supported Endpoints
    • 2.2.2. Built-in Configurations
    • 2.2.3. Third-Party Components
  • 2.3. Observability Dashboards
• 3. Observability Framework Deployments
  • 3.1. Before you Begin
  • 3.2. Evaluation Deployment
  • 3.3. Production Deployments
    • 3.3.1. Single Collector Service Instance
    • 3.3.2. Single Layer of Collector Service Instances
    • 3.3.3. Multiple Layers of Collector Service Instances
    • 3.3.4. Multiple Layers of Collector Service Instances with OpenTelemetry Collector
• 4. Security
  • 4.1. Secure Communication between Connext Applications and Collector Service
    • 4.1.1. Secure Communication between Connext Applications and Collector Service (Evaluation Deployment)
    • 4.1.2. Secure Communication between Connext Applications and Collector Service (Production Deployment)
  • 4.2. Secure Communication with Collector Service HTTP Servers
    • 4.2.1. Secure Collector Service HTTP Servers (Evaluation Deployment)
    • 4.2.2. Secure Collector Service HTTP Servers (Production Deployment)
  • 4.3. Secure Communication with Third-Party Component HTTP Servers
    • 4.3.1. Secure Third-Party Component HTTP Servers (Evaluation Deployment)
    • 4.3.2. Secure Third-Party Component HTTP Servers (Production Deployment)
  • 4.4. Generating the Observability Framework Security Artifacts
    • 4.4.1. Generating DDS Security Artifacts
    • 4.4.2. Generating HTTPS Security Artifacts
• 5. Installing and Running Observability Framework for Evaluation
  • 5.1. Configuring the Docker Workspace for Observability Framework
    • 5.1.1. Configure the JSON File
    • 5.1.2. Run the Observability script to create the Observability workspace
  • 5.2. Initialize and Run Docker Containers
  • 5.3. Verify Docker Containers are Running
  • 5.4. Configure Grafana
    • 5.4.1. Initial Login
    • 5.4.2. Configuration Options
  • 5.5. Stop Docker Containers
  • 5.6. Start Existing Docker Containers
  • 5.7. Stop and Remove Docker Containers
  • 5.8. Removing the Docker Workspace for Observability Framework
• 6. Getting Started Guide
  • 6.1. About the Observability Example
    • 6.1.1. Applications
    • 6.1.2. Data Model
    • 6.1.3. DDS Entity Mapping
    • 6.1.4. Command-Line Parameters
  • 6.2. Before Running the Example
    • 6.2.1. Set Up Environment Variables
    • 6.2.2. Compile the Example
    • 6.2.3. Install Observability Framework
    • 6.2.4. Start the Collection, Storage, and Visualization Docker Containers
  • 6.3. Running the Example
    • 6.3.1. Start the Applications
    • 6.3.2. Changing the Time Range in Dashboards
    • 6.3.3. Simulate Sensor Failure
    • 6.3.4. Simulate Slow Sensor Data Consumption
    • 6.3.5. Simulate Time Synchronization Failures
    • 6.3.6. Change the Application Logging Verbosity
    • 6.3.7. Change the Metric Configuration
    • 6.3.8. Remote Debugging with Admin Console
    • 6.3.9. Close the Applications
• 7. Telemetry Data
  • 7.1. What is Telemetry Data
    • 7.1.1. Levels
    • 7.1.2. Categories
  • 7.2. Resources
    • 7.2.1. Resource Pattern Definitions
  • 7.3. Metrics
    • 7.3.1. Metric Pattern Definitions
    • 7.3.2. Application Metrics
    • 7.3.3. Participant Metrics
    • 7.3.4. Topic Metrics
    • 7.3.5. DataWriter Metrics
    • 7.3.6. DataReader Metrics
    • 7.3.7. Derived Metrics Generated by Prometheus Recording Rules
  • 7.4. Non-Metric Observables
    • 7.4.1. Application Observables
    • 7.4.2. Participant Observables
    • 7.4.3. Type Observables
    • 7.4.4. Topic Observables
    • 7.4.5. Publisher Observables
    • 7.4.6. DataWriter Observables
    • 7.4.7. Subscriber Observables
    • 7.4.8. DataReader Observables
  • 7.5. Logs
    • 7.5.1. Syslog Levels and Facilities
    • 7.5.2. Activity Context
    • 7.5.3. Log Labels
    • 7.5.4. Collection and Forwarding Verbosity
• 8. Monitoring Library 2.0
  • 8.1. Enabling Monitoring Library 2.0
  • 8.2. Setting Initial Metrics and Log Configuration
    • 8.2.1. Enable All Metrics Manually
    • 8.2.2. Enable All Metrics Using a Snippet
    • 8.2.3. Enable a Custom Set of Metrics
  • 8.3. Configuring Distribution Settings
    • 8.3.1. Setting Application Name
    • 8.3.2. Changing the Default Observability Domain ID
    • 8.3.3. Setting Collector Service Initial Peers
  • 8.4. Configuring QoS for Entities
  • 8.5. Connecting to Collector Service Over WAN
• 9. Collector Service REST API Reference
  • 9.1. Definitions
  • 9.2. Root endpoint (base URL)
  • 9.3. API Overview
  • 9.4. API Reference
• 10. Observability Dashboards
  • 10.1. System Status Dashboards
    • 10.1.1. System Status Dashboard Common Elements
    • 10.1.2. Alert Home Dashboard
    • 10.1.3. Alert Category Dashboards
  • 10.2. Entity List Dashboards
  • 10.3. Entity Status List Dashboards
  • 10.4. Entity Status Dashboards
  • 10.5. Log Dashboards
    • 10.5.1. Log Dashboard
    • 10.5.2. Entity Log Dashboards
  • 10.6. Control Dashboards
    • 10.6.1. Log Control Dashboard
    • 10.6.2. Metric Control Dashboards
• 11. Troubleshooting Observability Framework
  • 11.1. Docker Container[s] Failed to Start
    • 11.1.1. Check for Port Conflicts
    • 11.1.2. Check that You Have the Correct File Permissions
  • 11.2. No Data in Dashboards
    • 11.2.1. Check that Collector Service has Discovered Your Applications
    • 11.2.2. Check that Prometheus can Access Collector Service
    • 11.2.3. Check that Grafana can Access Prometheus
    • 11.2.4. Check that Grafana can Access Loki
  • 11.3. Can Collector Service run in Windows or macOS?
• 12. Glossary
• 13. Release Notes
  • 13.1. Supported Platforms
  • 13.2. Compatibility
  • 13.3. Supported Docker Compose Environments
  • 13.4. Supported Docker Environments for Collector Service
  • 13.5. What’s New in 7.6.0
    • 13.5.1. Implemented scalable remote system debugging
    • 13.5.2. More flexibility when configuring Collector Service for remote debugging
    • 13.5.3. New built-in profile makes it easier to monitor Connext applications across geographically separated networks
    • 13.5.4. Decreased bandwidth usage when not actively debugging remote systems
    • 13.5.5. Third-Party Software Changes
  • 13.6. What’s Fixed in 7.6.0
    • 13.6.1. [Critical] Potential crash when removing unready resources
    • 13.6.2. [Critical] Possible memory leak due to error in processing child resources
    • 13.6.3. [Critical] Unbounded memory growth when using remote administration commands in Observability Framework
    • 13.6.4. [Major] Query condition errors on discovering more than four applications simultaneously
    • 13.6.5. [Major] Potential timeout on commands addressed to more than one application
    • 13.6.6. [Minor] RTI Collector Service sent samples that should have been discarded to the endpoints
  • 13.7. Previous Releases
    • 13.7.1. What’s New in 7.5.0
    • 13.7.2. What’s Fixed in 7.5.0
    • 13.7.3. What’s Fixed in 7.4.0