Language: Java
RTI Version: 5.3.1.22
Hi,
I am launching 3 routing service instances via the RTIRoutingService.bat files which launch rtiroutingservice.exe programs.
Due to the inability to configure the routing service to log directly to a file I am forced to use the RTI Distributed Logger to receive log messages.
We often experience intermittent DDS communication issues with the routing services in our system and as such need to have the logs available in an organized manner so that we can provide logs in the event that we need to create a support case to figure out what might be going wrong with the routing over the TCP transport we are using.
It seems RTI Distributed Logger provides no good human readable way to distiguish which log message belongs to what application since the messages are all being jammed into one logger topic. As far as I know, the only good distinguishing factor is the
logMessage.hostAndAppId.rtps_host_id
logMessage.hostAndAppId.rtps_app_id
fields of the LogMessage class. Is there any way to convert these raw integer values back into a useful value that can be used to name and separate log messages into their own files? I would be preferrable if the LogMessage would contain the AppName/RoutingService name as a string so that it could be used in that manner.
In my case I do not know the rtps_host_id or app_id of the routing services I am launching. So if that is the only information I am receiving about the log message origin, it be comes extremely difficult to have useful logging. I've been looking at the hello_file_logger example for the distributed logger and in the readme.txt documentation it states that it creates one large central log for every application running in a system. While this may be useful for finding any errors that occur it becomes not useful if one cannot distiguish with ease where the error originated from.
Hopefully my concern/confusion is understood.
Looking for options here.
Thanks
Since distributed logger is intended to work for any DDS application there may not be any routing service specific fields to use.
If you're interested in the details of the datatype you can look at the distributed logger IDL file located at: <install dir>/resource/idl/distlog.idl
But one potential solution would be to change the rtps_host_id or rtps_app_id to something that is useful to you. By default these are set to DDS_RTPS_AUTO_ID, but it looks like you can base these on the MAC or UUID if those help you determine more information about the remote application. Or you could even manually specify them yourself.
https://community.rti.com/static/documentation/connext-dds/6.0.1/doc/api/connext_dds/api_cpp/structDDS__WireProtocolQosPolicy.html
Hi Jake, I have a couple of other options for you. You mentioned:
There is a way for routing service to log locally instead of using distributed logger.
USER_QOS_PROFILES.xml:
<dds xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../xsd/rti_dds_qos_profiles.xsd"><qos_library name="DefaultLibrary"><qos_profile name="DefaultProfile1" is_default_participant_factory_profile ="true"><participant_factory_qos><logging><verbosity>ALL</verbosity><category>ALL</category><print_format>MAXIMAL</print_format><output_file>logfile.txt</output_file></logging></participant_factory_qos></qos_profile></qos_library></dds>If you wanted to log to both distributed logger and a file, you would probably have to save the distlog messages to a file, but then we're back to your original question.
There is an example application in this directory if you're interested in doing that: rti_workspace\6.0.1\examples\distributed_logger\java\hello_file_logger\
Other options:
Beyond specifying the host/app ID to know where each distributed logging message came from, since the distlog.idl struct HostandAppId is extensible you could add a new string field to specify a friendly name.
This could work well for your own application that may be generating distlog messages, but for routing service this may not be what you're looking for since routing service itself wouldn't be using this new field.
Also routing service already specifies a participant name of "RTI Routing Service: <application name>". The application name can be specified via an arguement, otherwise it uses the <routing_service name=""> specified in your routing service's xml configuration.
You can read this participant name via the built in topics and there may be a way to map this to the host/app ID of distlog messages.
Let me know if you're interested in these other options and I'll do some more digging to see how practical they are.
Hi Ross,
Thank you so much for your feedback.
What is the functionality of the logging to file mechanism? Does the file:
These are features that we would like to retain since our services can be left running for multiple days at a time at high verbosity.
We would like to be able to take advantage of RTI's distributed logger system so that we can log using one of the well established 3rd party logging mechanisms like log4j2, slf4j, or java.util.logging. This way we can totally customize file sizes, location, archiving, and roll over.
As far as I can tell the FileLogger example in templates/rtiworkspace does not provide file roll over options or any thing of that sort. I've also reviewed the other examples in the rtiworkspace for Java and none of them seem to provide a mechanism by which the source of a log message can be clearly determined without some kind of mapping logic.
In our routing service configurations we are forced to use AUTO_ID_FROM_MAC on one of our services that is using the TCP transport due to an issue we had in the past with duplicate GUID values being generated for different sets of routing services running on cloned test machine boxes. As such, I believe we lose access to the process ID of the application that typically resides in the app_id field when using AUTO_ID_FROM_IP which I could have potentially otherwise used.
I am highly interested in how you would map the host and app id's via the routing service participants like you mentioned. If you could provide an example of how to access that information of the routing service participants from a separate application that would be greatly helpful.
However, if the built in log directly to file xml example you provided in your previous post supports log file roll over and archiving like I mentioned that would work. If that is the case can you also provide a set of configuration options for that logger?
Thank you,
Jake
Hi Jake,
Yes the logging mechanism allows for rolling over to multiple files and overwriting the oldest log file.
The available options are explained here: https://community.rti.com/static/documentation/connext-dds/6.0.1/doc/manuals/connext_dds/html_files/RTI_ConnextDDS_CoreLibraries_UsersManual/index.htm#UsersManual/LOGGING_QosPolicy__DDS_Extension_.htm#8.4.1_LOGGING_QosPolicy_(DDS_Extension)%3FTocPath%3DPart%25202%253A%2520Core%2520Concepts%7C8.%2520Working%2520with%2520DDS%2520Domains%7C8.4%2520DomainParticipantFactory%2520QosPolicies%7C8.4.1%2520LOGGING%2520QosPolicy%2520(DDS%2520Extension)%7C_____0
I tested these features by adding these three lines to the xml above:
<output_file_suffix>suffix</output_file_suffix><max_bytes_per_file>10000</max_bytes_per_file><max_files>10</max_files>When I first execute the routing service, it created log files logfile.txt1suffix through logfile.txt5suffix (you probably want to change the above settings to something more sane like filename==logfile and suffix==.log)
If I let this run it will create the 10 files, each limited to 10,000 bytes and roll over overwriting the oldest/first file as expected.
If you don't specify these max_bytes_per_file or max_files then connext will create a single file that grows forever.
The other thing to note is that if you set max_bytes_per_file and leave max_files==1, then once the size limit is reached that single file will be overwritten so we suggest using max_files >=2 when max_bytes_per_file is not the default (unlimited)
One thing to note that is if you restart routing service it always seems to start over and log to the first log file regardless of if it didn't write all 10 log files in the previous execution (but it sounds like this may be what you want).