Without AppCreator, we are only able to define QoS to share across distributed applications. We’d prefer to have a single definition of all the types, topics, and QoS. Furthermore, we’d like to have data readers/writers for those topics defined since QoS applies there too. This helps to avoid problems of mismatched QoS for a topic. This makes use of AppCreator attractive.
However, AppCreator does not allow definition of publishers, subscribers, data readers, and data writers “generically”, as far as I can tell. They are defined only as part of a Participant.
Our project is split across two teams and each team has multiple platforms. The DDS-based interface shared between the teams is binding – it is our “public” contractual agreement about how Team 1’s platforms communicate with Team 2’s platforms. However, neither team should care about specifically *which* platform on the other team is honoring the pub/sub. Generally we've tried to keep everything in a single XML file defining the public interface.
Since XML AppCreator requires us to bind specific Participants to specific pub/sub services, I see no way to abstract this. I’d much rather provide a set of Publishers, Subscribers, Data readers, and Data writers bound to types & topics with associated QoS as a “menu” from which participants can then implement. This relieves binding of participants and allows agility in moving the “service providers” and “service consumers” around as needed without affecting the public interface and its contract.
The affect is not just agility – this helps configuration control and the benefits of abstracting interfaces.
Do you have recommended practices or suggestions for how to solve this problem?
Some ideas on solving this:
- Define one “giant” Participant, but then all platforms incur the expense of all the services defined in it when they are likely interested in 20-30% of it. This is impractical as some of our platforms are embedded VxWorks devices
- Abandon AppCreator XML as the authoritative “public” interface document and move to another form (Word doc, for example) and then author AppCreator XML docs from it. These XML files would be completely private to each team.
- Split the 1 "large" XML file into 1 shared QoS file + 1 XML file for each platform. It becomes a little more difficult to ensure all matched readers & writers remain matched as the definition is fragmented across files however this is manageable. Further testing reveal that the strict XML schema requires a domain_library and a participant in every XML file, and runtime parsing throws errors if we try to add to a named participant_library that was already specified in a file previously loaded (though this can perhaps be worked around by declaring a new participant library for each participant...)
Any insight would be appreciated.
Sincerely-
Todd Moody
Hi Todd,
Are your teams averse to the idea of using a single "giant participant" as the database of options, and then use 3d party tools to build application-specific configurations? The tools I'm thinking of would then pull out the required parts from the giant participant, based on the app-specific config, and generate from that an app-specific Xml App Creation configuration file.
The "giant" participant then goes under version control, as well as the selector configurations. Extraction based on the selector configs can be automated. The app-specific Xml App Creation configurations are then treated as ephemeral files for the purposes of the build/system process.
Regards,
Rip
I had at one time considered using a streamlined "top level" computer-consumable format for our own purposes and processing that to produce the individual AppCreator XML files. This format would be somewhat specific to our project and could contain some assumptions to help reduce the complexity of the interface definition associated with all the options available through RTI DDS.. I had disposed of that idea as too complicated but perhaps it isn't afterall. Thanks for suggesting it.
On another note, I was able to split the XML files but was forced to work around some of the validation & runtime parsing oddities. For example, each participant_library in each file needs to have a unique name. This doesn't solve the problem of abstracting the interfaces between public & private but does help us divide up the XML files for configuration management and team-local build processes.
Thanks-
Todd
Hi,
In my past I worked for an XML company. One of their tools could take several input sources, merge and manipulate, and generate output sinks. Input and Output could be XML, (structured) plain text (csv or fixed width field files for example), database rows, a few others.
It should be possible to generate a streamlined "this is what this app needs" XML structure, and then have the tool autogenerate an application that when run would take the giant file, the 'needs' file and generate from them the target app config. Also, the 'needs' format doesn't need to be particular to your environment, so I don't need to see what you're doing.
The tricky bit is that each entity can have both Type and QoS references to elements elsewhere in the Giant file, so the streamlined "needs" has to be featured enough to understand things like that. Let me see if I can come up with a generic solution and build an app (it will be in Java, because the c++ the tool generates is Linux Hostile) that solves this.
Rip
Hello Todd,
Unfortunately there is no straightforward way to do this partition with XML AppCreation. Regarding the three alternatives you suggest I would say that the second one, providing a custom public interface, is probably the most correct. You could try to have your own XML schema for that authorative document and then apply a transformation (for instance by using XSLT) that generates specific XML AppCreation documents for every application given a set of arguments.
That option would require however some work. The third option, splitting in smaller XML AppCreation files, would be possibly easier to do by having different participant library names. A participant library just represents a way to organize different participant configurations that can be loaded by an application, since they represent the top-level entity. In other words, a participant library is a concept local to the application. Domains on the other hand could be considered as generic profiles that can be put in one single document and be shared and referenced by participants in other files. So for this approach you could have one shared file containing the QoS and Domain definitions, and then each team would have its own private file containing one participant library.
In any case it would be nice to provide XML AppCreation with optional elements that allow to achieve this straightforwardly. First idea would be to incorporate definitions of publisher, subscriber, data writer and data reader libraries, and allow the entities defined in a participant to refer/inherit from a library. Basically, none element in the current schema would change but it would contain new elements:
This configuration file could represent the shared system information. Then each team could have its own participants by using a participant library:
The first issues I see is less cohesion among entities and domains, but I think it is a good approach to start with. Let me know your ideas and what you would expect to do for carrying out your goals.
Best,
Antonio
Thank you both for posting thoughtful, useful suggestions.
With respect to additional tooling, I believe there is opportunity for code generation for infrastructure around RTI even with AppCreator. For example, even with AppCreator we must maintain matching XML tags in the XML file and source code that looks up entities (via the xxx_exp() methods presently). I could see the tooling being useful for providing the interface abstraction and extended to even produce the scaffolding to use the entities created by AppCreator upon loading. These seem quite related and doing both would help with code maintenance. As you said, however, even in the simplest case this is a fair amount of effort. I could see using XSLT to split the XML or even a tool like Apache Velocity to generate code.
If we have a DDS-enabled Project 2.0, I might consider putting this tooling in place upfront.
Wrt modifications to the rti_dds_profiles schema, I think your suggestions make good sense. This would offer us the ability to define our public interfaces in AppCreator XML without committing specific participants to the services in that XML. Please let me know if RTI should decide they want to change the AppCreator XSD.
Given our project time constraints, I will likely split the XML files. In the shared QoS XML file, I define a dummy participant_library and dummy domain_participant that aren't used but satisfies validation. There is risk that we introduce mismatches between pubs & subs as the public interface is defined across at least 4 files but we'll have to manage that as part of our CM process.
Sincerely-
Todd
Thanks Todd for your feedback.
Just one more thing. Why do you need a dummy participant_library and domain_library? If you just want to use the file for QoS profiles you don't need to include these libraries. If you want the file to be validated, then do it against rti_dds_qos_profiles.xml instead of rti_dds_profiles.xml.
Best,
Antonio
Hi,
I agree with you and Antonio regarding the desirability of having libraries for more DDS entities, not just domain participants. I will be advocating we put this feature high in the list of future enhancements for XML AppCreator.
In the meantime I had a suggestion which is not as good but could pehaps get you half way towards your goal. If I understood teh primary goal you had is to define univocally the DDS Interfaces in terms of DDS DataWriters and DataReaders so that the "appication developers" can compose applications based on those interfaces in a consisent manner.
Given that the configuration of a DataWriter is basically its QoS and Topic you could achieve this in two steps.
First you create a "CommonSystem.xml" XML file to define all the types, the QoS profiles and the Domain (with all Topics and their bindings to types). In this step you follow the pattern of creating a separate QoS profile for each Topic name. For example for topic "Foo" you would have "Foo__Profile", or simply "Foo". To avoid duplication you can use profile inheritance, but the point is that this would be an implementation detail and as far as your users is concerned they will rely on having one Profile per Topic name.
The second step is that users will define their own "MySystem.xml" that will include the "CommonSystem.xml". In "MySystem.xml" they will just define the DomainParticipant entities with the constraint that they must rely on the Topics defined in "CommonSystem.xml" and also use the QoS profile with the corresponding name in that file. That is:
<!-- This contents would be in the MySystem.xml created by each developer --> <participant_library name="MyPartcipantLib"> <!-- Te participant must refer to the one defined in CommonSystem.xml --> <domain_participant name="MyParticipant" domain_ref = "CommonSystem::CommonDomain"> <publisher name="MyPublisher"> <!-- To publish topic "Foo" this entry must refer to that Topic which is defined in "CommonSystem::CommonDomain" and must use the Foo__Profile also defined in CommonSystem.xml--> <data_writer topic_ref="Foo" name="MyFooWriter"> <datawriter_qos base_name="CommonSystem::Foo__Profile"/> </data_writer> </publisher> <!-- To subscribe to topic "Foo" this entry must refer to that Topic which is defined in "CommonSystem::CommonDomain" and must use the Foo__Profile also defined in CommonSystem.xml--> <subscriber name="MySubscriber"> <data_reader topic_ref="Bar" name="MyBarReader"> <datareader_qos base_name="CommonSystem::Bar__Profile"/> </data_reader> </subscriber> </domain_participant> </participant_library>As you noted they may be some issues regarding the XML verification and the mechanics of multiple XML file inclusion, but it this seems a suitable approach for you we could explore how to best resolve the other issues...
Gerardo
Antonio,
In response to your question about the dummy participant, rti_dds_qos_profiles.xsd does not allow for definition of a domain_library so there would be no way to define registered types and topics, which is a useful thing to have in a "common" xml file. So I stuck with rti_dds_profiles.xsd and left a dummy participant in there.
-Todd
Hi Todd,
Good point! It's true that the XML schema does not allow to define an standalone domain library. It looks that it's restrictive and prevent from modularizing XML system definitions in different files. My suggestion is that it should be possible to define a domain_library in a single file withouth the mandatory participant library.
Antonio