Our basic runtime environment requires upwards of 200 DDS connections, spread across 75 to 100 executables per execution. The need exists to run hundreds of simultaneous executions. We forsee a limitation is total DDS connections, due to the ~230 domains with ~120 participants each. What could be the solution to this scalability issue? Are there any workarounds/alternatives?
The Global Leader in DDS. Your Systems working as one.
Hi,
Is your scalability issue related to the domain number limit? I need more information on your scenario to figure out a solution to your problem. From your description, I understand you have from 75 to 100 applications running DDS. Are these applications creating more than one participant? Each application manages a number of endpoints (datawriters and datareaders), I assume the number of connections corresponds to the total number of endpoints, am I right?
It would be great if you could provide a description of what each application creates: participants, topics, publishers/subscribers, and data writers/data readers.
Fernando.
The easy answer to your question is that many of the applications will have multiple domain participants, using multiple domains/partitions. Each domain participant will have up to 600 readers/writers. The issue become multiple run scalability, a set of applications being a "run". Current DDS limitations would constrict us to about 150 simultanious runs, and this is unacceptable. Is there any way around this.
Hello Jim,
Not sure if this is what you meant, but to be clear the limitation is not 120 DomainParticipants on a single DDS DomainId. Rather the limitation applies to the number of DomainParticipants on a single DDS domain running all on the same computer.
This limitation comes from the fact that each DomainParticipants allocates 2 ports (which must get unique port numbers within a single computer). Since by default a DDS DomainId reserves a range of 250 ports you have the limit of 120 participants per domain per computer.
The limit of 250 domains comes from the fact that each DDS DomainId. reserves 250 ports and the fact that the highest port number allowed is ~65000.
You can use this Spreadsheet to compute the UDP ports used by RTI Connext DDS to see the ports that would be assigned for each DomainId and participantId. By the way in RTI Connext DDS the formulas to assign ports are configurable. So it may be possible to use this feature in your situation. But before I recommend this option I would like to understand better your setup.
If I understood correctly you have a set of "executions" that you want to run in parallel. Each "execution" has contains 75-100 "executables" each of which may have multiple DomainParticipants. Each DomainParticipant having 600 DataReaders/DataWriters. Is this correct?
What you did not clarify is how many computers are you using to run the system. If all of these were to run on a single computer then I agree that you could not do it with the out-of-the-box settings due to the port number limitations.
However, if you were running in say 10 computers (or VMs) then you would be able to run up to 1200 participants in a single DomainId (120 per computer). If you had 20 (or VMs) then you could run up to 2400 per domainId, and so forth.
Regarding the use of separate DomainIds for each "execution". I agree this seems the best way to keep things separate. So if you can afford the number of DomainIds this would require it seems a good way to go.
I do not understand as well the need to have multiple DomainParticipants in a single "executable". You are certainly allowed to do it and some applications (like bridges) really need it. However a DomainParticipant is an expensive resource so it is often better to use a single DomainParticipant per executable when you can. If you need to isolate traffic then you can do it by creating multiple Publishers and Subscribers and assigning different partitions. In terms of matching and communications you will get the same effect as different DomainIds but you will not need additional ports to be open for this and thus you have no limits on the numbers of Partitions.
Finally you mention each DomainParticipant having up to 600 DataReaders/Writers. This sounds like a large number as well. DataReaders and DataWriters are network entities that need to be discovered and consume significant resources. The question is why you need so many. Is is because you have many different data-types of information? Is it that you have many Topics?
This may be something you already know. But I wanted to mention it for completeness. Note that in DDS the data you publish is structured and each Topic can have "keys" that further identifies separate information streams within the Topic. So you do not need to map each stream of information to a separate Topic and hence a different DataWriter. For example if you have a set of temperature sensors which you want to publish and subscribe to separately, you do not need topics like "Temperature/Sensor1", "Temperature/Sensor2", "Temperature/Sensor3", etc. Rather you can have a single Topic "Temperature" and define the corresponding data-type to have a string attribute 'sensor_name' to which you assign the name of the sensor. Then you only need on DataWriter to publish any temperature sensor and a DataReader to read from any Temperature sensor. You can still subscribe to a single temperature sensor (or to a collection of them) using ContentFilteredTopics.
Finally as I mentioned you could modify the formula that RTI Connext DDS used to do port assignment. This would allow you to trade-off the number of participants you can run on a single computer (or VM) against the number of independent DomainId. This configuration can be changed on the DomainParticipantQos by setting the WireProtocolQosPolicy.
Regards,
Gerardo
For clarification, we are writing a simulation framework to integrate multiple simulation/tactical components into a single "Enterprise". We will need to run multiple "Enterprises" simultaniously. There will be a "Master Controller" that will start and controll all execution. There will be a single "Simulation controller" per computer to launch processes and possibly terminate zombie processes. All "Simulation Entities" will communicate with each other directly, sending meta-data to the "Master Controller" for time advancement control.
We are running simulations and tactical software in Monte-Carlo fashion. These tactical components (~50 executables) use up to 4 domains for data encapsulation, many act as "bridges" like you reference earlier. We cannot change the need to run tactical software. Add in our simulation components (somewhere on the order of 20 executables) and Controllers and this leads to the 200 domain participants per "Enterprise". Due to the total DDS participant max (~27500) we could only run ~138 consecutive Monte-Carlo runs. We are broken at this point. This should answer your number of Domain Participants question.
The number of message IDLs will be on the order of 600 to support this tactical system, what we use in the "Simulation World" will be ~30 messages. Not all processes will pub/sub all messages, but the possibility exists that someone could.
We plan on using as many compute resources as possible to execute all runs. We have several racks of 6-10 servers, each server having between 16 and 24 processors.
On the topic of "Keys". What is the typical run-time cost of data exclusion using keys. This simulation has strict requirements to run real-time. If keys are "costly" they must be discarded, otherwise i am open to using them.
So back to the original question, is there a hard limit to the number of participants that can exist on a single network? From everything i have seen, that answer is "yes", although i would love to hear that is wrong.
If more clarification is needed, please let me know.
hi...i m suffering with very serious problem in DDS..my problem is we recieve data on two diffrent domain id (1 and 2)....but at run time we change the setting of domain id -2 and domain id 2 kill our listner and recieve data from domain id 1.this is depend upon runtime setting...and using this setting sometime domain 2 block his listner and recieve data from domain id 1.(for same topic data with diffrent domain id...content of topic is diffrent).how we handle this problem?
I apologize but I did not understand the description of your problem:
You said you receive data on two DDS domain IDs (1 and 2). This means when you create the respective DDS DomainParticipant you are specifying domainId=1 and domainId=2, correct?
You said that "at run-time we change the setting from domain id 2". What do you mean by "change the setting"? Which setting are you changing? Are you changing a QoS, re-stating the application? Could you explain more precisely what you are doing here?
You said "and domain id 2 kill our listener and receive data from domain id 1". I do not understand. What do you mean by "killing teh listener" Who is receiving data from domain id 1? Is that the DomainParticipant that was created in domain id 2? That should not be possible.
You said "domain 2 block his listener and recieve data from domain id 1" I also do not understand this. One domain id should never receive data froma different domain id, unless you have some bridging application like the RTI Routing Service.
Can you explain your problem again with more complete and precise description on exactly what you are doing and how you are doing it?
Gerardo
we are using same domain participant with 2 domain id.we are recieving same topic1 but diffrent content with diffrent (1-2_domain id.)dpmain id
we are recieving our data with two diffrent domain id..
setting means -for this topic1 normaly we recieve data with diffrent domain id with diffrent content(setting is another topic which decide that topic1 data recieving with diffrent domain id or both are recieving same domain id data) this is decided at runtime when we recieve setting topic value than run time we decide what we want
quote: "we are using same domain participant with 2 domain id"
This is technically impossible, using the standard definitions we have for domain participant, and domain id.
By Domain Participant, do you mean "an application which is enabled for DDS"? in which case, are you creating two Domain Participants, one for each domain id?
Please supply your long-form definitions of "domain participant" and "domain id". It would help if you could supply the actual code you use to create the single domain participant.
hi we are attachig one folder "problem"......in this folder we attach two idol file "code.idl" and " typedef.idl".,in typedef.idl we typedef (" typedef long variable1") variable1.and include this typedef.idl in code.idl file
#include "typedef.idl"
struct code
{
variable1 var;
};
and after doing this process when we generate automatic code for JAVA using DDS TOOL
and run this code in eclipse than we got "COMPILATION ERROR" and we can not complie this code....(error in code.java and codetypesupport.java)
tydef is not working ...
we attach our two idol file with automatic generated code
can u give me solution of this problem..
in same idl file when we generate c++ automatic code genration than its working properly.............but java code for this file is not working
can any person give me my problem solution?
By asking your questions on an unrelated and two-year-old thread, you've effectively hidden them from the people who might be able to answer them. Also, because you've asked multiple unrelated questions, I don't know which question you mean, when you say "can any person give me my problem solution"
Please open new topic threads (one for each unanswered question). Click on the "Technical Questions" link at the top of this page, then click on the "New Topic" link, and enter your problem discription.
rip