Performance Testing 5 Publisher 1 Subscriber Error

3 posts / 0 new
Log in or register to post comments
Last post
Sat, 11/16/2019 - 01:10
#1
Blitz3r
Offline
Last seen: 9 months 3 days ago
Joined: 02/15/2019
Posts: 44
Performance Testing 5 Publisher 1 Subscriber Error

Hello,

I am trying to run the following test:

start cmd /k perftest_java.bat -pub -dataLen 100 -domain 2 -latencyCount 1000 -numIter 1000000 -bestEffort -sendQueueSize 1 -pidMultiPubTest 0 -numSubscribers 1
 
start cmd /k perftest_java.bat -pub -dataLen 100 -domain 2 -latencyCount 1000 -numIter 1000000 -bestEffort -sendQueueSize 1 -pidMultiPubTest 1 -numSubscribers 1
 
start cmd /k perftest_java.bat -pub -dataLen 100 -domain 2 -latencyCount 1000 -numIter 1000000 -bestEffort -sendQueueSize 1 -pidMultiPubTest 2 -numSubscribers 1
 
start cmd /k perftest_java.bat -pub -dataLen 100 -domain 2 -latencyCount 1000 -numIter 1000000 -bestEffort -sendQueueSize 1 -pidMultiPubTest 3 -numSubscribers 1

start cmd /k perftest_java.bat -pub -dataLen 100 -domain 2 -latencyCount 1000 -numIter 1000000 -bestEffort -sendQueueSize 1 -pidMultiPubTest 4 -numSubscribers 1
 
start cmd /k perftest_java.bat -sub -dataLen 100 -domain 2 -bestEffort -numPublishers 5
 
When the test runs I see the following outcome:
 
Subscriber: Runs continuously without errors.
Publisher 0: Runs without errors and prints out results like it should.
Publisher 1/2/3: I get the following error:
 
 
 
I checked the code and found the following:
 
Line 1297 of PefTest.java brings me to this line of code:
 
reader_listener.print_summary_latency();
 
I am assuming that the reader_listener object may be null. I came across this line of code:
 
reader_listener = new LatencyListener(num_latency,_latencyTest?writer:null);
 
I then noticed the comment that states the following: "Only publisher with ID 0 will send/receive pings".
 
Does this also happen with multiple publisher tests like the one I am trying to run? Does this mean that running multiple publishers that publish to the same subscriber isn't possible?
 
Some clarification would be great.
 
Thanks!
 
Blitz3r
 
 
 
AttachmentSize
Image icon errar.png15.93 KB
Top
Mon, 11/18/2019 - 01:48
jmorales
jmorales's picture
Offline
Last seen: 1 month 2 weeks ago
Joined: 08/28/2013
Posts: 61

Hi Blitz3r,

Thanks for reporting this, and also for debugging the root cause. When using RTI Perftest to do a N-1 test (many to 1), latency is always reported by the Perftest Publisher with "pid" 0. This does not mean that the scenario you are executing is not supported, it just means that we will just answer to the 1st Publisher Application, by design.

Now, wrt your error, what version of RTI Perftest are you using? I mention this because that statement is, in the latest versions of Perftest, surrounded by this:

        if (pubID == 0) {
            reader_listener.print_summary_latency();
            reader_listener.end_test = true;
        } else {
            System.out.println("Latency results are only shown when -pidMultiPubTest = 0");
        }
 
So... the errors you are having should not happen (as the Pid would not be 0).
 
=============================================================
 

I also took the chance to review the test you are performing: I would like to ask:

-pub -dataLen 100 -domain 2 -latencyCount 1000 -numIter 1000000 -bestEffort -sendQueueSize 1 -pidMultiPubTest 4 -numSubscribers 1

Why are you setting the sendQueueSize to that number? That is going to slow down your system most likely. What kind of tests do you want to do? What are you measuring and what is the purpose of the test? I see you have been asking a bunch of questions about performance, perftest and particularly about the Java api for RTI Perftest. Could you let us know the use case (cases) you are interested in so O can somehow guide you?

Regards,

Javi

Top
Tue, 12/10/2019 - 09:16
Blitz3r
Offline
Last seen: 9 months 3 days ago
Joined: 02/15/2019
Posts: 44

Hi Javier,

Apologies for the lateness of this response. I had surgery recently so I was and still am recovering. Thank you for taking the time to answer my post.

Here are my answers to your questions (please excuse the misnumbering):

  1. Why are you setting the sendQueueSize to that number?
    1. I wanted to see how the test results differed under different queue sizes. You mention that this will slow down the system when setting the sendQueueSize to 1. Why do you say that?
  2. What kind of test do you want to do?
    1. I am interested in looking at how the latency changes under varying load using the throughput test.
  3. What are you measuring and what is the purpose of the test?
    1. I am measuring the latency. I want to see how the results differ under different loads.
  4. Could you let us know the use case (use cases) you are interested in so I can somehow guide you?
    1. I would like to analyse latencies under varying throughput whilst varying the number of publishers and subscribers.

 

Here are some examples of tests that I have run:

Test Set 1 [Throughput Test + Varied Latency Count]:

Constants: 500,000 Iterations + 1 Publisher + 10,000 Latency Count

Variables: 1/5/10 Subscribers

 

Test Set 2 [Best Effort VS Reliable]:

Constants: 1 Publisher + 1/5/10 Subscribers + 10,000 Latency Count + 500,000 Iterations

Variables: Best Effort/Reliable + Unicast/Multicast

 

Test Set 3 [100 VS 512 Bytes]:

Constants: 1 Publisher + 1/5/10 Subscribers + Best Effort Unicast + Best Effort Multicast + Reliable Unicast + Reliable Multicast

Variables: 100/512 Bytes

 

Test Set 4 [Instances]:

Constants: 1 Publisher + 1/5/10 Subscribers + 10 Instances

Variables: Best Effort/Reliable + Unicast/Multicast

 

I would like to run the following in the future:

 

Test Set 1 [Multipub 100 Byte]

Constants: 100 Bytes

Variables: 1/5/10 Publishers + 1/5/10 Subscribers + Best Effort/Reliable + Unicast/Multicast

 

Test Set 2 [Multipub 512 Byte]

Constants: 512 Bytes

Variables: 1/5/10 Publishers + 1/5/10 Subscribers + Best Effort/Reliable + Unicast/Multicast

 

Test Set 3 [Multipub 100 Byte]

Constants: 100 Bytes

Variables: 20/50/100 Publishers + 20/50/100 Subscribers + Best Effort/Reliable + Unicast/Multicast

 

Your guidance would be much appreciated and I look forward to hearing from you soon!

 

Thanks,

Blitz3r

Top