4 ms·

> And there's probably a bunch more I've forgotten + Latency vs throughput. What would be really informative (and this is not remotely directed only at Jian Z

> And there's probably a bunch more I've forgotten + Latency vs throughput. What would be really informative (and this is not remotely directed only at Jian Zhen's work here) is to see metrics across a spectrum per a given modality per a standard topology and capacity. This way it would be almost clear at a glance at which approach is the best fit for a given domain. Also I wonder if the time has come for the software geeks to visit their fellow geeks in the Mech-E departments and check out the work to date on flow analysis … ;)

zhenjl 12y ago
Excellent question regarding latency. Tyler Treat who wrote an excellent blog post on MQ performance tested this and got mean latency for 1000000 messages of 98.751015 ms. https://github.com/tylertreat/mq-benchmarking/pull/5 https://github.com/tylertreat/mq-benchmarking/pull/5
- eternalban 12y ago
  谢谢. Have you ever read this? http://zedshaw.com/archive/programmers-need-to-learn-statistics-or-i-will-kill-them-all/ http://zedshaw.com/archive/programmers-need-to-learn-statist...
  - zhenjl 12y ago
    I have not. But I am going to assume you are telling me to learn more about how to calculate my performance numbers. :) Fair point and I will bookmark for future reading.
    - jacques_chester 12y ago
      Good benchmarking requires a lot of work. What you did is actually useful within the narrow context you're applying it to -- quickly validating design changes. But if, for example, you wanted to send it into SIGMETRICS for publication, they'd expect a whole bunch of extra work and background material. One thing that's tricky in our profession is that the space of all possible configurations and inputs is gigantic and we only tend to sample the very, very small parts of it that occur to us.
      - tylertreat 12y ago
        Yeah, for the record, the benchmarks I've run aren't at all scientific. Should also be looking at percentiles. That said, surge looks promising so nice work, Jian :) My dream would be to build a framework which orchestrates testing across distributed senders/receivers. That would give you a much more accurate representation of performance and reliability instead of a single machine.

zhenjl 12y ago

Excellent question regarding latency. Tyler Treat who wrote an excellent blog post on MQ performance tested this and got mean latency for 1000000 messages of 98.751015 ms. https://github.com/tylertreat/mq-benchmarking/pull/5 https://github.com/tylertreat/mq-benchmarking/pull/5

eternalban 12y ago
谢谢. Have you ever read this? http://zedshaw.com/archive/programmers-need-to-learn-statistics-or-i-will-kill-them-all/ http://zedshaw.com/archive/programmers-need-to-learn-statist...
- zhenjl 12y ago
  I have not. But I am going to assume you are telling me to learn more about how to calculate my performance numbers. :) Fair point and I will bookmark for future reading.
  - jacques_chester 12y ago
    Good benchmarking requires a lot of work. What you did is actually useful within the narrow context you're applying it to -- quickly validating design changes. But if, for example, you wanted to send it into SIGMETRICS for publication, they'd expect a whole bunch of extra work and background material. One thing that's tricky in our profession is that the space of all possible configurations and inputs is gigantic and we only tend to sample the very, very small parts of it that occur to us.
    - tylertreat 12y ago
      Yeah, for the record, the benchmarks I've run aren't at all scientific. Should also be looking at percentiles. That said, surge looks promising so nice work, Jian :) My dream would be to build a framework which orchestrates testing across distributed senders/receivers. That would give you a much more accurate representation of performance and reliability instead of a single machine.