3 ms·

How is it good to have 1million queued messages but only utilize 5% of the cpu?

by Meai 15y ago

darklajid 15y ago
I think the argument is that queuing should be cheap and the consumers should actually use the CPU. He has no consumers yet, he just fires messages like crazy and they are left pending. That leads me to the question what this benchmark is good for.. Someone tried to measure ~something~, but it's useless right now and targets localhost. I'm not impressed.
- chubs 15y ago
  Thanks for having a look! The main thing i wanted to check for as of now is the performance (in terms of memory usage) of the hash/dictionary/list library i'm using (klib). In which case, i'm personally impressed with the work of klib's author (who, as it turns out, has chosen to be anonymous!)
  - tom_b 15y ago
    What led you to klib? Looks interesting . . . given the fasta/fastq format parser in the source code base, we can guess that super high performance/low memory footprint development is pretty critical for the klib author. The macro-oriented dev stuff is kind of new to me, but I'm sadly years away from writing C.
    - chubs 15y ago
      I googled for C hash libraries, and came across one of those 'shootout' blog posts where they compare memory size and cpu usage, and klib looked to be a winner. Plus it's only one or two header files to make it work. Documentation was poor, so i contributed some. So i'm happy with my choice. Yes, the macro-oriented stuff is a headache, but the raw speed it gives you is worth it. And don't think you're years away from C! I work in C# and javascript mainly (and recently, obj-c) so, for me, this is as much a learning exercise as anything.
  - darklajid 15y ago
    You get big props from me for - replying with further details and - responding in a very friendly way although my tone was neither constructive nor friendly. Thank you for that great response.
    - chubs 15y ago
      Holy moly! What a turnaround. You're welcome :)
chubs 15y ago
Those are orthogonal stats, true. More relevant is queued messages vs RAM used, which i'm quite happy with: 1m messages = 65m ram. Also think about the 1000 msgs/sec vs 5% cpu, which i'm also happy with. Thanks a lot for having a look!

How is it good to have 1million queued messages but only utilize 5% of the cpu?

by Meai 15y ago

How is it good to have 1million queued messages but only utilize 5% of the cpu?

I think the argument is that queuing should be cheap and the consumers should actually use the CPU. He has no consumers yet, he just fires messages like crazy and they are left pending. That leads me to the question what this benchmark is good for.. Someone tried to measure ~something~, but it's useless right now and targets localhost. I'm not impressed.

chubs 15y ago
Thanks for having a look! The main thing i wanted to check for as of now is the performance (in terms of memory usage) of the hash/dictionary/list library i'm using (klib). In which case, i'm personally impressed with the work of klib's author (who, as it turns out, has chosen to be anonymous!)
- tom_b 15y ago
  What led you to klib? Looks interesting . . . given the fasta/fastq format parser in the source code base, we can guess that super high performance/low memory footprint development is pretty critical for the klib author. The macro-oriented dev stuff is kind of new to me, but I'm sadly years away from writing C.
  - chubs 15y ago
    I googled for C hash libraries, and came across one of those 'shootout' blog posts where they compare memory size and cpu usage, and klib looked to be a winner. Plus it's only one or two header files to make it work. Documentation was poor, so i contributed some. So i'm happy with my choice. Yes, the macro-oriented stuff is a headache, but the raw speed it gives you is worth it. And don't think you're years away from C! I work in C# and javascript mainly (and recently, obj-c) so, for me, this is as much a learning exercise as anything.
- darklajid 15y ago
  You get big props from me for - replying with further details and - responding in a very friendly way although my tone was neither constructive nor friendly. Thank you for that great response.
  - chubs 15y ago
    Holy moly! What a turnaround. You're welcome :)

chubs 15y ago

Those are orthogonal stats, true. More relevant is queued messages vs RAM used, which i'm quite happy with: 1m messages = 65m ram. Also think about the 1000 msgs/sec vs 5% cpu, which i'm also happy with. Thanks a lot for having a look!