8 ms·
This is a great little library! Well done! Several years ago I wrote this tiny (very flakey) in memory topic producer cache - https://github.com/jeremylcarter/
by jeremycarter 4y ago
This is a great little library! Well done!
Several years ago I wrote this tiny (very flakey) in memory topic producer cache - https://github.com/jeremylcarter/PartitionCache https://github.com/jeremylcarter/PartitionCache. The purpose was to create something like Event Hubs and/or Kafka but in memory only - just to split up workloads by a producer key. I have since seen how bad my initial implementation was, and I did rewrite it - but haven't pushed the branch up. It was designed for localhost/integration testing usage only.
Perhaps there is a use case for you to have keyed producers. Whereby the same items from the same producer ID are always processed by the same worker - even if it were just round robin assignment to a "batch". This however can lead to "noisy" batches (unless some batch resizer/mover algorithm was created to distribute the load evenly).
I have also dabbled in the Qoollo Turbo library which has a nice QueueProcessor which is easy to extend (see QueueAsyncProcessor<T>) - you should check that out too for inspiration.