3 ms·
It's misleading for sure but they're writing 250 'events' per document.
by uncoder0 7y ago
It's misleading for sure but they're writing 250 'events' per document.
- MuffinFlavored 7y ago200k documents per second is a lot less impressive, no?
- Scarbutt 7y agoand less catchy.
- Benfromparis 7y ago"200k documents per second" would be less catchy you are right, but it also doesn't reflect the complexity to handle that throughput. Most of the benchmark we can see around like "1m document per second" are using small documents in POC environement. In our setup, each of the 250 fields are store and indexed in ES, making it CPU and I/O intensive, in a production environment.
- virgilp 7y agoI don't know. Have you tried it? I worked on a Kafka streams service written in Java that processed "changelog" messages (it involved one query to CosmosDB per message, and logging the result to Kafka for downstream processing by other systems). Now, we had a rather limited number of workers (4 or 8? I don't remmember), but getting to 100k messages per second was rather challenging.
- hmschreck 7y ago200k documents per second really isn't that much for Elasticsearch. The single-instance setup we have in my small company (around 25 people total) has been sent in excess of 40k/s at times, and even then it doesn't slow down noticeably.