3 ms·
One can efficiently stream data even when there's a need to combine multiple streams. I oftentimes have data sorted on its ID, for instance (usually when it com
by drej 7y ago
One can efficiently stream data even when there's a need to combine multiple streams. I oftentimes have data sorted on its ID, for instance (usually when it comes from a database), and then can easily join and group by this ID (the same way `uniq` would).
All using Python's standard library, here's a quick post I wrote about this. I last used it a week ago and got from 950 MB RAM usage down to about 45 megs. https://kokes.github.io/blog/2018/11/25/merging-streams-python.html https://kokes.github.io/blog/2018/11/25/merging-streams-pyth...