3 ms·
The Past, Present and Future of Stream Processing
- treyfitty 3y agoThis “article” uses a lot of words to say so little. There is very little to be gained by reading this. It reads like a search engine optimization attempt.
- ambicapter 3y agoFully agree. "The Future of Stream Processing is Open Source and Cloud" Isn't that an oxymoron?
- mgaunard 3y agoI've never seen so much bad technology quoted in a single article.
- SonOfLilit 3y agoAsk HN: I've recently started using Spark for some simple stream processing amenable to embarrassing parallelism (that also involves making a REST call per row). It's been a horrible, horrible experience, troubleshooting IT issues and debugging code are extremely time consuming. Am I missing something or abusing the system or is this the norm?
- 392 3y agoSounds like my experience when tinkering as well. https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html https://adamdrake.com/command-line-tools-can-be-235x-faster-...
- appplication 3y agoThis is normal for spark. We have very little expectation even our senior engineers can optimize their own jobs. It’s a difficult system to learn but can be very powerful. That said, making REST calls inside spark sounds like a recipe to accidentally DDoS your target endpoint.
- thekhatribharat 3y agoHere's a bunch of interesting deep dives: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs...
- deleted 3y ago[deleted]