4 ms·
Not every application is a web application. There is this quaint old framework for distributed computing called Hadoop, maybe you've heard of it? It has a littl
by jeremyjh 8y ago
Not every application is a web application. There is this quaint old framework for distributed computing called Hadoop, maybe you've heard of it? It has a little brother Spark. Anyway the whole point is to "ship code to data". Its an interesting idea, but I don't think it will ever go anywhere.
- scarface74 8y agoAnd surprisingly that use case works with Kinesis and queues etc.
- jeremyjh 8y agoNo, map reduce is not stream processing at all. It’s a transformation/computation over what may be many terabytes of data.
- scarface74 8y agohttps://aws.amazon.com/blogs/compute/ad-hoc-big-data-processing-made-simple-with-serverless-mapreduce/ https://aws.amazon.com/blogs/compute/ad-hoc-big-data-process...
- jeremyjh 8y agoYes its possible, and the SP describes the shortcomings involved in giving up data locality and cross-node interaction.