4 ms·
For many tasks a spark cluster is overkill. Streaming will get you a really long way before you have to tackle running a while spark cluster. And you'll probabl
by zaphar 7y ago
For many tasks a spark cluster is overkill. Streaming will get you a really long way before you have to tackle running a while spark cluster. And you'll probably be able to still do it on your laptop.
- threeseed 7y agoYou can't just apply a streaming paradigm to a batch style workload. And 90% of all tasks are batch orientated.
- zaphar 7y agoBatch is just streaming in really large chunks. And besides if you are messing around with data on your laptop you probably aren't at the stage of working out a batch workload yet. Jumping immediately to this needs to be a batch workload before you have had a chance to play with it is probably overkill.