3 ms·
The fact people opposed to your idea / work means it is valuable enough for people to say something against and not ignore it. I must confess I miss native exe
by eb0la 7y ago
The fact people opposed to your idea / work means it is valuable enough for people to say something against and not ignore it.
I must confess I miss native execution of (big)Data jobs.
I know moving jvm bytecode between nodes to be run is portable, but nowadays nobody has mixed architecture (intel/mips/sparc/arm) servers, so... why do we need a bytecode execution layer?
Maybe I am too critic, but bare metal - hypervisor - jvm - app looks too much layers to me.
- s_Hogg 7y agoTotally, I think there are a lot of cases where all that machinery is actually completely superfluous anyway because thinking about the problem you're trying to solve could lead you to a way of doing it that doesn't need all that compute. There's a blog post about this I read a while back where someone did a simple graph search using spark and then did it in a slightly smarter way on a commodity laptop and it outran spark on a large-ish amount of data. Wish I could find it.
- buckminster 7y agoThis one? http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...
- s_Hogg 7y agoYep, that's the one!
- eb0la 7y agoI remember 4-5 years ago I tried to slice a big CSV file with Spark and took soo loong it didn't finished after some hours. Later on the day I realized it could be done from the command line with 'cut' and 'grep'. Took less than 15 minutes.
- andoriyu 7y agoI wonder how long it would have taken with `cut` and `ripgrep`