3 ms·
This is exactly what I'm talking about. Even layman people can use their regular computers for more interesting activities like machine learning on decent sized
by switch33 13y ago
This is exactly what I'm talking about. Even layman people can use their regular computers for more interesting activities like machine learning on decent sized data if it wasn't so focused on just throw your cloud instance up using hadoop and run stuff that way.
There are many people out there that can accomplish some of these tasks with better written code. There is also probably a need for a better hadoop. One that is less about clusters/multiple cpus and more about rented out bigger power computers.
- asperous 13y agoHadoop is specifically for clustered computer, it doesn't really make sense otherwise. Like the speaker said, it's really simplified (though you can do a lot of stuff with map/reduce). In some ways the guy is probably blowing some steam, simply grabbing a bigger computer isn't always the best option, that's why clustered computing was even invented. I do think he has a point that people reach for it sooner than they need to. I've made that mistake before, and I just ended up settling for the pipeline approach.
- switch33 13y agoThe other point i'm trying to really make is maybe hadoop does serve some purpose but there could be an easier/better approach for parallelism. If we focused harder on reliability and another service "like hadoop" would probably be created but for non-cluster/multicpu. And doing some of the decently sized tasks we would have better, faster, and more reliable tools at hand.