4 ms·
I agree but if their problem is that they want a scalable solution for processing a growing amount of data cheaply and robustly and potentially doing more compl
by faizshah 11y ago
I agree but if their problem is that they want a scalable solution for processing a growing amount of data cheaply and robustly and potentially doing more complex work in the future (it's a growing business). This is basically the tagline of the current batch of hadoop ecosystem projects.
Investing in working with the hadoop ecosystem is a much better long term solution (if this is the case).
- stuartaxelowen 11y agoIMO investing in a hadoop system constrains you - make your data accessible via S3, Kafka, and most importantly, a good data model, and you can attach the right data processing tool when a problem that requires it comes along.
- faizshah 11y agoI agree that a good data model is key, in fact the way you structure your data is probably the most important thing. But the reason that the hadoop ecosystem is so good is because it's a growing ecosystem with a lot of momentum and a focus on processing large and growing amounts of data at scale and in a distributed/parallel fashion. If this is what you need the ecosystem is perfect.
- haddr 11y agoIf at some point of time you use hbase, using hadoop on top of that becomes a straightforward task. Both integrate very well with each other. But I think even without hadoop you can do. If the processing is relatively light (e.g. calculating some aggregates, etc) you can do it in near real time just connecting with another consumer to kafka topic.