2 ms·
I'm a little sad that neither article (I've read some more beside the ones linked there) about mentions that the idea of logging here and processing there is pr
by erikb 12y ago
I'm a little sad that neither article (I've read some more beside the ones linked there) about mentions that the idea of logging here and processing there is pretty much the Unix principle of each program handling text (mostly lines of colon/comma separated data records) as input and creating another stream of text as output. Also I don't know if the idea of a central data storage (which is Krep's suggestion as far as I can understand) is really the solution to all our problems in a time of distributed everything.
- fintler 12y agoHaving several datacenters running Kafka with MirrorMakers between then (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330 https://cwiki.apache.org/confluence/pages/viewpage.action?pa...) is far from being central data storage, it's more like a graph of (reasonably) ordered data distributed across the globe, which is available to any application. I don't really see much of a relationship to unix text stream processing. Maybe in something like a Kafka -> Samza -> Kafka style system -- but even then, it feels like a stretch. There's much more of a focus on the ordering of messages rather than the concept of log here, process there.