4 ms·
I work in the oil and gas industry where legacy systems runs on their last breath. Kafka is a fantastic tool and solves a shit ton of problem. We have millions
by olavgg 5y ago
I work in the oil and gas industry where legacy systems runs on their last breath. Kafka is a fantastic tool and solves a shit ton of problem. We have millions of sensors on an offshore installation, these all send data into kafka, where we generate events on new topics from different timeseries. Other data services consume these topics and get data updated in near realtime.
No more daily SQL dumps from offshore to onshore and big batch procedures to genereate outdated events.
- elcano 5y agoWhat legacy systems is the oil and gas using? MQTT? OPC-DA? OPC-UA?
- berkes 5y agoSounds like you have Serious Problems, for which Kafka is a very good solution. For me, Kafka sits in the same area of solutions as Kubernetes, Hadoop clusters, or anything "webscale": you don't need it. Untill you do, but by then you'll (i) have Serious Problems which such systems solve and (ii) the manpower and budgets to fix them. With which I don't mean to avoid Kafka at all costs. By all means, play around with it: if anything, the event-driven will teach you things that make your a better Rails/Flask/WordPress developer if that is what you do.
- fatbird 5y agoI'm in the same situation in the paper making industry. Kafka is an almost perfect match for our needs: high volume, durable storage, decoupled stream processing.