7 ms·
They didn't replace the sticky part of the logging pipeline. "Logging" is * ingestion / message schema * queue * ETL * storage / analytics
by cduzz 5y ago
They didn't replace the sticky part of the logging pipeline.
"Logging" is
* ingestion / message schema
* queue
* ETL
* storage / analytics
The stickiest part of a logging pipeline is the ingestion and message format (scribe in this case); those are the gravity and speed of light of your organization, embedded at the lowest layer of everything everywhere.
Three years from now they could / will write a new blog post "today we're open sourcing smackhouse, a log analytics platform based on smooshing logs into clickhouse! We ETL the data using this zany rust thing (zaptl) from kafka into clickhouse, and made some zippy UI on top. We still stream logs into hadoop and goolens and splunk; any teams that want to use these other infrastructures just set a config in a fargml setting to zaptl to request the log stream and sample rate."
They're not using all of splunk; they're only using it to solve one of the harder parts of logging -- log analytics. They're not using it to solve the stickiest part of logging.
Good for them. BTW -- this reads like a cross between a proof of life video and a newsletter written by the intern. Glad it's earning them a huge discount from splunk.
(edited for formatting)
- tantalor 5y agoWhich of these words did you just make up
- kingcharles 5y agoIf you're not already using smackhouse to smoosh your logs into clickhouse, how can you even call yourself a developer, bro? Who do you code for? Standard Oil? Ma Bell?