7 ms·

Given that Kafka books are about impenetrable bureaucratic worlds, why is Kafka called Kafka? Sounds like it’s a tool to make sure your infrastructure is hard t

Given that Kafka books are about impenetrable bureaucratic worlds, why is Kafka called Kafka? Sounds like it’s a tool to make sure your infrastructure is hard to comprehend.

yongjik 5y ago
It does tend to prompt a developer onto a sudden fit of self-awareness ("Why am I reading this code? Why are we using this? What am I doing with my life?"), so I think it's quite aptly named.
grzm 5y ago
> Jay Kreps chose to name the software after the author Franz Kafka because it is "a system optimized for writing", and he liked Kafka's work. https://en.wikipedia.org/wiki/Apache_Kafka#History https://en.wikipedia.org/wiki/Apache_Kafka#History
0des 5y ago
You know it's just young people working at these bigcorps making our tooling, right?
zwkrt 5y ago
Comment I made last summer: It’s rare a piece of tech has a more fitting name! “Is your orgs politics so complicated that direct team-to-team communication has broken down? Is your business process subject to unannounced violent change? Bogged down by consistent DB schemas and API versioning? Tired of retries on failed messages? Introducing Kafka by Apache: an easy to use, highly configurable, persistently stored ephemeral centralized federated event based messaging data analytics Single Source of Truth as a Service that can scale with any enterprise-sized dysfunction!”
- abledon 5y ago
  i wish there was some CLI like homebrew / manpages... but had paragraphs like this comment for each techonology.... so when your team starts talking about it you can look it up. > hninsight -q kafka
  - jacquesm 5y ago
    I often do this when I find some interesting new tech: https://www.google.com/search?q=site:news.ycombinator.com%20productname https://www.google.com/search?q=site:news.ycombinator.com%20... it almost always turns up good stuff.
- almeria 5y ago
  Anything more about Kafka you can tell us? Seriously, the one time I was in a situation where much of the team seemed hellbent on this "just put all in Kafka" idea (without really understanding why, exactly) the arguments they came up with were not too dissimilar from what you've shared with us above. It all seemed to come down to "OMG databases are hard, schemas are hard, our customers don't understand the data they're shoving at us. But Kafka will take care of all of that for us. Because, you know, shiny." That said I'd still like to have a more ... balanced understanding of why Kafka may not necessarily be The Answer, and/or have more hidden complexity or other negative tradeoffs than we may have bargained for.
  - yongjik 5y ago
    I only had a brief exposure to it, but my impression is that it's sort of a message queue optimized for very large data (TB or more). So, for example, there's no way to easily answer questions like "How many requests did server X generate between 1pm and 2pm and how many of them were served by server Y?" because when your data doesn't fit in a single machine, supporting such queries requires a lot of bookkeeping. If you never need them, you don't want to pay for them. Of course, when have a few megabytes of data and you route it through Kafka, then all you get is an opaque message queue where you can't see which message went from where to where. Good luck debugging any issues. But, hey, you got to use Kafka.
    - EdwardDiego 5y ago
      > for example, there's no way to easily answer questions like "How many requests did server X generate between 1pm and 2pm and how many of them were served by server Y?" There's many ways to answer that using data streamed over Kafka - ingest it into your preferred query engine, go query it. Kafka is a distributed log, that's it.
  - zwkrt 5y ago
    My take was humorous but it didn’t hide anything. Kafka was built so that LinkedIn could shove all its real-time click data through a single funnel—terabytes upon terabytes. It has since been evangelized and created a cottage industry of Confluent salespeople who will give your manager a course in how to lobby their engineers into using Kafka. Have scaling problems? Kafka. Have business events that need to be ordered? Kafka! Have “changing schemas”? KAFKA!! I’m always suspicious when a company gives a product away for free tbut then charges $$$ for “support”. I worked for a high profile recently-failed project from a company that rhymes with Brillo, and our data was just beginning to be too big for google sheets (!). However, we were also having organizational problems because the higher ups were seeing the failing project losing money so they of course decided to hire 100 extra engineers. Our communications (both human and programmatic) were failing and the confluent salespeople began circling like buzzards. Of course by the time it was suggested we we use it the project was already 6 months past the point of no return. My advice is that if your data fits in a database, use a database. Anyone who says that isn’t scalable should have to tell you the actual reason it doesn’t scale and the number of requests/users/GBs/uptime/ etc that is the bottleneck.
- damontal 5y ago
  In The Castle, any phone call from the town to the castle causes all telephones in the castle to ring. Anyone can pick up. I thought this was the origin of the name but I’m probably wrong.
  - wodenokoto 5y ago
    I’ve never read the castle so I’m sure I’m missing some context, but isn’t that how landline phones always work?
    - damontal 5y ago
      The castle is full of administrative offices and various bureaucratic departments that run the town. The protagonist gets the number of a bureaucrat and wants to contact him at his office in the castle. The fact that the number he’s given causes all phones in all offices in the castle to ring, and the people who answer don’t know or care about what he’s after, just adds a layer of confusion and difficulty to his goal of getting to the castle.
Lio 5y ago
I went to talk by RMS some time in the late 90s, early 00s. The one thing I remember RMS saying after all this time is that “the whole point of writing software is so that you can give it a funny name”. All things considered, I still think that’s good advice.
- perl4ever 5y ago
  The whole point of naming things is so you can find them later. Why can't people follow the simple rule of not reusing any word or name that's already in use? They don't seem to have a problem with racehorses. In fact, why not get the most comprehensive list of racehorse names to date, and start using them for new software projects - then nobody will have to be creative until they run out.
  - egypturnash 5y ago
    "I'm an expert in Potoooooooo". https://en.wikipedia.org/wiki/Potoooooooo https://en.wikipedia.org/wiki/Potoooooooo
    - jazzyjackson 5y ago
      wonder if the creators of phhhoto had this horse in mind
    - nzach 5y ago
      >He is now best known for the unusual spelling of his name, pronounced 'Potatoes'.
- Jiro 5y ago
  Names are used to communicate. If you have to tell people "it's pronounced G-new" like a Monty Python sketch, you make it harder for people to take you seriously.
marcosdumay 5y ago
That's only one of his famous books. There is another about bugs appearing for no good reason on unexpected places.
- xenocyon 5y ago
  Impenetrable bureaucracies are actually the central theme of two out of three of his full-length novels. (Incidentally, at least in my opinion his three novels are his finest work, but they seem to attract much less popular attention than his short stories and novellas.)
EdwardDiego 5y ago
Sadly it was so named because Kafka as a writer Jay Kreps admired, and Kafka was optimised for writes. That's it :(

Given that Kafka books are about impenetrable bureaucratic worlds, why is Kafka called Kafka? Sounds like it’s a tool to make sure your infrastructure is hard to comprehend.

yongjik 5y ago

It does tend to prompt a developer onto a sudden fit of self-awareness ("Why am I reading this code? Why are we using this? What am I doing with my life?"), so I think it's quite aptly named.

grzm 5y ago

> Jay Kreps chose to name the software after the author Franz Kafka because it is "a system optimized for writing", and he liked Kafka's work. https://en.wikipedia.org/wiki/Apache_Kafka#History https://en.wikipedia.org/wiki/Apache_Kafka#History

0des 5y ago

You know it's just young people working at these bigcorps making our tooling, right?

zwkrt 5y ago

Comment I made last summer: It’s rare a piece of tech has a more fitting name! “Is your orgs politics so complicated that direct team-to-team communication has broken down? Is your business process subject to unannounced violent change? Bogged down by consistent DB schemas and API versioning? Tired of retries on failed messages? Introducing Kafka by Apache: an easy to use, highly configurable, persistently stored ephemeral centralized federated event based messaging data analytics Single Source of Truth as a Service that can scale with any enterprise-sized dysfunction!”

abledon 5y ago
i wish there was some CLI like homebrew / manpages... but had paragraphs like this comment for each techonology.... so when your team starts talking about it you can look it up. > hninsight -q kafka
- jacquesm 5y ago
  I often do this when I find some interesting new tech: https://www.google.com/search?q=site:news.ycombinator.com%20productname https://www.google.com/search?q=site:news.ycombinator.com%20... it almost always turns up good stuff.
almeria 5y ago
Anything more about Kafka you can tell us? Seriously, the one time I was in a situation where much of the team seemed hellbent on this "just put all in Kafka" idea (without really understanding why, exactly) the arguments they came up with were not too dissimilar from what you've shared with us above. It all seemed to come down to "OMG databases are hard, schemas are hard, our customers don't understand the data they're shoving at us. But Kafka will take care of all of that for us. Because, you know, shiny." That said I'd still like to have a more ... balanced understanding of why Kafka may not necessarily be The Answer, and/or have more hidden complexity or other negative tradeoffs than we may have bargained for.
- yongjik 5y ago
  I only had a brief exposure to it, but my impression is that it's sort of a message queue optimized for very large data (TB or more). So, for example, there's no way to easily answer questions like "How many requests did server X generate between 1pm and 2pm and how many of them were served by server Y?" because when your data doesn't fit in a single machine, supporting such queries requires a lot of bookkeeping. If you never need them, you don't want to pay for them. Of course, when have a few megabytes of data and you route it through Kafka, then all you get is an opaque message queue where you can't see which message went from where to where. Good luck debugging any issues. But, hey, you got to use Kafka.
  - EdwardDiego 5y ago
    > for example, there's no way to easily answer questions like "How many requests did server X generate between 1pm and 2pm and how many of them were served by server Y?" There's many ways to answer that using data streamed over Kafka - ingest it into your preferred query engine, go query it. Kafka is a distributed log, that's it.
- zwkrt 5y ago
  My take was humorous but it didn’t hide anything. Kafka was built so that LinkedIn could shove all its real-time click data through a single funnel—terabytes upon terabytes. It has since been evangelized and created a cottage industry of Confluent salespeople who will give your manager a course in how to lobby their engineers into using Kafka. Have scaling problems? Kafka. Have business events that need to be ordered? Kafka! Have “changing schemas”? KAFKA!! I’m always suspicious when a company gives a product away for free tbut then charges $$$ for “support”. I worked for a high profile recently-failed project from a company that rhymes with Brillo, and our data was just beginning to be too big for google sheets (!). However, we were also having organizational problems because the higher ups were seeing the failing project losing money so they of course decided to hire 100 extra engineers. Our communications (both human and programmatic) were failing and the confluent salespeople began circling like buzzards. Of course by the time it was suggested we we use it the project was already 6 months past the point of no return. My advice is that if your data fits in a database, use a database. Anyone who says that isn’t scalable should have to tell you the actual reason it doesn’t scale and the number of requests/users/GBs/uptime/ etc that is the bottleneck.
damontal 5y ago
In The Castle, any phone call from the town to the castle causes all telephones in the castle to ring. Anyone can pick up. I thought this was the origin of the name but I’m probably wrong.
- wodenokoto 5y ago
  I’ve never read the castle so I’m sure I’m missing some context, but isn’t that how landline phones always work?
  - damontal 5y ago
    The castle is full of administrative offices and various bureaucratic departments that run the town. The protagonist gets the number of a bureaucrat and wants to contact him at his office in the castle. The fact that the number he’s given causes all phones in all offices in the castle to ring, and the people who answer don’t know or care about what he’s after, just adds a layer of confusion and difficulty to his goal of getting to the castle.

Lio 5y ago

I went to talk by RMS some time in the late 90s, early 00s. The one thing I remember RMS saying after all this time is that “the whole point of writing software is so that you can give it a funny name”. All things considered, I still think that’s good advice.

perl4ever 5y ago
The whole point of naming things is so you can find them later. Why can't people follow the simple rule of not reusing any word or name that's already in use? They don't seem to have a problem with racehorses. In fact, why not get the most comprehensive list of racehorse names to date, and start using them for new software projects - then nobody will have to be creative until they run out.
- egypturnash 5y ago
  "I'm an expert in Potoooooooo". https://en.wikipedia.org/wiki/Potoooooooo https://en.wikipedia.org/wiki/Potoooooooo
  - jazzyjackson 5y ago
    wonder if the creators of phhhoto had this horse in mind
  - nzach 5y ago
    >He is now best known for the unusual spelling of his name, pronounced 'Potatoes'.
Jiro 5y ago
Names are used to communicate. If you have to tell people "it's pronounced G-new" like a Monty Python sketch, you make it harder for people to take you seriously.

marcosdumay 5y ago

That's only one of his famous books. There is another about bugs appearing for no good reason on unexpected places.

xenocyon 5y ago
Impenetrable bureaucracies are actually the central theme of two out of three of his full-length novels. (Incidentally, at least in my opinion his three novels are his finest work, but they seem to attract much less popular attention than his short stories and novellas.)

EdwardDiego 5y ago

Sadly it was so named because Kafka as a writer Jay Kreps admired, and Kafka was optimised for writes. That's it :(