7 ms·
Bruce – Unix domain socket to Kafka daemon
- endersshadow 12y agoInteresting. I really like the idea of using the UNIX sockets to be, essentially, a message queue before going to Kafka. This may be used on my next project.
- dspeterson 12y agoThere is a dedicated thread for monitoring the socket so clients don't block trying to write. In case of transient load spike or slowness in Kafka, the bulk of the queueing should occur inside the daemon, where you can see queue lengths via web interface. God luck with your project :-)
- burke 12y agoWe do a similar thing using SysV Message Queues. Other than being operationally weird due to their age, SysVMQs are actually a really great fit for this problem.
- AceJohnny2 12y agoI hadn't heard of Apache Kafka before. As branding goes, it's hard to do worse. In my mind, Kafka is synonym with complexity, being obscure or byzantine, and as opposite to "useful" as I can imagine. See also: nightmare.
- th0ma5 12y agoMaybe they were trying to channel the idea of the run on sentence which is somewhat more taboo in English than in German, and leads to the Kafka labored style, which he purposefully did. If I try to think of a positive angle, it is that the system can "keep going" perhaps? Anyway, all this stuff is great though.
- dragonwriter 12y agoRun-on sentences aren't taboo, they are just grammatically incorrect. Long sentences may be taboo, but that's a different thing.
- kodablah 12y agoWe have implemented a similar pattern with https://github.com/wavii/darner https://github.com/wavii/darner and a daemon to take the messages and move them to AMQP out of band. It is a healthy pattern to send asynchronous messages to a local daemon before sending them out. Takes the external server out of the path of failure.