3 ms·

> When you bring in technologies like Kafka to orchestrate this, you'll end up with a more reliable system that you can fix if something goes wrong. It depends

> When you bring in technologies like Kafka to orchestrate this, you'll end up with a more reliable system that you can fix if something goes wrong. It depends... If your services are servicing typical user requests, and expect responses in O(<1s), then eventing architectures are exactly the opposite of what you want. Using message brokers (Kafka) as an ersatz network layer is the path to sadness -- you want backpressure, RPC semantics, etc. https://programmingisterrible.com/post/162346490883/how-do-you-cut-a-monolith-in-half https://programmingisterrible.com/post/162346490883/how-do-y... If you're doing ETC/event semantics, the game is different. But very few orgs actually work that way.

monksy 5y ago
> If your services are servicing typical user requests, and expect responses in O(<1s), then eventing architectures are exactly the opposite of what you want. Using message brokers (Kafka) as an ersatz network layer is the path to sadness -- you want backpressure, RPC semantics, etc. Correct. Most cases you need to see what the current stored state is. Processing can happen in the background and update that state without being requested. I'm assuming that you're architecture your system as having an ingest service that brings everything into the topic, multiple apps that'll refine/enhance, the data, and then an app that sends it out to a more permanent storage. Your service endpoint should look at the premant storage to make a response. (Or read and post out updates via a websocket if you want frequent updates to an existing page) Kafka is not a network technology.