Kafka deserves some credits here, because it destroys these kind of architectural problems however. It completely changes how to think about service based arch

Kafka deserves some credits here, because it destroys these kind of architectural problems however. It completely changes how to think about service based architectures. In fact, with a properly designed architecture sitting around Kafka APIs and a plethora of queues magically disappear, because instead of function and request-response driven the system becomes reactively data driven, sitting around a single bus. This makes designing actual services a lot easier because instead of thinking about a few hundred function names all you need to think about are actual data flows (data comes in at a central point -> gets transformed -> gets moved out at a central point). As a added bonus it scales like crazy, to the point that hammering through a terabyte of data looks like nothing. A great talk from Ben Stopford (Confluent employee, so mostly Kafka centric) about this subject can be found here: https://www.slideshare.net/benstopford/devoxx-london-2017-rethinking-services-withstatefulstreams https://www.slideshare.net/benstopford/devoxx-london-2017-re...