As with most questions like this, the answer is "Yes, of course, sometimes." I work on IBM Streams, which is a distributed stream processing system [1,2]. When

As with most questions like this, the answer is "Yes, of course, sometimes." I work on IBM Streams, which is a distributed stream processing system [1,2]. When the cost of communicating between hosts is greater than the cost of your computation, distributed processing is not going to help. When your computation is basically a form of word count, then, yes, using multiple hosts may not help. But if your computation is, say, online speech to text for a large call center [3], then the application will eat just about as much compute resources as you can throw at it. That's going to mean using multiple machines, even if each machine is quite large. As a plug for our system: we scale down as well as up. If you only have a single host, we will run the entire application in a single process, using threads as needed to achieve parallelism using either dedicate threads or a lightweight operator scheduler [4]. Users can configure which without changing application logic. If you have multiple machines, we will automatically partition your application across those machines for you [5]. By default, we do this all for you, without needing any input. But if you want to explicitly control what executes where, and using how many host machines, that's easy to configure, also without changing application logic. [1] https://www.ibm.com/cloud/streaming-analytics https://www.ibm.com/cloud/streaming-analytics [2] http://ibmstreams.github.io/ http://ibmstreams.github.io/ [3] https://www.ibm.com/case-studies/verizon https://www.ibm.com/case-studies/verizon [4] Low-Synchronization, Mostly Lock-Free, Elastic Scheduling for Streaming Runtimes. Scott Schneider and Kun-Lung Wu. PLDI 2017. http://www.scott-a-s.com/files/pldi2017_lf_elastic_scheduling.pdf http://www.scott-a-s.com/files/pldi2017_lf_elastic_schedulin... [5] Automatic Fusion and Threading. https://developer.ibm.com/streamsdev/docs/automatic-fusion-and-threading/ https://developer.ibm.com/streamsdev/docs/automatic-fusion-a...