4 ms·

> Erlang's message passing style is interesting here. it just hides who owns the message data object though, no ? The problem is still here. e.g. for instance

> Erlang's message passing style is interesting here. it just hides who owns the message data object though, no ? The problem is still here. e.g. for instance I work on a C++ software where there is a lot of message passing across threads, but the OS thread on which "malloc" and "free" are called for the messages and any data they may contain really, really matters - some threads must never call "malloc" or "free" under any circumstance. Would I be able to manage this with Erlang, knowing that every thread should be able to send message to pretty much any other ? Also, looking at that code... https://www.erlang.org/blog/message-passing/ https://www.erlang.org/blog/message-passing/ I don't really see what this gains me over the trivial C++ solution: processing_thread.message_queue.send([=] (Context& ctx) { int x = ... some computation ... ctx.ui_thread.message_queue.send([x] { ... show some info ... }); }); which can become even simpler with coroutines in C++20 if one does not like callback style

eru 4y ago
> it just hides who owns the message data object though, no ? The problem is still here. Erlang messages have a semantic as-if there are copied. (Because that's the only way to do it over a network, and Erlang wants to be agnostic over whether two 'processes' are on the same machine or not.) Erlang doesn't need to worry too much about ownership, because it has a garbage collector. If you want to explicitly control which thread uses malloc/free, then Erlang isn't at the right level of language in this case.
- mwcampbell 4y ago
  > Erlang messages have a semantic as-if there are copied. (Because that's the only way to do it over a network, and Erlang wants to be agnostic over whether two 'processes' are on the same machine or not.) I think that approach could be called pessimization. The assumption that some components are on the same machine, or better yet the same address space, unlocks some significant optimizations. Sometimes you can be sure that a particular system won't need to scale beyond one machine. And of course, the more efficient the implementation is, the bigger a system can get without having to scale beyond one machine. (Aside: I was reading earlier today about how Uber heavily uses microservices. I wonder if Uber could be a monolith running on one machine given the right architecture and implementation.)
  - eru 4y ago
    > Aside: I was reading earlier today about how Uber heavily uses microservices. I wonder if Uber could be a monolith running on one machine given the right architecture and implementation. Throughput-wise, that might be possible for all I know. For latency reasons, you'd probably want to have at least one monolith per city. Similar objections apply for redundancy and resilience. Splitting your system into individual services ensures that there are well-defined interfaces between the different parts. In principle, you can always wring more performance out of a system, if you are allowed to break down these interfaces. But that makes maintaining and further developing these systems harder and harder to do for mere mortals. Have a look at how much of a kludgy mess our genome is for an example. --- In principle, you could write something as individual services but compile them into a single binary that runs in a single memory space on bare metal (see unikernel architecture). A sufficiently strong type system could make sure that any one service being buggy wouldn't bring down the rest of the application. You could probably even work on a system like the above that still allows you to replace individual parts without restarting everything else. This could be much faster and smaller than traditional micro-services that live in separated processes or even machines, but even this utopian system would still be hampered by the conceptual walls between services that can only talk to each other via well-defined interfaces. (Even if your compiler is smart enough to do a lot of inlining etc.) --- The discussion reminds me of exokernels. Have you heard of them? Basically the idea is that traditionally operating systems are supposed to have at least two functions: abstract hardware, and safely multiplex different uses and users. Serving two masters means serving neither of them well. So the exokernel people say: let's just use libraries for abstracting over hardwarde, and let the operating system present the hardware as raw as possible and concentrate on safely multiplexing. See eg https://www.classes.cs.uchicago.edu/archive/2019/winter/33100-1/papers/hotos-jeremiad.pdf https://www.classes.cs.uchicago.edu/archive/2019/winter/3310...
  - deepsun 4y ago
    > I wonder if Uber could be a monolith running on one machine given the right architecture and implementation. You seem to simplify Uber's tasks to only ridesharing. But there are many other typical tasks for large tech orgs. For example (never worked at Uber, taking from my imagination): 1. Handle ride-sharing business (the easiest part maybe). 2. Make a pipeline to extract data for legal requests from courts, processing all Uber's historical logs. 3. CEO asked to make a forecast (like chart), using billing data from their financial processor. 4. Serve video and quizes and check tests during driver's onboarding process. 5. Monitor/alert internal network for network security purposes. 6. Gather some analytics over Uber's cloud resources spent and give recommendations on where to optimize. Not sure how to pack those all into a monolith.

eru 4y ago

> it just hides who owns the message data object though, no ? The problem is still here. Erlang messages have a semantic as-if there are copied. (Because that's the only way to do it over a network, and Erlang wants to be agnostic over whether two 'processes' are on the same machine or not.) Erlang doesn't need to worry too much about ownership, because it has a garbage collector. If you want to explicitly control which thread uses malloc/free, then Erlang isn't at the right level of language in this case.

mwcampbell 4y ago
> Erlang messages have a semantic as-if there are copied. (Because that's the only way to do it over a network, and Erlang wants to be agnostic over whether two 'processes' are on the same machine or not.) I think that approach could be called pessimization. The assumption that some components are on the same machine, or better yet the same address space, unlocks some significant optimizations. Sometimes you can be sure that a particular system won't need to scale beyond one machine. And of course, the more efficient the implementation is, the bigger a system can get without having to scale beyond one machine. (Aside: I was reading earlier today about how Uber heavily uses microservices. I wonder if Uber could be a monolith running on one machine given the right architecture and implementation.)
- eru 4y ago
  > Aside: I was reading earlier today about how Uber heavily uses microservices. I wonder if Uber could be a monolith running on one machine given the right architecture and implementation. Throughput-wise, that might be possible for all I know. For latency reasons, you'd probably want to have at least one monolith per city. Similar objections apply for redundancy and resilience. Splitting your system into individual services ensures that there are well-defined interfaces between the different parts. In principle, you can always wring more performance out of a system, if you are allowed to break down these interfaces. But that makes maintaining and further developing these systems harder and harder to do for mere mortals. Have a look at how much of a kludgy mess our genome is for an example. --- In principle, you could write something as individual services but compile them into a single binary that runs in a single memory space on bare metal (see unikernel architecture). A sufficiently strong type system could make sure that any one service being buggy wouldn't bring down the rest of the application. You could probably even work on a system like the above that still allows you to replace individual parts without restarting everything else. This could be much faster and smaller than traditional micro-services that live in separated processes or even machines, but even this utopian system would still be hampered by the conceptual walls between services that can only talk to each other via well-defined interfaces. (Even if your compiler is smart enough to do a lot of inlining etc.) --- The discussion reminds me of exokernels. Have you heard of them? Basically the idea is that traditionally operating systems are supposed to have at least two functions: abstract hardware, and safely multiplex different uses and users. Serving two masters means serving neither of them well. So the exokernel people say: let's just use libraries for abstracting over hardwarde, and let the operating system present the hardware as raw as possible and concentrate on safely multiplexing. See eg https://www.classes.cs.uchicago.edu/archive/2019/winter/33100-1/papers/hotos-jeremiad.pdf https://www.classes.cs.uchicago.edu/archive/2019/winter/3310...
- deepsun 4y ago
  > I wonder if Uber could be a monolith running on one machine given the right architecture and implementation. You seem to simplify Uber's tasks to only ridesharing. But there are many other typical tasks for large tech orgs. For example (never worked at Uber, taking from my imagination): 1. Handle ride-sharing business (the easiest part maybe). 2. Make a pipeline to extract data for legal requests from courts, processing all Uber's historical logs. 3. CEO asked to make a forecast (like chart), using billing data from their financial processor. 4. Serve video and quizes and check tests during driver's onboarding process. 5. Monitor/alert internal network for network security purposes. 6. Gather some analytics over Uber's cloud resources spent and give recommendations on where to optimize. Not sure how to pack those all into a monolith.