4 ms·

I didn't realize that one-thread-per-socket was still something people considered; especially given that many programming languages nowadays have excellent supp

I didn't realize that one-thread-per-socket was still something people considered; especially given that many programming languages nowadays have excellent support for async/await which does a great job at managing multiple concurrent connections within a single process/thread and doesn't have the overhead of context-switching.

deleted 3y ago
[deleted]
flohofwoe 3y ago
In an MMO backend architecture, the way client connections are handled is really just a tiny implemention detail at the edge of the architecture diagram, only affecting a very small part of the overall architecture. Ideally you'd move that stuff out into proxy servers sitting between the clients and the actual game servers with "one connection per client" on the public-network side and "one connection per game server" on the internal-network side. The proxy servers would also deal with validating the incoming traffic (on the message protocol level at least), encryption and of course routing messages between clients and game servers. PS: now that I actually RTFA, the same concept is already mentioned there, it's just called Frontend Server.
otabdeveloper4 3y ago
Context switching will happen regardless of where your thread implementation lives. Reimplementing scheduling in a bespoke fashion in usermode will not give you the gains that you hope for. Basically, learn how an OS works first before trying to reimplement it badly.
- flohofwoe 3y ago
  I guess the idea is that an async/await context switch would always be cheaper than an OS thread context switch. Whether that is actually true probably depends a lot on the specific operating system and async/await runtime (I think there's a lot of mysticism involved seeing async/await as some sort of silver bullet instead of relying on cold hard performance numbers alone).
  - otabdeveloper4 3y ago
    There's no reason to believe it should be cheaper. The contents of the context and the process of switching it is the same regardless of where in the software stack you're doing it. (Async/await was invented in interpreted languages to circumvent their global interpreter locks, which is a whole different problem not related to context switching at all.)
    - ReactiveJelly 3y ago
      That's now how I heard it. Event loops were old hat in GUIs, and web servers like Nginx adopted them so they could service thousands of requests concurrently without allocating stack space for thousands of threads. And the context is not the same. Switching threads requires you to go through the kernel scheduler, so it has to change page tables and stuff in the CPU, right? It lets you write simpler code too, because some events can be handled in the loop without involving mutexes and thread safety. If async/await is, as you claim, a pointless hack for interpreted languages, why did Nginx, written in C, get so much traction with essentially the same architecture?
    - yazaddaruvala 3y ago
      I’m not sure where you’ve gotten this information but I would recommend you do some more research. There is a lot of great info about operating systems - specifically ring0 vs ring1 for the context switch overhead. Meanwhile, only Python and Ruby had GILs. Async/await became popular through JavaScript and C#. Now very popular in Rust as well. None of these runtimes have a GIL. Async/await or Fibers (C++, Go, Java 21) are entirely about context switching overhead.
      - vitus 3y ago
        > Async/await became popular through JavaScript and C#. Now very popular in Rust as well. None of these runtimes have a GIL. To be fair, while typical JS implementations do not have a GIL, it's usually unnecessary due to the runtime being single-threaded. In case anyone else is curious when the above languages grew this feature: (F# piloted this in the 1.x days, back in 2009 or earlier.) It seems like C# developed this feature first in 5.0 in 2012, followed by JS (Dec 2016 in Chrome, widely supported across other major browsers within a few months). Python 3.7 came out in 2018, Ruby 3.0 came out in 2020. Async/await landed in C++20 via coroutines, and Java 21 landed last month. Rust grew this feature in 2018, which landed in the stable channel in 1.39 (Nov 2019). Golang has had goroutines / channels since day 1.
fenesiistvan 3y ago
The one client = one socket assumtion is also bad. You could use a single UDP server socket to handle all clients.
not_the_fda 3y ago
async/await is just hiding the threads from you. There are threads behind the scenes. You want a thread per IO point, whether explicitly or using async/await.
- ReactiveJelly 3y ago
  No it's not. You can set up async runtimes that run on a single thread and service thousands of long-running connections with slow clients. Tokio, the most popular Rust async runtime, starts one thread per CPU core by default, and you can have thousands of tasks running on just a few OS threads.
  - Snacklive 3y ago
    "No it's not" Proceeds to say exactly what the guy above said

deleted 3y ago

[deleted]

flohofwoe 3y ago

In an MMO backend architecture, the way client connections are handled is really just a tiny implemention detail at the edge of the architecture diagram, only affecting a very small part of the overall architecture. Ideally you'd move that stuff out into proxy servers sitting between the clients and the actual game servers with "one connection per client" on the public-network side and "one connection per game server" on the internal-network side. The proxy servers would also deal with validating the incoming traffic (on the message protocol level at least), encryption and of course routing messages between clients and game servers. PS: now that I actually RTFA, the same concept is already mentioned there, it's just called Frontend Server.

otabdeveloper4 3y ago

Context switching will happen regardless of where your thread implementation lives. Reimplementing scheduling in a bespoke fashion in usermode will not give you the gains that you hope for. Basically, learn how an OS works first before trying to reimplement it badly.

flohofwoe 3y ago
I guess the idea is that an async/await context switch would always be cheaper than an OS thread context switch. Whether that is actually true probably depends a lot on the specific operating system and async/await runtime (I think there's a lot of mysticism involved seeing async/await as some sort of silver bullet instead of relying on cold hard performance numbers alone).
- otabdeveloper4 3y ago
  There's no reason to believe it should be cheaper. The contents of the context and the process of switching it is the same regardless of where in the software stack you're doing it. (Async/await was invented in interpreted languages to circumvent their global interpreter locks, which is a whole different problem not related to context switching at all.)
  - ReactiveJelly 3y ago
    That's now how I heard it. Event loops were old hat in GUIs, and web servers like Nginx adopted them so they could service thousands of requests concurrently without allocating stack space for thousands of threads. And the context is not the same. Switching threads requires you to go through the kernel scheduler, so it has to change page tables and stuff in the CPU, right? It lets you write simpler code too, because some events can be handled in the loop without involving mutexes and thread safety. If async/await is, as you claim, a pointless hack for interpreted languages, why did Nginx, written in C, get so much traction with essentially the same architecture?
  - yazaddaruvala 3y ago
    I’m not sure where you’ve gotten this information but I would recommend you do some more research. There is a lot of great info about operating systems - specifically ring0 vs ring1 for the context switch overhead. Meanwhile, only Python and Ruby had GILs. Async/await became popular through JavaScript and C#. Now very popular in Rust as well. None of these runtimes have a GIL. Async/await or Fibers (C++, Go, Java 21) are entirely about context switching overhead.
    - vitus 3y ago
      > Async/await became popular through JavaScript and C#. Now very popular in Rust as well. None of these runtimes have a GIL. To be fair, while typical JS implementations do not have a GIL, it's usually unnecessary due to the runtime being single-threaded. In case anyone else is curious when the above languages grew this feature: (F# piloted this in the 1.x days, back in 2009 or earlier.) It seems like C# developed this feature first in 5.0 in 2012, followed by JS (Dec 2016 in Chrome, widely supported across other major browsers within a few months). Python 3.7 came out in 2018, Ruby 3.0 came out in 2020. Async/await landed in C++20 via coroutines, and Java 21 landed last month. Rust grew this feature in 2018, which landed in the stable channel in 1.39 (Nov 2019). Golang has had goroutines / channels since day 1.

fenesiistvan 3y ago

The one client = one socket assumtion is also bad. You could use a single UDP server socket to handle all clients.

not_the_fda 3y ago

async/await is just hiding the threads from you. There are threads behind the scenes. You want a thread per IO point, whether explicitly or using async/await.

ReactiveJelly 3y ago
No it's not. You can set up async runtimes that run on a single thread and service thousands of long-running connections with slow clients. Tokio, the most popular Rust async runtime, starts one thread per CPU core by default, and you can have thousands of tasks running on just a few OS threads.
- Snacklive 3y ago
  "No it's not" Proceeds to say exactly what the guy above said