15 ms·

I know what building an MMO is like, having worked on one 10 years ago that was already 10 years old and still exists today (small audience though). Building s

I know what building an MMO is like, having worked on one 10 years ago that was already 10 years old and still exists today (small audience though). Building something like these people are talking about requires more $ than a Marvel superhero movie to get anywhere. We could manage about 100 simultaneous players visible to each other in a reasonable time but no more; the internet is not fast enough to manage more than that many "people" with anything resembling real behavior, even then its hard to maintain the illusion of fluidity with the random latency of so many streams using prediction. Supporting millions of users like these people claim is fantasy even if you only saw a handful. Maintaining state in a single world at that volume is impossible with real people (you can fake a lot of "AI" characters but not actual players). There is reason why most MMO and similar games limit the number of players, or like Eve throttle the game time and slow everything down if too many ships appear in the same place (not possible in a "on ground" sort of world). Also building something of this magnitude and even getting it to do even a small subset of what they want requires people with lots of experience, as this is highly complex and specialized programming, not to mention an enormous amount of art creation, lighting, audio, story and other content, unless you go the No Man's Sky route and generate everything. But they don't support huge numbers of"local" players either.

650REDHAIR 5y ago
See star citizen for a recent example
- reiichiroh 5y ago
  Everything will work when /if server meshing works
- reiichiroh 5y ago
  Everything will work when or rather if server meshing works
rcxdude 5y ago
Exactly, on the server side player interaction is an N² problem (where N is the number of players who are currently interacting). Each of N players need updates on the (N-1) other players they can see. And without strong in-game reasons not to, players will test the limits, no matter how efficient your code (see EVE where despite them making huge improvements to their servers and code as well as already using an extremely low update rate, player battles are still basically limited by how many you can get into a system before the server topples over).
- deepnotderp 5y ago
  Is this really true? I would imagine you get something more like Zipf’s law in practice, N log N. Even if there were interactions between everyone, presumably you could still exploit locality, like: https://homes.cs.washington.edu/~pedrod/papers/aaai10b.pdf https://homes.cs.washington.edu/~pedrod/papers/aaai10b.pdf
  - fulafel 5y ago
    In practice in other existing MMOs, you have bounded areas of the game where people are in view of each oher, with player counts designed to work within the technical constrians of the tech. Then teleport/portal around to get to other areas. But even within an area you can cut down on the n^2 in various ways, with one set of estabilished techniques cutting down server side compute and other set for network communications.
angrais 5y ago
Could you explain how world of Warcraft works then? You can certainly have more than 200 in one place at once... Are they doing some tricks to make this appear as if it's working but in reality it is not? Would love to hear your opinion.
- ElectricalUnion 5y ago
  > 200 in one place 200 players in one map instance. But not 200 visible player with interactions refreshing on real time. Most likely you see diminishing level of details, if anything besides complete fabrication or no data at all after a certain visibility range threshold.
  - throwaway316943 5y ago
    The spiffing Brit managed to get 500 people into a single Factorio game, not all visible at the same time but all interacting with the environment. It was an interesting video, the performance was fairly good.
    - swiley 5y ago
      I saw that video, the game definitely struggled. It's certainly doable though.
    - imtringued 5y ago
      Buildings are more CPU intensive than players unless those players are all busy fighting. If they just mine or place buildings they simply don't matter.
  - p1necone 5y ago
    This doesn't really matter though, after a certain point there isn't much difference to players between 50 people and 200 people within visible distance. You can just stop showing others past some cap, or throttle position updates to like 1 every 20 seconds or so and fill in the gaps with AI.
MattieTK 5y ago
This was part of the exciting suggestion of the original Stadia trailers: with the entire game and all the interactions happening in Google datacentres, you would only need to receive a rendered version, and suddenly all this would be possible. Of course none of that has happened and Google folded their games studio, which is par for them, but does go some way to reinforcing this point that there's a reason this stuff isn't done.
- zolland 5y ago
  Isn't that how multiplayer games work today? You receive a rendering of the server's source of truth, and send actions you'd like to perform. The current implementation just sends a lot less data to the client by sending deltas and positional data rather than full screen renderings right? It seems like that wouldn't fix any of the issues above, and might just introduce the complexity of streaming video to all the clients, and replicating world state to all the video streaming servers.
  - georgeecollins 5y ago
    Yes. Typically you are seeing an interpolations between the last agreed source of truth - 1 time interval and the last agreed source of truth. The idea is that all inputs will be received for +1 time interval before you get to the actual last agreed on source of truth (which becomes the last agreed source of truth -1 time interval ).
  - gloriousternary 5y ago
    With Stadia et al. your computer sends keystrokes and mouse movements to Google's servers, and the server returns a video feed that's then displayed on your screen. Today's multiplayer games send and receive movement information and such, but the graphics rendering, which I believe is the main issue here, is still done client-side.
    - gameswithgo 5y ago
      we try to send something as close as just keystrokes in games too, to prevent cheating among other reasons. sometimes we have to send actual state though
mimixco 5y ago
This makes sense. Forza Horizon 4 with all the money and servers Microsoft can throw at it only supports 79 live humans in the same session. It seems like a small number but with all the real-time variables from cars and liveries to backgrounds and driving behavior, it's an enormous amount of state to manage.
mekkkkkk 5y ago
What do you mean when you say "the internet is not fast enough to manage more than [100 people] with anything resembling real behavior"? Is that based on some objective metric, or just based on your experience?
aaronetz 5y ago
There's a 5,000 human player experimental event happening tomorrow [1]. As a participant in a previous invite-only event, I can assure you that it works very well. Disclaimer: I work under the same company as the developers of this tech (although I am not involved in this product). My opinions are entirely my own. [1] https://www.escapistmagazine.com/v2/scavengers-large-scale-multiplayer-event-mode-scavlab-kicks-off-this-weekend/ https://www.escapistmagazine.com/v2/scavengers-large-scale-m...
pyuser583 5y ago
Which MMO did you work at?
rasz 5y ago
Eve could go more granular than single threaded Server Blade per Solar System, go down to at lest server node per Grid, or even per 250km mini grid with BSP tree partition mechanism. They could also work on real dynamic scaling. Instead last time I looked into it they migrated Solar systems around once a day during mandatory downtime relocating the ones where huge battles were pre-planned to happen in near future to the dedicated high MHz blade nodes, and everything was single threaded. Result was two small battles in different parts of one solar system loaded the node as much as one bigger battle. Oh, and they ran a lot of Stackless Python code for that extra slowness.
iEchoic 5y ago
I remember regularly participating in real-time battles with over 200 people on Dark age of Camelot, 20 years ago. Did they do something special? Some examples: https://youtu.be/lkl_fAjmLmA?t=38 https://youtu.be/lkl_fAjmLmA?t=38 https://youtu.be/7_BNNteCtCU?t=48 https://youtu.be/7_BNNteCtCU?t=48
devit 5y ago
A contemporary broadband connection can manage way more than 100 simultaneous characters. You only need to send an updated position and direction on every frame, which can fit in 10 bytes (16-bit screen space fixed point coordinates, 3 position coordinates + 1 orientation + one 16-bit animation/action ID). With a 100mbps connection you can send 160 KB per frame at 60fps and thus support 16K characters.
- clappski 5y ago
  100mbps is much much faster than a the majority of consumer broadband connections, I don’t think that’s a particularly realistic figure. Don’t these games have to support a lowest common denominator type speed rather than a fantastic one? I’m not in game dev so not sure if there’s ways to get around things like that
  - devit 5y ago
    100mbps is at or below entry level for fiber in major cities, and outside major cities it's what Starlink claims to provide, and also in the VDSL and 4G range. A "fantastic" customer connection speed currently would be 2.5gbps or 10gbps.
    - otabdeveloper4 5y ago
      Do you know anybody who doesn't use WiFi to connect to the Internet? I don't. (But maybe I just don't know any professional gamers.) And 100mbps is not realistic for modern WiFi setups.
- fulafel 5y ago
  You can do a lot more compression and encoding. The data is very regular and you can use pretty low precisions, even lower if server tracks the accumulated error at the client. Most players aren't moving on every tick. Network tick rate is typically much slower than fps with interpolation used in presentation. Players typically move in very constrained ways (walking speed is constant). Etc.
zamalek 5y ago
> 100 simultaneous players visible to each other in a reasonable time but no more; the internet is not fast enough to manage more than that many "people" with anything resembling real behavior I participated in a tech demo test about 2 years back, with roughly 3000 players interacting in non-trivial ways (server authoritative). UDP exists. GPGPU exists (NVIDIA helped improve PhysX for this specific scenario). A massive amount of users is not impossible. This MMO does otherwise otherwise impossible.

650REDHAIR 5y ago

See star citizen for a recent example

reiichiroh 5y ago
Everything will work when /if server meshing works
reiichiroh 5y ago
Everything will work when or rather if server meshing works

rcxdude 5y ago

Exactly, on the server side player interaction is an N² problem (where N is the number of players who are currently interacting). Each of N players need updates on the (N-1) other players they can see. And without strong in-game reasons not to, players will test the limits, no matter how efficient your code (see EVE where despite them making huge improvements to their servers and code as well as already using an extremely low update rate, player battles are still basically limited by how many you can get into a system before the server topples over).

deepnotderp 5y ago
Is this really true? I would imagine you get something more like Zipf’s law in practice, N log N. Even if there were interactions between everyone, presumably you could still exploit locality, like: https://homes.cs.washington.edu/~pedrod/papers/aaai10b.pdf https://homes.cs.washington.edu/~pedrod/papers/aaai10b.pdf
- fulafel 5y ago
  In practice in other existing MMOs, you have bounded areas of the game where people are in view of each oher, with player counts designed to work within the technical constrians of the tech. Then teleport/portal around to get to other areas. But even within an area you can cut down on the n^2 in various ways, with one set of estabilished techniques cutting down server side compute and other set for network communications.

angrais 5y ago

Could you explain how world of Warcraft works then? You can certainly have more than 200 in one place at once... Are they doing some tricks to make this appear as if it's working but in reality it is not? Would love to hear your opinion.

ElectricalUnion 5y ago
> 200 in one place 200 players in one map instance. But not 200 visible player with interactions refreshing on real time. Most likely you see diminishing level of details, if anything besides complete fabrication or no data at all after a certain visibility range threshold.
- throwaway316943 5y ago
  The spiffing Brit managed to get 500 people into a single Factorio game, not all visible at the same time but all interacting with the environment. It was an interesting video, the performance was fairly good.
  - swiley 5y ago
    I saw that video, the game definitely struggled. It's certainly doable though.
  - imtringued 5y ago
    Buildings are more CPU intensive than players unless those players are all busy fighting. If they just mine or place buildings they simply don't matter.
- p1necone 5y ago
  This doesn't really matter though, after a certain point there isn't much difference to players between 50 people and 200 people within visible distance. You can just stop showing others past some cap, or throttle position updates to like 1 every 20 seconds or so and fill in the gaps with AI.

MattieTK 5y ago

This was part of the exciting suggestion of the original Stadia trailers: with the entire game and all the interactions happening in Google datacentres, you would only need to receive a rendered version, and suddenly all this would be possible. Of course none of that has happened and Google folded their games studio, which is par for them, but does go some way to reinforcing this point that there's a reason this stuff isn't done.

zolland 5y ago
Isn't that how multiplayer games work today? You receive a rendering of the server's source of truth, and send actions you'd like to perform. The current implementation just sends a lot less data to the client by sending deltas and positional data rather than full screen renderings right? It seems like that wouldn't fix any of the issues above, and might just introduce the complexity of streaming video to all the clients, and replicating world state to all the video streaming servers.
- georgeecollins 5y ago
  Yes. Typically you are seeing an interpolations between the last agreed source of truth - 1 time interval and the last agreed source of truth. The idea is that all inputs will be received for +1 time interval before you get to the actual last agreed on source of truth (which becomes the last agreed source of truth -1 time interval ).
- gloriousternary 5y ago
  With Stadia et al. your computer sends keystrokes and mouse movements to Google's servers, and the server returns a video feed that's then displayed on your screen. Today's multiplayer games send and receive movement information and such, but the graphics rendering, which I believe is the main issue here, is still done client-side.
  - gameswithgo 5y ago
    we try to send something as close as just keystrokes in games too, to prevent cheating among other reasons. sometimes we have to send actual state though

mimixco 5y ago

This makes sense. Forza Horizon 4 with all the money and servers Microsoft can throw at it only supports 79 live humans in the same session. It seems like a small number but with all the real-time variables from cars and liveries to backgrounds and driving behavior, it's an enormous amount of state to manage.

mekkkkkk 5y ago

What do you mean when you say "the internet is not fast enough to manage more than [100 people] with anything resembling real behavior"? Is that based on some objective metric, or just based on your experience?

aaronetz 5y ago

There's a 5,000 human player experimental event happening tomorrow [1]. As a participant in a previous invite-only event, I can assure you that it works very well. Disclaimer: I work under the same company as the developers of this tech (although I am not involved in this product). My opinions are entirely my own. [1] https://www.escapistmagazine.com/v2/scavengers-large-scale-multiplayer-event-mode-scavlab-kicks-off-this-weekend/ https://www.escapistmagazine.com/v2/scavengers-large-scale-m...

pyuser583 5y ago

Which MMO did you work at?

rasz 5y ago

Eve could go more granular than single threaded Server Blade per Solar System, go down to at lest server node per Grid, or even per 250km mini grid with BSP tree partition mechanism. They could also work on real dynamic scaling. Instead last time I looked into it they migrated Solar systems around once a day during mandatory downtime relocating the ones where huge battles were pre-planned to happen in near future to the dedicated high MHz blade nodes, and everything was single threaded. Result was two small battles in different parts of one solar system loaded the node as much as one bigger battle. Oh, and they ran a lot of Stackless Python code for that extra slowness.

iEchoic 5y ago

I remember regularly participating in real-time battles with over 200 people on Dark age of Camelot, 20 years ago. Did they do something special? Some examples: https://youtu.be/lkl_fAjmLmA?t=38 https://youtu.be/lkl_fAjmLmA?t=38 https://youtu.be/7_BNNteCtCU?t=48 https://youtu.be/7_BNNteCtCU?t=48

devit 5y ago

A contemporary broadband connection can manage way more than 100 simultaneous characters. You only need to send an updated position and direction on every frame, which can fit in 10 bytes (16-bit screen space fixed point coordinates, 3 position coordinates + 1 orientation + one 16-bit animation/action ID). With a 100mbps connection you can send 160 KB per frame at 60fps and thus support 16K characters.

clappski 5y ago
100mbps is much much faster than a the majority of consumer broadband connections, I don’t think that’s a particularly realistic figure. Don’t these games have to support a lowest common denominator type speed rather than a fantastic one? I’m not in game dev so not sure if there’s ways to get around things like that
- devit 5y ago
  100mbps is at or below entry level for fiber in major cities, and outside major cities it's what Starlink claims to provide, and also in the VDSL and 4G range. A "fantastic" customer connection speed currently would be 2.5gbps or 10gbps.
  - otabdeveloper4 5y ago
    Do you know anybody who doesn't use WiFi to connect to the Internet? I don't. (But maybe I just don't know any professional gamers.) And 100mbps is not realistic for modern WiFi setups.
fulafel 5y ago
You can do a lot more compression and encoding. The data is very regular and you can use pretty low precisions, even lower if server tracks the accumulated error at the client. Most players aren't moving on every tick. Network tick rate is typically much slower than fps with interpolation used in presentation. Players typically move in very constrained ways (walking speed is constant). Etc.

zamalek 5y ago

> 100 simultaneous players visible to each other in a reasonable time but no more; the internet is not fast enough to manage more than that many "people" with anything resembling real behavior I participated in a tech demo test about 2 years back, with roughly 3000 players interacting in non-trivial ways (server authoritative). UDP exists. GPGPU exists (NVIDIA helped improve PhysX for this specific scenario). A massive amount of users is not impossible. This MMO does otherwise otherwise impossible.