4 ms·
Since I'm writing a new client, in Rust, for Second Life/Open Simulator, I'm very aware of these issues. A metaverse client for a high-detail virtual world has
by Animats 5y ago
Since I'm writing a new client, in Rust, for Second Life/Open Simulator, I'm very aware of these issues.
A metaverse client for a high-detail virtual world has most of the problems of an MMO client plus many of the problems of a web browser. First, much of what you're doing is time-sensitive. You have a stream of high-priority events in each direction that have to be dealt with quickly but don't have a high data volume. Then you have a lot of stuff that's less time critical.
The event stream is usually over UDP in the game world. Since you might lose a packet, that's a problem.
Most games have "unreliable" packets, which, if lost, are superseded by later packets. ("Where is avatar now" is a typical use.) You'd like to have that stream on a higher quality of service than the others, if only ISPs and routers actually paid attention to that.
Then you have the less-critical stuff, which needs reliability. ("Object X enters world" is a typical use.)
I'd use TCP for that, but Second Life has its own not very good UDP-based protocol, with a fixed retransmit timer. Reliable delivery, in-order delivery, no head of line blocking - pick two. TCP chooses the first two, SL's protocol chooses the first and third ones. Out of order delivery after a retransmit can cause avatars to lose clothing items, because the child item arrived before the parent item.
Then you have asset fetching. In Second Life/Open Simulator this is straight HTTP/1. But there are some unusual tricks. Textures are stored in progressive JPEG 2000. It's possible to open a connection and just read a few hundred bytes to get a low-rez version. Then, the client can stop reading for a while, put the low-rez version on screen, and wait to see if there's a need to keep reading, or just close the connection because a higher-rez version is not needed. The poor server has to tolerate a large number of stalled connections. Worse, the actual asset servers on AWS are front-ended by Akamai, which is optimized for browser-type behavior. Requesting an asset from an Akamai cache results in fetching the entire asset from AWS, even if only part of it is needed. There's a suspicion that large numbers of partial reads and stalled reads from clients sometimes causes Akamai's anti-DDOS detection to trip and throttle the data flow.
So those are just some of the issues "the HTTP of VR" must handle. Most are known to MMO designers. The big difference in virtual worlds is there's far more dynamic asset loading. How well that's managed has a strong influence on how consistent the world looks. It has to be constantly re-prioritized as the viewpoint moves.
(Demo, from my own work: https://vimeo.com/user28693218 https://vimeo.com/user28693218 This shows the client frantically trying to load the textures from the network before the camera gets close. Not all the tricks to make that look good are in this demo.)
It's not an overwhelmingly hard problem, but botch it and you will be laughed off Steam.
- moron4hire 5y agoThis is why I think it's a joke to be building metaverse apps in Unity. Unity and dynamic asset loading are not happy bed fellows. There's not a lot I liked about Unity when I was working with it full-time a few years ago. But the one thing I could acknowledge that it has that was generally missing from open source web development was the asset pipeline. But dynamic, user-uploaded assets won't be able to use the asset pipeline. So one of the biggest drivers for using Unity goes right out the window.
- Animats 5y agoUnity and dynamic asset loading are not happy bed fellows. Not Unreal Engine 4, either. UE5 has "asset streaming" and "open worlds", but mostly static and loaded from a local SSD on a Playstation 5. That's working nicely. Asset management from the network is the real difference with seamless, modifiable virtual world systems. Otherwise, it's a minute of "...LOADING..." when you move to the next area. You need clients, servers, file formats, and protocols designed for it. It's a moderately hard engineering problem, and, as yet, there are no good off the shelf solutions. There's a "check out, check in" approach. Decentraland uses that. You check out your parcel into a local Unity environment, edit, and check in the whole parcel to make it visible to others. The Spatial OS people, Improbable, did some of this, but their solution cost so much to operate server side that all four of the games that used it went broke. So Improbable is trying to pivot to military simulation. Probably by UE6 this will all be standard. It's one of those things that has to be done to move the metaverse from hype to usefulness.
- bullen 5y agoBut why does it have to load dynamically, when you can just package releases instead?
- sbierwagen 5y agoFor Second Life at least, it's hard to package stuff. World geometry, player models, lighting and props can all be edited in realtime. You can't use most of the tricks that video game devs are used to: no simple way to prebake lighting or reflections, no occlusion culling or binary space partition. The asset base is enormous: hundreds or thousands of terabytes. You can bet that every player character will have a completely unique set of textures and models, very little is shared between characters. This is why long time SL users tend to stick to one location, because actually exploring is painful-- move 100 meters, then stand still for a few minutes as your framerate plummets and everything slowly loads in. Many clubs or hangout spaces are on islands or boxes in the sky high enough that nothing else loads in. This reduces the problem just to rendering avatars, which is hard enough by itself. It's also very hard to depreciate old systems. The game is 18 years old, but there were real money transactions from day one. If a user bought something ten years ago, they expect to still be able to use it! You can imagine a clean-sheet design that does away with all this. Make the world static, player avatars pre-baked. Most SL competitors do this. (Facebook Spaces, Playstation Home etc) This gives you better FPS and a much more consistent aesthetic, since all the assets are made in-house. But now what? The classic Metaverse problem, there's just not that much to do. Cutting features from SL makes this worse, not better.