9 ms·
The Technology Behind A Low Latency Cloud Gaming Service
- chrisd1100 10y agoHey guys, I'm the author of the post. Let me know if you have any questions!
- lynxaegon 10y agoGood job! I see that your first picture is from Rocket League, i ussualy play rocket league using the steam controllers :) It would be really nice if the steam controllers could work with it.
- chrisd1100 10y agoYeah, I hear you, we're getting there. Our next major feature release will be Raspberry Pi support with controller support.
- justinsaccount 10y agoDoes that also imply general linux support, or would you only have a closed source arm binary?
- chrisd1100 10y agoLuckily the video processing libraries provided by Intel/NVIDIA/AMD are cross platform (Win32/Linux). So the arm specific video processing code for the RPI is a different can of worms, the rest of the client (window creation, polling for input, etc) should be able to support Linux and X11 generally.
- justinsaccount 10y agoCool. How 'gaming' specific is the platform itself? It seems very similar to an optimized version of PCoIP.
- chrisd1100 10y agoActually not gaming specific at all. You can think of it like a high performance remote desktop app that happens to work really well for games.
- erichocean 10y agoI'm very interested is streaming 3D content creation UIs. A major issue with virtual filmmaking is the datasets are huge, which makes it very, very difficult to work with a remote team, because you can't effectively get the dataset to people's machines even just to work on. What I'd like to do is host our dataset AND the 3D content creation apps on our own hardware at Switch (in Las Vegas), and then have our employees and contractors throughout California access 3D content creation apps remotely, via something like your streaming approach. Thoughts?
- chrisd1100 10y agoWe've heard of something like this before, and while we're very much focusing on gaming right now, our goal is to make Parsec general purpose enough to fit into a lot of different use-cases. I see no reason why it couldn't be used for what you're describing.
- erichocean 10y agoObviously, remote desktop "works" (as in: functions) today, we just want something with better performance, specifically, lower latency. :)
- kayoone 10y agoI just tested this service with a full blown 3D Development Environment like Unity3d using an AWS Instance and it works amazingly well and is very responsive. Through Amazon Gigabit connection everything installs in seconds as well. I had a fairly low resolution (800p) set, but i can totally see this being viable for CAD/3D Development etc.
- erichocean 10y agoThanks for the feedback!
- pshc 10y agoHey, nice post! Quite a detailed write-up and an impressively optimized video pipeline. I'm curious how much end-to-end latency you experience on average?
- chrisd1100 10y agoOn newer hardware (Geforce GTX 900+ series on the server side, recent Intel HD Graphics on the client) our system only adds 8-10 ms. On older rigs it might be somewhere around 20. Of course the network latency will go on top of that.
- ddalex 10y agoOn the technology - can this be used to stream the games from the beefy machine I have in the basement? On the business side of view, how do you plan to make moniez?
- c0g 10y agoSteam in home streaming works very well for local streaming. You can add non-steam games to the library.
- brazzledazzle 10y agoI wonder how well it would work with this setup if you were connected to it over a VPN. Presumably (despite the latency) it would consider it the same network.
- chrisd1100 10y agoYeah, it can be used for in-home streaming or over the WAN. Making money, we plan to wrap up the cloud experience for you nicely and take a margin on it. But for personal use it will remain free.
- ohstopitu 10y agoGiven a nice price-point (~ the cost of Netflix, which I believe varies from region to region), I'll be a very happy customer. I have a beefy home PC that is showing it's age and I'd prefer to have a SaaS solution that I can access everywhere rather than role on my own! So if you guys do release in the next 4 months or so...consider me a customer.
- ddalex 10y agoWhat would it take to port this tech to Linux, on both sides (server/client)?
- chrisd1100 10y agoClient side, we've already done it (we have a Raspberry Pi client that actually works well, but still needs a bit of polish.). Server side I'm not sure. I know the Windows API for grabbing frames is really good, and I assume there's a similar API for Linux. Shouldn't be too bad. But you will likely see a macOS server first...
- coldcode 10y agoWould you not gain something by using Metal instead of OpenGL on Macs?
- chrisd1100 10y agoWe actually tinkered a bunch with Metal, it will likely one day be the default, but we went with the broader support of OpenGL for now.
- Matthias247 10y agoYou wrote about H.264 compression, but not on how you transport the compressed stream over the network. Can you elaborate a little bit about that? E.g. do you multiplex into a mpeg transport stream? RTP? UDP?
- kayoone 10y agoReally cool stuff, i have just read the article that inspired you to do this today because i wanted to actually test it. If i understand correctly you also have a Streaming Server to install in the local network, so it's basically doing what Steam in Home Streaming does ? How does it compare to that in terms of performance ?
- jstanley 10y agoThis is really interesting, I did not know it was possible yet. A future advancement that could be made possible by this technology is the complete elimination of "laggy hitbox" type effects: if the server that handles physics is also handling rendering, it can make sure to show a consistent picture of the environment to every player.
- neckro23 10y agoThat would be equivalent to disabling lag compensation, which isn't possible anyways if you're just streaming video. Some older FPS games (Quake 1 original netcode, Halo) don't do lag compensation, and it's not pretty. The "laggy hitbox" effect only happens because the client predicts non-player actions before the action data actually arrives. The alternative is to have actions "officially" occur when the packet hits the server instead of when the player actually acts, which means that players with high latency will be at a disadvantage. With streaming video it seems this problem is unavoidable.
- jstanley 10y agoAll of the rendering would be server-side, so you won't even see the muzzle flash until the server is aware of it. With client-side rendering, the client draws a muzzle flash as soon as you fire, and there could still be an enemy in the line of fire at the time the muzzle flash goes off, and the shot still not hit if the enemy has already jumped out of the way.
- Paul_S 10y agoPeople laughed when onlive said they could do it. And then onlive went bankrupt. As you admit yourself you can't do anything about network lag and reliability. Even if you get down to 0ms video enc-dec this will be not viable for any games where the camera moves.
- chrisd1100 10y agoI've played CS:GO from NY to Virginia with Parsec (to an AWS machine) and topped the scoreboard consistently (in pugs). Also did a 3 hour raid night on the new Legion content with it on WoW, lot of dynamic camera movement there. Cleared Nythendra on Heroic as the 4th DPS. May not be to everyone's liking, but worked for me.
- ttul 10y agoHmmm. I have 10Gbps fiber in my office that is 3ms away from us-west-2...
- benologist 10y agoYou're right, networking will and can never, ever get better.
- reitzensteinm 10y agoAt the end of the day it's just about how local you can make the data centers. If you have a few ms of ping, great bandwidth and hardware to encode the video, it's absolutely feasible. While the speed of light isn't getting any faster, bandwidth and hardware encoding both are (relative to the speed at which we're increasing resolution and image quality). And the really nice thing about it is that you can afford to spend an order of magnitude on the hardware if it's shared around and played casually than if it just sits in your closet most of the time. And you can game on laptops or even mobile devices easily. The experience could actually be fantastic, and hugely social; imagine reading a review with screenshots from the game, and clicking on them allows you to start to playing in under a second from that exact moment, just like you can send a YouTube link mid video. The trouble is, there are a hell of a lot of cities in the world. In Chicago, everyone could be gaming like this in ten years; but in rural New Zealand?
- vvanders 10y agoThis stuff always seems to be just around the corner, however there's some fundamentals that are always going to keep it from working. 1. Some games just aren't built for any latency. I'm talking fighting games(1 frame windows), majority of FPS' and anything that has a reaction instead of a prediction game model. These can be made network compatible with a time-rewind gamestate resolution(which many good ones do), however they aren't drop-in. 2. The market that's interested in this tends to be much more demanding than your traditional consumer. Drop ~150ms of packets at a critical time? You've got a really frustrated customer where a game designed for network degradation will handle this gracefully. Kudos to them for trying but you're not dealing with just technical issues, there's fundamentals of design that need to be adjusted for that can't be done without direct developer involvement.
- EGreg 10y ago1) You can simply have vector clocks, and resolve differences like anything else. Most of the time the game's prediction would be correct. Sometimes there would be a small adjustment. 2) Yes, anything real-time is like this.
- felipesoc 10y agoPrediction requires full game emulation on client, defeating the purpose of streaming.
- EGreg 10y agoIt doesn't defeat the purpose of streaming. You don't have to emulate everything in the game, just everything in your vicinity that can possibly affect what you see and hear.
- stupidcar 10y agoThat is how client-server games already work. The server has the full world state, and each client has a subset necessary to render the world for that player. Ideally the absolute minimum subset, so as to reduce the potential for cheating (wallhacks, etc.). The purpose of streaming is to remove the need for the simulation and rendering at all. The client is a dumb terminal that just renders frames and records input. The moment you start making the client smart again — making it aware of a subset of the world state, making it do its own rendering based on the input — you've just reinvented current client-server gaming.
- wmf 10y agoDon't NVIDIA, AMD, and Steam already provide this?
- atemerev 10y agoTried the AWS configuration, things went great (from Geneva to eu-east, symmetrical 1Gbit at home). Indeed, major source of frustration were videocodecs, not networking.
- ddavidn 10y agoInteresting. I'll have to try it and see how it compares to my current gaming AMI where I just use Steam or Remotr to stream things like Dirt and ARK.
- isaaaaah 10y agoI do not support the idea one bit. Sure, great stuff an all, if you do not care having everything behind a bottleneck, in the hands of other people. I do.
- goda90 10y agoI've been wanting to do a project to let a group of friends play multiplayer in a certain turned based 4X game by streaming from a remote server like this. The latency wouldn't need to be super low since it's turned based, so a browser based streaming client could work. The game has a hotseat multiplayer mode, so one person could take a turn, and then the server could go into a suspended mode while it waits for the next person to log in. The idea is to save the time of having to load up the game to take one turn, and having to be at a computer that has it installed already.
- witty_username 10y agoRDP/VNC should work for this.
- ksec 10y agoUm.... could PlayStation 4 implement something like this and have the game played via local network?
- justinsaccount 10y agoYou mean PlayStation TV?
- chadnickbok 10y agoThey already do! Playstation bought Gaikai and turned it into Playstation Now. They also use the technology for PS4 Remote Play. Its awesome and you should check it out :D
- landongn 10y agoPS4 has been doing this for years through PlayStation Now and Gaikai tech. Remote play works phenomenally well, as does PS Now. You should go pick up a trial of the PC app (psnow.com) and try it out for yourself.
- seanalltogether 10y agohttps://remoteplay.dl.playstation.net/remoteplay/lang/en/index.html https://remoteplay.dl.playstation.net/remoteplay/lang/en/ind...
- falcolas 10y agoAs someone who does not live near any major datacenters, this kind of service just doesn't work for me. At all. The absolute lowest ping I can reliably maintain from my house to a major datacenter are those in CA, and it's around 60ms. So, I'm effectively attempting to play a game at 15fps. Workable for some 4x games and a few single player action games, but nowhere near pleasant. Since my real world pings to game servers are much closer to 100/150ms (10fps) with the occasional burst of packet loss (especially in the evenings), cloud gaming is unrealistic (PS Now basically says "No, you can't use this service"). I'm getting 60ms averages from work (which is on fibre) to AWS-East (where parsec hosts their website from), so there's no way this would ever be viable for me. Seems like it would have limited appeal to non-casual gamers, if I'm honest. You have to near a major datacenter with reliable broadband, and you could never play competitively in a broad set of latency-sensitive games. On the other hand, I could definitely see the appeal for those who want to play the latest Peggle or Parking Dash games.
- deleted 10y ago[deleted]
- jerf 10y agoYou can't convert latency into FPS like that. You can have seconds of latency but stream at 60 fps, or 5fps streams with very low latency. The two numbers are independent.
- kayoone 10y agoThe US is kind of at a disadvantage for this because distances are so huge, but in smaller european countries it's not uncommon to get <20ms pings to gameservers around the country if you have a decent broadband connection.
- PureSin 10y agoI saw the post on using AWS to play games too. Used it to play the division for a while. Glad to see a service being built for this. Signed up and will be trying it out this weekend.
- pankajkumar229 10y agoWe built the exact same technology at Agawi(http://arstechnica.com/gaming/2012/09/report-cable-companies-planning-to-stream-games-to-subscribers/ http://arstechnica.com/gaming/2012/09/report-cable-companies...). The only difference was that we did not have the parker API. We worked very closely with Microsoft and NVidia to make it work back then with full headless Windows GPU servers. H264 encoding both in GPU and CPU. We could have reduced latency by distributing the servers but we did not get to the stage of distributing GPU cloud back then. But the business never took off. I was not on the business side so I cannot tell exactly why. Probably latency but there are 3D strategy type games that you could stream. If you need, I can ask the business head of our team and he can elaborate.
- deleted 10y ago[deleted]
- kabdib 10y agoOne hard nut: the cost of provisioning datacenters with enough GPU capacity to meet the demand curve of a "hot" title, and still make sense in terms of inevitable idle time and depreciation. Either you have insufficient capacity to meet day 1 load when everyone piles onto a hot new title, or you over-provision, meet that demand and have much of that hardware doing nothing during doldrum seasons (or when a title bombs). Probably you need to figure out how to make GPU capacity useful when it's not rendering games, and sell that as a service as well (GPU-based machine learning?). It doesn't help that OSes have been prickly about letting processes share GPU resources; I imagine there are a fair number of thorny security problems, even with GPU MMUs. This doesn't seem like something a cloud gaming start-up can really tackle; lots of capitalization, with lots of competition from entrenched providers, and no really compelling reason to put games in the cloud to begin with. The bigger (console) players probably realized that having consumers buy their own compute is not only cheaper and more resilient depreciation-wise, but also causes a nice platform lock-in effect once the customers have purchased a few titles.
- kukx 10y agoTo sceptics: John Carmack believes that cloud gaming can be successful. Here's his view on this (2013): https://www.youtube.com/watch?v=Uooh0Y9fC_M&t=26m24s https://www.youtube.com/watch?v=Uooh0Y9fC_M&t=26m24s
- AstralStorm 10y agoELI5 version: - use a video encoder variant with low latency and CPU/GPU usage - keep video output in video memory as much as possible - no solution to the problem of getting low latency high bandwidth internet to the customers