4 ms·

Planimeter founder here. I remember doing back of the napkin calculations on frame time and latency years ago and thinking that the fundamental constraints game

Planimeter founder here. I remember doing back of the napkin calculations on frame time and latency years ago and thinking that the fundamental constraints game developers face could not be overcome. The closest you could get would be to have edge nodes everywhere, and even then, you couldn't do it. That is to say, there is no future in this universe where real-time, predicted gameplay can currently fit within 16ms (60 FPS) let alone 6ms (144 FPS). You blow the budget immediately. Today's implementations lag several frames (overbudget). For myself and other game developers I work with, that's just unacceptable, and I don't know anyone right now who is considering software-level or gameplay-level accommodations for streaming the game over a service. Take an empty game (update nothing, draw nothing) and send black over the wire. You blow the budget. Let's not dice words: cloud gaming is the cloud industry's perpetual motion machine.

helen___keller 4y ago
> Take an empty game (update nothing, draw nothing) and send black over the wire. You blow the budget. You say that, but as a user I wouldn’t notice the difference in this game played on cloud versus locally.
- AustinDev 4y ago
  Speak for yourself. I've tried every iteration of cloud gaming and the frame lag is painfully obvious to me. Maybe it's because I've been gaming on a PC for 28 years.
  - _manifold 4y ago
    This dichotomy of player opinions seems to be present any time I see a discussion about Stadia or cloud gaming. I haven't tried it myself so I don't have a truly informed opinion, but just based on technical factors I'm extremely skeptical. However there does seem to be legitimate portion of the audience that either can't tell or simply doesn't care about frame lag/latency issues. I would guess that group is relatively small though. Also as PC and console hardware continues to improve and 4k+/144hz+ gaming becomes more commonplace, cloud gaming is just going to be left in the dust.
    - AustinDev 4y ago
      Whenever I've advised VCs on cloud gaming tech, I always point out that you can have the beefiest most expensive hardware at the edge, but you'll never be faster than a one meter HDMI/VGA/RCA cable. It's just physics. Cloud gaming will work great when we have FTL transmission of data or possibly a datacenter in every 7-11/Walmart.
  - ngokevin 4y ago
    Well, they did speak for themself.
TillE 4y ago
Absolutely. There are certainly classes of games where several frames of input lag don't matter, but look at the big AAA titles, look at the popularity of challenging action games like Elden Ring, of multiplayer shooters and MOBAs. It is, at best, a niche service.
- paulryanrogers 4y ago
  Isn't Elden Ring the kind of game where you start an animated attack then basically wait for it to finish? So the game locks in your moves? Meaning some lag may be less noticeable?
  - ranguna 4y ago
    It locks you after you start the animation, but the key here is when you start the animation. If you see someone is going to attack you, you want to dodge right away, wait for the animation to finish and attack after. With cloud gaming you see someone is going to attack you, press dodge, get attacked, you die.
    - hypertele-Xii 4y ago
      Unless you compensate. Receiveing a button press means the button was pressed <latency> ago. Rewind simulation to that moment, apply input to game state, replay to current moment, send result. Of course, single-player games don't usually do this, because it's very complicated.
im3w1l 4y ago
You are sort of comparing apples to oranges. Like consider a beefy gaming computer, but the mouse is some junk with an input latency of 10ms. Can this computer do 144fps? It most certainly can. Despite the input latency, 144 frames are rendered every second, and 144 frames are displayed on the monitor every second. Movement and animations will be butter smooth. For the player the latency is frustrating but consider the friend looking over his shoulder. He can't tell. All he knows is that there are in fact 144 frames per second. So you gotta ask yourself, well when does this make sense? Does it make sense in a twitch shooter? It doesn't. Does it make sense in a popular mmorpg that runs with a tickrate of 600ms? Yes. Does it make sense in a RTS? Probably, except at maybe the highest levels. Does it make sense in a train simulator? Yes.
donmcronald 4y ago
I think it works if you do it over a generation (of people). Never give a kid less than 50ms of latency and they won’t even realize they’re missing out. The whole industry will attack the goal from multiple angles and make it happen. I think making GPU ownership expensive to improve the value proposition of streaming services is one of the goals. On paper I think it should be a total failure. In reality, I tried the RTX 3080 tier of GeForce Now and I couldn’t notice the input latency playing Last Epoch. If triple buffering was accepted, 3 frames of latency for streaming will be too, won’t it? I hope it fails because never owning anything sucks, but I think it’ll be extremely successful.
- norwalkbear 4y ago
  Latency perception is highly game dependent. I worked on the netcode for EA games at one point in my life. The highly popular battle royal and competitive shooters will never go cloud.
- hypertele-Xii 4y ago
  Triple buffering does not inherently introduce any latency. It's a technique of using more memory to smooth out synchronization issues. Single buffering: You're drawing pixels directly on screen. Problem: Drawing isn't instant across whole screen, so human eye sees a blend of old and new image (and since screens draw top to bottom, this produces a horizontal "tear" somewhere, thus "screen tearing"). Double buffering: You're drawing pixels on a hidden screen while the previous image is displayed, then flip screens. This flip action is so fast it's practically instant, so tearing is solved. However, screens can only flip at set intervals (refresh rate). Problem: What if your drawing finishes even a little bit late, just after refresh rate sync? Now your program has to wait for the next opportunity to flip, which just makes everything hang. Triple buffering: 3 is the magic number to fix all these problems. You always have two virtual screens to flip between, and a third one you can draw to regardless of flipping.
  - cpeterso 4y ago
    But isn’t the drawn screen always two frames (32 ms) behind the game loop’s current frame?
pyinstallwoes 4y ago
Well, that may be true with having to send the entire payload over a distance of space-time. However, if AI-based rendering could sufficiently predict the next frame, then you can just simulate the next frame without lag. Given it can predict enough frames in the future in relation to compute requirements, then all renders are local-only with "online synchronization" - at least, this will probably be how we do long-distance "calling" when we are a multi-solar system species; unless we find a way to break the speed of light limit. In a way everyone you talk to really would be a simulation; and they'd "sync" up to new data every so often for alignment.
- donmcronald 4y ago
  > However, if AI-based rendering could sufficiently predict the next frame, then you can just simulate the next frame without lag. How would you predict / simulate when an opponent jumps or crouches? And how much processing power would AI rendering take? I don’t know anything about AI. I think the idea is to run the client side on a potato.
SilverBirch 4y ago
I'm not convinced that 16ms or 6ms is the right budget to talk about. Sure, if you want someone to see the picture, make a decision and reflect it back in the next picture, that would be the budget. But take a comparison to telephony, the ITU says the budget for unnoticeable latency is 150ms. So I can talk to you on the phone with 150ms latency and not even notice. Now, I think gaming is probably lower than 150 because the timing of speech is fairly slow compared to twitch shooters, but I don't think it's 6ms. You might be right it's still unachievable (and I agree, the only viable solution might be to boil the ocean by sticking data centres on the end of every street and create a new guaranteed low latency protocol that takes precedence over all other TCP traffic), but I'm not as certain.
zamalek 4y ago
Don't forget about the baseline input latency that even local games experience. The best case scenario is PC, which would add in the realm of 10-20ms of input latency to your network latency. TVs (which are the target for this stuff) have hilariously large input latency. You're teetering on the budget before you even hit the network.
deleted 4y ago
[deleted]

helen___keller 4y ago

> Take an empty game (update nothing, draw nothing) and send black over the wire. You blow the budget. You say that, but as a user I wouldn’t notice the difference in this game played on cloud versus locally.

AustinDev 4y ago
Speak for yourself. I've tried every iteration of cloud gaming and the frame lag is painfully obvious to me. Maybe it's because I've been gaming on a PC for 28 years.
- _manifold 4y ago
  This dichotomy of player opinions seems to be present any time I see a discussion about Stadia or cloud gaming. I haven't tried it myself so I don't have a truly informed opinion, but just based on technical factors I'm extremely skeptical. However there does seem to be legitimate portion of the audience that either can't tell or simply doesn't care about frame lag/latency issues. I would guess that group is relatively small though. Also as PC and console hardware continues to improve and 4k+/144hz+ gaming becomes more commonplace, cloud gaming is just going to be left in the dust.
  - AustinDev 4y ago
    Whenever I've advised VCs on cloud gaming tech, I always point out that you can have the beefiest most expensive hardware at the edge, but you'll never be faster than a one meter HDMI/VGA/RCA cable. It's just physics. Cloud gaming will work great when we have FTL transmission of data or possibly a datacenter in every 7-11/Walmart.
- ngokevin 4y ago
  Well, they did speak for themself.

TillE 4y ago

Absolutely. There are certainly classes of games where several frames of input lag don't matter, but look at the big AAA titles, look at the popularity of challenging action games like Elden Ring, of multiplayer shooters and MOBAs. It is, at best, a niche service.

paulryanrogers 4y ago
Isn't Elden Ring the kind of game where you start an animated attack then basically wait for it to finish? So the game locks in your moves? Meaning some lag may be less noticeable?
- ranguna 4y ago
  It locks you after you start the animation, but the key here is when you start the animation. If you see someone is going to attack you, you want to dodge right away, wait for the animation to finish and attack after. With cloud gaming you see someone is going to attack you, press dodge, get attacked, you die.
  - hypertele-Xii 4y ago
    Unless you compensate. Receiveing a button press means the button was pressed <latency> ago. Rewind simulation to that moment, apply input to game state, replay to current moment, send result. Of course, single-player games don't usually do this, because it's very complicated.

im3w1l 4y ago

You are sort of comparing apples to oranges. Like consider a beefy gaming computer, but the mouse is some junk with an input latency of 10ms. Can this computer do 144fps? It most certainly can. Despite the input latency, 144 frames are rendered every second, and 144 frames are displayed on the monitor every second. Movement and animations will be butter smooth. For the player the latency is frustrating but consider the friend looking over his shoulder. He can't tell. All he knows is that there are in fact 144 frames per second. So you gotta ask yourself, well when does this make sense? Does it make sense in a twitch shooter? It doesn't. Does it make sense in a popular mmorpg that runs with a tickrate of 600ms? Yes. Does it make sense in a RTS? Probably, except at maybe the highest levels. Does it make sense in a train simulator? Yes.

donmcronald 4y ago

I think it works if you do it over a generation (of people). Never give a kid less than 50ms of latency and they won’t even realize they’re missing out. The whole industry will attack the goal from multiple angles and make it happen. I think making GPU ownership expensive to improve the value proposition of streaming services is one of the goals. On paper I think it should be a total failure. In reality, I tried the RTX 3080 tier of GeForce Now and I couldn’t notice the input latency playing Last Epoch. If triple buffering was accepted, 3 frames of latency for streaming will be too, won’t it? I hope it fails because never owning anything sucks, but I think it’ll be extremely successful.

norwalkbear 4y ago
Latency perception is highly game dependent. I worked on the netcode for EA games at one point in my life. The highly popular battle royal and competitive shooters will never go cloud.
hypertele-Xii 4y ago
Triple buffering does not inherently introduce any latency. It's a technique of using more memory to smooth out synchronization issues. Single buffering: You're drawing pixels directly on screen. Problem: Drawing isn't instant across whole screen, so human eye sees a blend of old and new image (and since screens draw top to bottom, this produces a horizontal "tear" somewhere, thus "screen tearing"). Double buffering: You're drawing pixels on a hidden screen while the previous image is displayed, then flip screens. This flip action is so fast it's practically instant, so tearing is solved. However, screens can only flip at set intervals (refresh rate). Problem: What if your drawing finishes even a little bit late, just after refresh rate sync? Now your program has to wait for the next opportunity to flip, which just makes everything hang. Triple buffering: 3 is the magic number to fix all these problems. You always have two virtual screens to flip between, and a third one you can draw to regardless of flipping.
- cpeterso 4y ago
  But isn’t the drawn screen always two frames (32 ms) behind the game loop’s current frame?

pyinstallwoes 4y ago

Well, that may be true with having to send the entire payload over a distance of space-time. However, if AI-based rendering could sufficiently predict the next frame, then you can just simulate the next frame without lag. Given it can predict enough frames in the future in relation to compute requirements, then all renders are local-only with "online synchronization" - at least, this will probably be how we do long-distance "calling" when we are a multi-solar system species; unless we find a way to break the speed of light limit. In a way everyone you talk to really would be a simulation; and they'd "sync" up to new data every so often for alignment.

donmcronald 4y ago
> However, if AI-based rendering could sufficiently predict the next frame, then you can just simulate the next frame without lag. How would you predict / simulate when an opponent jumps or crouches? And how much processing power would AI rendering take? I don’t know anything about AI. I think the idea is to run the client side on a potato.

SilverBirch 4y ago

I'm not convinced that 16ms or 6ms is the right budget to talk about. Sure, if you want someone to see the picture, make a decision and reflect it back in the next picture, that would be the budget. But take a comparison to telephony, the ITU says the budget for unnoticeable latency is 150ms. So I can talk to you on the phone with 150ms latency and not even notice. Now, I think gaming is probably lower than 150 because the timing of speech is fairly slow compared to twitch shooters, but I don't think it's 6ms. You might be right it's still unachievable (and I agree, the only viable solution might be to boil the ocean by sticking data centres on the end of every street and create a new guaranteed low latency protocol that takes precedence over all other TCP traffic), but I'm not as certain.

zamalek 4y ago

Don't forget about the baseline input latency that even local games experience. The best case scenario is PC, which would add in the realm of 10-20ms of input latency to your network latency. TVs (which are the target for this stuff) have hilariously large input latency. You're teetering on the budget before you even hit the network.

deleted 4y ago

[deleted]