5 ms·

Building a Cross-Browser, Cross-Platform, Real-Time Game Streaming Protocol

120 points by andrewmd5 8y ago

Malp 8y ago
To anybody who's interested in these blog posts, I highly recommend checking out the Rainway Github: https://github.com/RainwayApp https://github.com/RainwayApp
monetus 8y ago
Thanks for making these posts, open sourcing spitfire, and supporting multiple browsers. This is really interesting. I am very eager for browsers to improve their real-time audio/video potential.
kupiakos 8y ago
Ah darn, on first glance, it looks like the server is Windows-only.
- jeremycarter 8y ago
  Not for long
- benbristow 8y ago
  Makes sense seeing as Windows is the primary PC gaming platform
  - dvlsg 8y ago
    Even for servers? I would've guessed most game servers ran on Linux. Seems like the licensing costs for windows servers would be fairly restrictive for that.
    - bestest 8y ago
      A "server" in this particular case is a machine capable of running most video games (and Steam, GOG, any other game client), ergo a Windows machine.
fulafel 8y ago
What do they mean with WebSockets hole punch on port 443 and Let's Encrypt? The first thing that comes to mind is running a TLS web/WS server on the endpoints, but I didn't think you can do TCP+TLS servers with WebRTC. Or can you? A new open source server side WebRTC implementation would be very welcome, though this does seem to use some of the old C++ libraries too. I wonder how much exposure of C++ to hostile bits they managed to reduce?
- cjbprime 8y ago
  That entire section reads like nonsense to me (and I understand NAT hole punching), seems like some kind of confusion. Why do you need special TLS certs for WebSockets, but not WebRTC? WebRTC is end-to-end encrypted like TLS, using a Diffie-Hellman exchange so that the signaling server doesn't see private keys.
  - vijaybritto 8y ago
    I think they verify the clients using letsencrypt and avoid using a turn server everytime to establish a connection. Not sure. It's quite confusing
  - andrewmd5 8y ago
    We cover this issue in a separate blog here https://blog.rainway.io/encryption-for-all-3383217e4194 https://blog.rainway.io/encryption-for-all-3383217e4194 To summarize, you are correct WebRTC is end-to-end encrypted via DTLS and WebRTC handles setting all of this up. WebSockets, however, are not encrypted by default, and when attempting to connect to a WebSocket server from a secure origin, a valid TLS certificate is required. Because each user has their WebSocket server running from their Rainway instance, we need to create unique and valid certificates for each user to avoid having a shared private key. This is how we avoid needing a TURN server and maintain low latency. I apologize that it wasn't clear in the blog I was describing how we handle WebRTC failing.
    - vijaybritto 8y ago
      So the TLS certificate for websocket users are stored in the server? And its generated by letsencrypt?
k__ 8y ago
I don't understand why they don't use WebRTC clients on the server too. Wouldn't this get rid of the NAT problem all together? I even read that people had scaling problems with P2P WebRTC. (I know that WebRTC is intended to be used as P2P technology, but noone stops you from treating the server as a client)
- andrewmd5 8y ago
  The author here, I apologize for any confusion from my writing. We do use WebRTC in a server-like fashion. The browser attempts to establish a P2P connection with the host computer, and the peer negotiation is similar to if you wanted Chrome to talk to Firefox for a video call. The network of either end could both cause connectivity to fail which is when we fall back to direct WebSockets rather than a costly TURN server.
  - k__ 8y ago
    Do you have any numbers on how much % of the users would need TURN?
    - cjbprime 8y ago
      I believe Google has previously published numbers (from Hangouts data) of around 10% worldwide.
mahgnous 8y ago
The technology is cool, but this is going to signal the end of GPU power in the home and monetizing everything. I really enjoy building gaming machines.
fabricexpert 8y ago
Why is game streaming suddenly becoming popular? I remember being blown away by onlive back in 2011: https://en.wikipedia.org/wiki/OnLive https://en.wikipedia.org/wiki/OnLive https://www.youtube.com/watch?v=jG6Ahg_d_BU https://www.youtube.com/watch?v=jG6Ahg_d_BU but they couldn't make it work commercially and shut down in 2015. Bandwidth hasn't really gone up, so what will be different this time round? Edit: they actually laid off all employees in 2012, then relaunched and still failed in 2015
- zamadatix 8y ago
  My first guess is 2012 lacked easily accessible hardware accelerated h.264 encoding on the host and the cheaper clients lacked easily accessible hardware accelerated h.264 decoding. Now you can find hardware accelerated decode in your $20 IoT device. Also I don't think your statement that bandwidth hasn't really gone up is true. https://www.statista.com/statistics/616210/average-internet-connection-speed-in-the-us/ https://www.statista.com/statistics/616210/average-internet-...
  - WorldMaker 8y ago
    Piggybacking off that, OnLive never seemed to know if they were a hardware company, a software company, or a cloud company, and depending on which exec was talking and the day of the week seemed to have a different focus each press release. Today you have major players that already cover all three bases (Microsoft and Sony) interested and playing in the space. You also have more opportunity to see minor players enter the space with less of need to be hardware companies (because more users already have capable hardware devices), or cloud companies (because more commodity streaming cloud options are available than ever before on AWS/Azure/etc), or even software companies as more platforms including the web platform have increasingly more high level APIs for a lot of the building blocks of game streaming.
- ArtWomb 8y ago
  Maybe pricing arbitrage? Every gamer I know has like 20-30 titles in their queue they want to try out. To purchase all of them only for a few moments play would be prohibitive. Pay per use makes more sense if it falls below current gpu cloud cost (~$1/hr). What's amazing is that if you can deliver video games at 4K 60fps. You can deliver all but the most intensive any application this way.
deleted 8y ago
[deleted]
wpdev_63 8y ago
How does this service compare to parsec[0]? It looks good but parsec just came out with in browser streaming for its services and runs on top of aws. [0]:parsecgaming.com
- andrewmd5 8y ago
  I will avoid getting into how our streaming performs vs. theirs because we have a lot of resources on why our tech is so fast[0] That being said, as far as I know, Parsec's new browser-based client suffers from all the issues I mentioned in this blog and at the end of the day, our user experiences are completely different. Rainway being self-hostable you can put it on everything from Azure[1] to your home computer. We have a very unique "Games First"[3] approach and view ourselves more as a gaming service than a piece of game streaming software. As we continue to fledge out our voice, I think you'll be pleased with the results. [0]:https://blog.rainway.io/here-whats-new-in-rainway-0-5-0-3a66714a8b46 https://blog.rainway.io/here-whats-new-in-rainway-0-5-0-3a66... [1]:https://www.youtube.com/watch?v=OlMq-3hmm5Y&list=PLaVZpLEYEQNz34jHAiPcHTZ8ANr7Vkn3T https://www.youtube.com/watch?v=OlMq-3hmm5Y&list=PLaVZpLEYEQ... [2]:https://blog.rainway.io/our-core-mission-games-first-78671e496201 https://blog.rainway.io/our-core-mission-games-first-78671e4...
ganaz 8y ago
so awesome

120 points by andrewmd5 8y ago

Malp 8y ago

To anybody who's interested in these blog posts, I highly recommend checking out the Rainway Github: https://github.com/RainwayApp https://github.com/RainwayApp

monetus 8y ago

Thanks for making these posts, open sourcing spitfire, and supporting multiple browsers. This is really interesting. I am very eager for browsers to improve their real-time audio/video potential.

kupiakos 8y ago

Ah darn, on first glance, it looks like the server is Windows-only.

jeremycarter 8y ago
Not for long
benbristow 8y ago
Makes sense seeing as Windows is the primary PC gaming platform
- dvlsg 8y ago
  Even for servers? I would've guessed most game servers ran on Linux. Seems like the licensing costs for windows servers would be fairly restrictive for that.
  - bestest 8y ago
    A "server" in this particular case is a machine capable of running most video games (and Steam, GOG, any other game client), ergo a Windows machine.

fulafel 8y ago

What do they mean with WebSockets hole punch on port 443 and Let's Encrypt? The first thing that comes to mind is running a TLS web/WS server on the endpoints, but I didn't think you can do TCP+TLS servers with WebRTC. Or can you? A new open source server side WebRTC implementation would be very welcome, though this does seem to use some of the old C++ libraries too. I wonder how much exposure of C++ to hostile bits they managed to reduce?

cjbprime 8y ago
That entire section reads like nonsense to me (and I understand NAT hole punching), seems like some kind of confusion. Why do you need special TLS certs for WebSockets, but not WebRTC? WebRTC is end-to-end encrypted like TLS, using a Diffie-Hellman exchange so that the signaling server doesn't see private keys.
- vijaybritto 8y ago
  I think they verify the clients using letsencrypt and avoid using a turn server everytime to establish a connection. Not sure. It's quite confusing
- andrewmd5 8y ago
  We cover this issue in a separate blog here https://blog.rainway.io/encryption-for-all-3383217e4194 https://blog.rainway.io/encryption-for-all-3383217e4194 To summarize, you are correct WebRTC is end-to-end encrypted via DTLS and WebRTC handles setting all of this up. WebSockets, however, are not encrypted by default, and when attempting to connect to a WebSocket server from a secure origin, a valid TLS certificate is required. Because each user has their WebSocket server running from their Rainway instance, we need to create unique and valid certificates for each user to avoid having a shared private key. This is how we avoid needing a TURN server and maintain low latency. I apologize that it wasn't clear in the blog I was describing how we handle WebRTC failing.
  - vijaybritto 8y ago
    So the TLS certificate for websocket users are stored in the server? And its generated by letsencrypt?

k__ 8y ago

I don't understand why they don't use WebRTC clients on the server too. Wouldn't this get rid of the NAT problem all together? I even read that people had scaling problems with P2P WebRTC. (I know that WebRTC is intended to be used as P2P technology, but noone stops you from treating the server as a client)

andrewmd5 8y ago
The author here, I apologize for any confusion from my writing. We do use WebRTC in a server-like fashion. The browser attempts to establish a P2P connection with the host computer, and the peer negotiation is similar to if you wanted Chrome to talk to Firefox for a video call. The network of either end could both cause connectivity to fail which is when we fall back to direct WebSockets rather than a costly TURN server.
- k__ 8y ago
  Do you have any numbers on how much % of the users would need TURN?
  - cjbprime 8y ago
    I believe Google has previously published numbers (from Hangouts data) of around 10% worldwide.

mahgnous 8y ago

The technology is cool, but this is going to signal the end of GPU power in the home and monetizing everything. I really enjoy building gaming machines.

fabricexpert 8y ago

Why is game streaming suddenly becoming popular? I remember being blown away by onlive back in 2011: https://en.wikipedia.org/wiki/OnLive https://en.wikipedia.org/wiki/OnLive https://www.youtube.com/watch?v=jG6Ahg_d_BU https://www.youtube.com/watch?v=jG6Ahg_d_BU but they couldn't make it work commercially and shut down in 2015. Bandwidth hasn't really gone up, so what will be different this time round? Edit: they actually laid off all employees in 2012, then relaunched and still failed in 2015

zamadatix 8y ago
My first guess is 2012 lacked easily accessible hardware accelerated h.264 encoding on the host and the cheaper clients lacked easily accessible hardware accelerated h.264 decoding. Now you can find hardware accelerated decode in your $20 IoT device. Also I don't think your statement that bandwidth hasn't really gone up is true. https://www.statista.com/statistics/616210/average-internet-connection-speed-in-the-us/ https://www.statista.com/statistics/616210/average-internet-...
- WorldMaker 8y ago
  Piggybacking off that, OnLive never seemed to know if they were a hardware company, a software company, or a cloud company, and depending on which exec was talking and the day of the week seemed to have a different focus each press release. Today you have major players that already cover all three bases (Microsoft and Sony) interested and playing in the space. You also have more opportunity to see minor players enter the space with less of need to be hardware companies (because more users already have capable hardware devices), or cloud companies (because more commodity streaming cloud options are available than ever before on AWS/Azure/etc), or even software companies as more platforms including the web platform have increasingly more high level APIs for a lot of the building blocks of game streaming.
ArtWomb 8y ago
Maybe pricing arbitrage? Every gamer I know has like 20-30 titles in their queue they want to try out. To purchase all of them only for a few moments play would be prohibitive. Pay per use makes more sense if it falls below current gpu cloud cost (~$1/hr). What's amazing is that if you can deliver video games at 4K 60fps. You can deliver all but the most intensive any application this way.

deleted 8y ago

[deleted]

wpdev_63 8y ago

How does this service compare to parsec[0]? It looks good but parsec just came out with in browser streaming for its services and runs on top of aws. [0]:parsecgaming.com

andrewmd5 8y ago
I will avoid getting into how our streaming performs vs. theirs because we have a lot of resources on why our tech is so fast[0] That being said, as far as I know, Parsec's new browser-based client suffers from all the issues I mentioned in this blog and at the end of the day, our user experiences are completely different. Rainway being self-hostable you can put it on everything from Azure[1] to your home computer. We have a very unique "Games First"[3] approach and view ourselves more as a gaming service than a piece of game streaming software. As we continue to fledge out our voice, I think you'll be pleased with the results. [0]:https://blog.rainway.io/here-whats-new-in-rainway-0-5-0-3a66714a8b46 https://blog.rainway.io/here-whats-new-in-rainway-0-5-0-3a66... [1]:https://www.youtube.com/watch?v=OlMq-3hmm5Y&list=PLaVZpLEYEQNz34jHAiPcHTZ8ANr7Vkn3T https://www.youtube.com/watch?v=OlMq-3hmm5Y&list=PLaVZpLEYEQ... [2]:https://blog.rainway.io/our-core-mission-games-first-78671e496201 https://blog.rainway.io/our-core-mission-games-first-78671e4...

ganaz 8y ago

so awesome