6 ms·

Putting a Price on Latency: Startup Network Next Aims to Fix Internet Routing

31 points by gafferongames 7y ago

ptah 7y ago
How does this affect net neutrality
- jsty 7y ago
  In the typical sense of net neutrality[0], not very much - you're effectively bidding for a faster / lower-latency route across the internet. You can already pay for faster routes with e.g. Google's premium networking or Cloudflare's Argo. If I understand correctly, this offering just means that instead of having to procure your network in advance and lock in a particular price for all traffic, you can dynamically route traffic based on the cost of doing so. [0] Wikipedia's definition: "Network neutrality, or simply net neutrality, is the principle that Internet service providers (ISPs) must treat all Internet communications equally, and not discriminate or charge differently based on user, content, website, platform, application, type of equipment, or method of communication."
- runeks 7y ago
  I don't know. But if net neutrality prevents me from paying extra, in order to decrease latency for a particular connection, I don't want it.
  - gafferongames 7y ago
    I'd like to just hop in and say that Network Next doesn't ask for any money from the player. Our customer is the game developer, who so far is very happy to target some bottom n% of the players who are having a terrible experience, and make that better with Network Next!
- toast0 7y ago
  This seems like it's something that would be happening on the content end and not the ISP end, because the ISPs generally don't care about latency or jitter. My understanding of net neutrality was that it is supposed to be a restriction on consumer residential/wireless ISPs' actions. If you had an effective choice in ISPs, you could pick the one with properties you cared for, like lower latency connections; early internet gaming networks had prefered dial-up providers that shaved about 10ms off of cross-US ping times.
  - PaulHoule 7y ago
    It is. Riot Games has built its own network from major ISPs to their data centers to support League of Legends. This is not a terribly latency sensitive game, but it is worth it for them to do it because players are more likely to keep playing if they are mad at themselves and their team for losing as opposed to being mad at "the internet". I assume Riot puts the service together by working with wholesalers and they probably have an "auction" on a timescale of months or years to determine routes.
PaulHoule 7y ago
Easier said then done. If you have 30 ms latency (e.g. my DSL to Riot Games) then you had better have the auction over and done with in much less than 30 ms. Also the latency that people perceive is not the 30 ms average latency, it is the spikes of 300 ms or more latency that sometimes happen. I think what people hate about computers the most is the way they become unresponsive for a few moments and you don't know if it is going to come back OK or not. I would rather have 30 ms latency that is rock solid as opposed to 20 ms average with 99% at 200 ms. Any route flapping at all is going to contribute to the latter.
- zemo 7y ago
  not a great example. Riot literally lays their own fiber. A lot of their traffic flows over private networks. See here: https://www.polygon.com/2016/2/17/11037220/league-of-legends-was-too-slow-so-riot-built-a-private-internet-to https://www.polygon.com/2016/2/17/11037220/league-of-legends... > it is the spikes of 300 ms or more latency that sometimes happen yes that's jitter, that's part of what they're trying to address. That's like ... the whole point; you're paying to avoid routing through intermediaries with unreliable performance characteristics. edit: I see your other comment now. It wasn't there when I wrote this comment. Leaving it as it was for posterity.
  - dylz 7y ago
    I find it hilarious that I peer with them for a total of 2 players. The thing about a lot of residential connections is that probably a not-insignificant amount of the problems are very close to last mile, as opposed to bandwidth availability at backbone or major interconnection points
    - izzydata 7y ago
      Someone 1 block away from me gets as low as 20ms latency for the same things that I get 50ms on. Pretty sure the last mile has the biggest potential improvements.
  - wbl 7y ago
zemo 7y ago
this makes me sad. I've been following Glenn on Twitter for a few years. I like his writing on netcode. I'm a multiplayer server engineer at my dayjob. The crux of this system is that the amount of benefit you get out of it is a function of how much money you put into it. I can't believe that he would say ... this, of all things: > What happening that I think is terrible is that the public Internet is turning into this neglected commons He's directly creating a platform that exacerbates that very problem by finding loopholes in how network neutrality policy is structured to create fast lanes. This in turn will create incentives for providers to put their most performant systems on private networks and their least performant systems on the public internet. The greater the gap in performance between the public internet and the services of Network Next, the more valuable Network Next is. It's a feedback loop that will print money. It's a brilliant act of capitalism that is doomed to succeed at the expense of the already-sparse indie multiplayer scene.
- gafferongames 7y ago
  > What happening that I think is terrible is that the public Internet is turning into this neglected commons This is a past tense. It has already happened. What if indie games could access the same benefits of something like Riot Direct, democratized so they can use it at a reasonable price without having to build 10s of millions of dollars worth of their own private network infrastructure? Where we're doing at Network Next is democratizing access to the private networks, not creating more of them.
  - zemo 7y ago
    I think it has a very good chance of succeeding, and I think it's a very smart business strategy. I know who you are, and I know what your work means in the game industry. And honestly I've learned a lot from your writing and talks, and if you get filthy rich, well, you've put in the work and if anyone deserves it, sure, great. It strikes me as odd to describe your actions as "democratizing access" when what you're doing is basically the textbook definition of market making. Markets structure access by capital; democracies are ideally structured to provide equal access that is unrelated to participants' access to capital. By transforming private networks into fungible commodities, you will ideally raise the utilization of said private networks and in turn raise the return on investment of building such networks. You say you're not creating more private networks, but again, I think this is a useful turn of phrase that is not representative of the consequences of your actions. Even when you're not making more private networks yourself, if you build an efficient market for commodifying access to private networks, shouldn't that create incentives for others to create new private networks? I'm not questioning your motives; I'm questioning what the consequences of your actions will be. Since the bill here is footed by the game developer (and not the player), the cost barrier that indies face in making multiplayer games is further tilted in the favor of AAA studios. In short: great for AAA studios since it makes it harder for smaller studios to compete, great for private network operators since it provides liquidity, great for gamers that want to play AAA multiplayer games, bad for indie game studios because it will widen the latency gap between public and private networks, bad for gamers that want to play indie multiplayer games. The end result of this appears to be a system whose consequence will be to create market forces that encourage the centralization of capital and the creation of new private networks, while also adversely affecting the landscape of which game developers are capable of creating latency-sensitive multiplayer games. That is what I find concerning.
halbritt 7y ago
This is a non-starter. There is zero chance these guys will be able to influence peering relationships. As for the public Internet, it's been dead at any real level for over a decade. I'm not sure what kind of latency issues other folks experience, but they are typically constrained by speed of light. People with big content networks like Google are going to directly peer with people with big eyeball networks like AT&T. Presuming that peering relationship is functional, the interconnects are not going to be over-subscribed and latency is going to be optimal between any two given points with no quality of service required, nor really any change for a third party to influence the quality of the interconnect. This all presumes the network use case is gaming which has orders of magnitude less throughput required than streaming high quality video which is a different problem altogether.
- wbl 7y ago
  Define dead and public. If you say public is without peering then that is just the Tier 1s you need to pay money to in order to use.
  - halbritt 7y ago
    "Public" in the old school sense like PAIX or MAE-West or some such. "Dead" in the sense that you can't use them to move any meaningful amount of traffic. There are still IXPs in the world (i.e. Equinix) where one can move a meaningful amount of traffic and you're perfectly correct in that you'll still need to pay for transit for the tier 1s.
- a13n 7y ago
  This comment doesn't really show a great understanding of the problems that gamers experience when it comes to latency. Distance / the speed of light has nothing to do with it. If you're in the US connecting to a US game server, neither you nor the game developers have control over what path the internet will choose to send your traffic there. Your packets could be blocked by other packets being sent first. You could experience packet loss, which will create a rubber banding effect in game. You could experience a variation in packet delivery time, which makes it hard for client-side prediction to work effectively. These issues frustrate gamers to the point where they'll stop playing if it's bad enough. Lag is hated by games and developers, so this problem is super worth solving. Imagine what it'd be worth to Epic Games to reduce their player churn by just 0.1% monthly... probably millions! This is a big market. If you read up on Network Next, they solve this problem by sending your traffic on private networks where you do have more control over delivery and consistency, rather than public networks where you don't. I'm really excited to see Network Next improve gameplay, and it's 100% not a non-starter.
musicale 7y ago
Paying for low-latency/low-bandwidth (e.g. game traffic, voip) data pipes is the opposite of network neutrality, but I'm kind of OK with it as long as 1) there is clarity/visibility in terms of how it operates, and 2) control isn't taken away from end users who want control. Limiting it to low-bandwidth would discourage misuse and could enable offloading the decision of how and when to use it to the end user or application. Some users could reserve it for gaming, others for voip, others for something else. Some might try using it for netflix or web browsing but they would probably find that to be undesirable because of the low bandwidth. You don't really need a private network for it either - there's no reason you couldn't have low-latency/low-bandwidth peering across regular ISPs.

31 points by gafferongames 7y ago

ptah 7y ago

How does this affect net neutrality

jsty 7y ago
In the typical sense of net neutrality[0], not very much - you're effectively bidding for a faster / lower-latency route across the internet. You can already pay for faster routes with e.g. Google's premium networking or Cloudflare's Argo. If I understand correctly, this offering just means that instead of having to procure your network in advance and lock in a particular price for all traffic, you can dynamically route traffic based on the cost of doing so. [0] Wikipedia's definition: "Network neutrality, or simply net neutrality, is the principle that Internet service providers (ISPs) must treat all Internet communications equally, and not discriminate or charge differently based on user, content, website, platform, application, type of equipment, or method of communication."
runeks 7y ago
I don't know. But if net neutrality prevents me from paying extra, in order to decrease latency for a particular connection, I don't want it.
- gafferongames 7y ago
  I'd like to just hop in and say that Network Next doesn't ask for any money from the player. Our customer is the game developer, who so far is very happy to target some bottom n% of the players who are having a terrible experience, and make that better with Network Next!
toast0 7y ago
This seems like it's something that would be happening on the content end and not the ISP end, because the ISPs generally don't care about latency or jitter. My understanding of net neutrality was that it is supposed to be a restriction on consumer residential/wireless ISPs' actions. If you had an effective choice in ISPs, you could pick the one with properties you cared for, like lower latency connections; early internet gaming networks had prefered dial-up providers that shaved about 10ms off of cross-US ping times.
- PaulHoule 7y ago
  It is. Riot Games has built its own network from major ISPs to their data centers to support League of Legends. This is not a terribly latency sensitive game, but it is worth it for them to do it because players are more likely to keep playing if they are mad at themselves and their team for losing as opposed to being mad at "the internet". I assume Riot puts the service together by working with wholesalers and they probably have an "auction" on a timescale of months or years to determine routes.

PaulHoule 7y ago

Easier said then done. If you have 30 ms latency (e.g. my DSL to Riot Games) then you had better have the auction over and done with in much less than 30 ms. Also the latency that people perceive is not the 30 ms average latency, it is the spikes of 300 ms or more latency that sometimes happen. I think what people hate about computers the most is the way they become unresponsive for a few moments and you don't know if it is going to come back OK or not. I would rather have 30 ms latency that is rock solid as opposed to 20 ms average with 99% at 200 ms. Any route flapping at all is going to contribute to the latter.

zemo 7y ago
not a great example. Riot literally lays their own fiber. A lot of their traffic flows over private networks. See here: https://www.polygon.com/2016/2/17/11037220/league-of-legends-was-too-slow-so-riot-built-a-private-internet-to https://www.polygon.com/2016/2/17/11037220/league-of-legends... > it is the spikes of 300 ms or more latency that sometimes happen yes that's jitter, that's part of what they're trying to address. That's like ... the whole point; you're paying to avoid routing through intermediaries with unreliable performance characteristics. edit: I see your other comment now. It wasn't there when I wrote this comment. Leaving it as it was for posterity.
- dylz 7y ago
  I find it hilarious that I peer with them for a total of 2 players. The thing about a lot of residential connections is that probably a not-insignificant amount of the problems are very close to last mile, as opposed to bandwidth availability at backbone or major interconnection points
  - izzydata 7y ago
    Someone 1 block away from me gets as low as 20ms latency for the same things that I get 50ms on. Pretty sure the last mile has the biggest potential improvements.
- wbl 7y ago

zemo 7y ago

this makes me sad. I've been following Glenn on Twitter for a few years. I like his writing on netcode. I'm a multiplayer server engineer at my dayjob. The crux of this system is that the amount of benefit you get out of it is a function of how much money you put into it. I can't believe that he would say ... this, of all things: > What happening that I think is terrible is that the public Internet is turning into this neglected commons He's directly creating a platform that exacerbates that very problem by finding loopholes in how network neutrality policy is structured to create fast lanes. This in turn will create incentives for providers to put their most performant systems on private networks and their least performant systems on the public internet. The greater the gap in performance between the public internet and the services of Network Next, the more valuable Network Next is. It's a feedback loop that will print money. It's a brilliant act of capitalism that is doomed to succeed at the expense of the already-sparse indie multiplayer scene.

gafferongames 7y ago
> What happening that I think is terrible is that the public Internet is turning into this neglected commons This is a past tense. It has already happened. What if indie games could access the same benefits of something like Riot Direct, democratized so they can use it at a reasonable price without having to build 10s of millions of dollars worth of their own private network infrastructure? Where we're doing at Network Next is democratizing access to the private networks, not creating more of them.
- zemo 7y ago
  I think it has a very good chance of succeeding, and I think it's a very smart business strategy. I know who you are, and I know what your work means in the game industry. And honestly I've learned a lot from your writing and talks, and if you get filthy rich, well, you've put in the work and if anyone deserves it, sure, great. It strikes me as odd to describe your actions as "democratizing access" when what you're doing is basically the textbook definition of market making. Markets structure access by capital; democracies are ideally structured to provide equal access that is unrelated to participants' access to capital. By transforming private networks into fungible commodities, you will ideally raise the utilization of said private networks and in turn raise the return on investment of building such networks. You say you're not creating more private networks, but again, I think this is a useful turn of phrase that is not representative of the consequences of your actions. Even when you're not making more private networks yourself, if you build an efficient market for commodifying access to private networks, shouldn't that create incentives for others to create new private networks? I'm not questioning your motives; I'm questioning what the consequences of your actions will be. Since the bill here is footed by the game developer (and not the player), the cost barrier that indies face in making multiplayer games is further tilted in the favor of AAA studios. In short: great for AAA studios since it makes it harder for smaller studios to compete, great for private network operators since it provides liquidity, great for gamers that want to play AAA multiplayer games, bad for indie game studios because it will widen the latency gap between public and private networks, bad for gamers that want to play indie multiplayer games. The end result of this appears to be a system whose consequence will be to create market forces that encourage the centralization of capital and the creation of new private networks, while also adversely affecting the landscape of which game developers are capable of creating latency-sensitive multiplayer games. That is what I find concerning.

halbritt 7y ago

This is a non-starter. There is zero chance these guys will be able to influence peering relationships. As for the public Internet, it's been dead at any real level for over a decade. I'm not sure what kind of latency issues other folks experience, but they are typically constrained by speed of light. People with big content networks like Google are going to directly peer with people with big eyeball networks like AT&T. Presuming that peering relationship is functional, the interconnects are not going to be over-subscribed and latency is going to be optimal between any two given points with no quality of service required, nor really any change for a third party to influence the quality of the interconnect. This all presumes the network use case is gaming which has orders of magnitude less throughput required than streaming high quality video which is a different problem altogether.

wbl 7y ago
Define dead and public. If you say public is without peering then that is just the Tier 1s you need to pay money to in order to use.
- halbritt 7y ago
  "Public" in the old school sense like PAIX or MAE-West or some such. "Dead" in the sense that you can't use them to move any meaningful amount of traffic. There are still IXPs in the world (i.e. Equinix) where one can move a meaningful amount of traffic and you're perfectly correct in that you'll still need to pay for transit for the tier 1s.
a13n 7y ago
This comment doesn't really show a great understanding of the problems that gamers experience when it comes to latency. Distance / the speed of light has nothing to do with it. If you're in the US connecting to a US game server, neither you nor the game developers have control over what path the internet will choose to send your traffic there. Your packets could be blocked by other packets being sent first. You could experience packet loss, which will create a rubber banding effect in game. You could experience a variation in packet delivery time, which makes it hard for client-side prediction to work effectively. These issues frustrate gamers to the point where they'll stop playing if it's bad enough. Lag is hated by games and developers, so this problem is super worth solving. Imagine what it'd be worth to Epic Games to reduce their player churn by just 0.1% monthly... probably millions! This is a big market. If you read up on Network Next, they solve this problem by sending your traffic on private networks where you do have more control over delivery and consistency, rather than public networks where you don't. I'm really excited to see Network Next improve gameplay, and it's 100% not a non-starter.

musicale 7y ago

Paying for low-latency/low-bandwidth (e.g. game traffic, voip) data pipes is the opposite of network neutrality, but I'm kind of OK with it as long as 1) there is clarity/visibility in terms of how it operates, and 2) control isn't taken away from end users who want control. Limiting it to low-bandwidth would discourage misuse and could enable offloading the decision of how and when to use it to the end user or application. Some users could reserve it for gaming, others for voip, others for something else. Some might try using it for netflix or web browsing but they would probably find that to be undesirable because of the low bandwidth. You don't really need a private network for it either - there's no reason you couldn't have low-latency/low-bandwidth peering across regular ISPs.