3 ms·

Excellent! Thanks for taking the time to write up a more technical look at the changes you guys made. I'm still not clear on how Akamai helps speed things up.

by bprater 18y ago

Excellent! Thanks for taking the time to write up a more technical look at the changes you guys made. I'm still not clear on how Akamai helps speed things up. I understand how CDNs work: they move static files to edge servers that are physically closer to customers, which mean less hops that data has to move through. But this isn't static stuff. Let me ask: is the general concept this -- prior to Akamai, users would have to hop through a dozen servers but now they only have to go through 2 or 3 because Akamai uses its own network? (Essentially bypassing the general Internet and using a type of private internet.)

gecko 18y ago
Basically, yes. We're hopping off the Internet at the nearest Akamai hub, bypassing the normal backbone, and then hopping back on close to the endpoint. I'm in the office at the moment, so I can't test this for obvious reasons, but you should see the effect by doing a traceroute to webnlbcopilot.fogcreek.com and comparing it to web1copilot.fogcreek.com. The former should go through Akamai, the latter will route directly to us.
- bprater 18y ago
  Thanks for the insight. I didn't realize that CDN's like Akamai used a private type of network. Makes good sense.
- jrockway 18y ago
  Interesting. I actually ran the traceroute. To the webnlbcopilot server, it's 18 hops, and a few of the machines along the way are named like "deploy.akamaitechnologies.com". (So I assume this is the Akamai route.) The latency on this route is 38ms or so. The other route is only 14 hops, but the ping time is 54ms. So yeah, interesting. More hops, but measurably faster.
- andrewtj 18y ago
  From Melbourne Australia: traceroute to a259.srip1.akasrip.net.a9502240.1.cn.akasripcn.net (59.167.187.149), 64 hops max, 40 byte packets <snip> 12 64.34.60.28 (64.34.60.28) 248.487 ms !H 255.732 ms !H 257.461 ms !H traceroute to web1copilot.fogcreek.com (64.34.80.167), 64 hops max, 40 byte packets <snip> 19 64.34.60.28 (64.34.60.28) 291.594 ms !H 288.784 ms !H 289.873 ms !H I'd guess that's still way too high for CoPilot to be comfortable to use but I'm impressed with the improvement none the less.
  - bemmu 18y ago
    From Tampere Finland: traceroute to webnlb.fogcreek.com (64.34.80.173), 64 hops max, 40 byte packets <snip> 14 64.34.60.28 (64.34.60.28) 147.040 ms !H 147.266 ms !H 148.065 ms !H traceroute to web1copilot.fogcreek.com (64.34.80.167), 64 hops max, 40 byte packets <snip> 14 64.34.60.28 (64.34.60.28) 147.379 ms !H 146.844 ms !H 147.589 ms !H The routes look identical. Wonder if I tracerouted to the wrong place? Not sure where andrewtj got a259.srip1.akasrip.net.a9502240.1.cn.akasripcn.net
    - mhp 18y ago
      Your first traceroute should be to webnlbcopilot.fogcreek.com not webnlb.fogcreek.com
- Andys 18y ago
  So to cut a long story short, there are now two Internets. Or many Internets, if you like. There's the default basic one, which is the slowest and laggiest "consumer grade" Internet. Then you can pay more to a company like Akamai to have your traffic go over a different network that is properly designed and provisioned for less data contention.
  - blasdel 18y ago
    You're right that there are many Internets, but there isn't a "default basic internet" any more than there's a "default basic airplane route" The distributions of Airlines and Bandwidth Providers are fairly similar in number and size, and the customer patterns are quite similar too. Very few people buy directly from the providers, they go through retail intermediaries to get the lowest prices (booking agents/ISPs). The people who do buy directly either have some sort of 'relationship' with the provider, or are massive enough to avoid buying anything at the retail level. The hub-and-spoke networks with routes merging mid-stream are also quite similar topographically (though not geographically). Akamai's network isn't any more "properly designed", they're just willing to pay the actual Bandwidth Providers more, and more importantly willing to pay up front for guaranteed bandwidth. Peering bandwidth is normally billed by volume used, with a fee based on your maximum thouroughput over the billing period.
    - Andys 18y ago
      Anyone could always buy point to point IP links with lower contention ratios, but this is the first time I've seen a company (Akamai) with enough of them internationally to have a serious attempt to have full Internet coverage.

by bprater 18y ago

gecko 18y ago

Basically, yes. We're hopping off the Internet at the nearest Akamai hub, bypassing the normal backbone, and then hopping back on close to the endpoint. I'm in the office at the moment, so I can't test this for obvious reasons, but you should see the effect by doing a traceroute to webnlbcopilot.fogcreek.com and comparing it to web1copilot.fogcreek.com. The former should go through Akamai, the latter will route directly to us.

bprater 18y ago
Thanks for the insight. I didn't realize that CDN's like Akamai used a private type of network. Makes good sense.
jrockway 18y ago
Interesting. I actually ran the traceroute. To the webnlbcopilot server, it's 18 hops, and a few of the machines along the way are named like "deploy.akamaitechnologies.com". (So I assume this is the Akamai route.) The latency on this route is 38ms or so. The other route is only 14 hops, but the ping time is 54ms. So yeah, interesting. More hops, but measurably faster.
andrewtj 18y ago
From Melbourne Australia: traceroute to a259.srip1.akasrip.net.a9502240.1.cn.akasripcn.net (59.167.187.149), 64 hops max, 40 byte packets <snip> 12 64.34.60.28 (64.34.60.28) 248.487 ms !H 255.732 ms !H 257.461 ms !H traceroute to web1copilot.fogcreek.com (64.34.80.167), 64 hops max, 40 byte packets <snip> 19 64.34.60.28 (64.34.60.28) 291.594 ms !H 288.784 ms !H 289.873 ms !H I'd guess that's still way too high for CoPilot to be comfortable to use but I'm impressed with the improvement none the less.
- bemmu 18y ago
  From Tampere Finland: traceroute to webnlb.fogcreek.com (64.34.80.173), 64 hops max, 40 byte packets <snip> 14 64.34.60.28 (64.34.60.28) 147.040 ms !H 147.266 ms !H 148.065 ms !H traceroute to web1copilot.fogcreek.com (64.34.80.167), 64 hops max, 40 byte packets <snip> 14 64.34.60.28 (64.34.60.28) 147.379 ms !H 146.844 ms !H 147.589 ms !H The routes look identical. Wonder if I tracerouted to the wrong place? Not sure where andrewtj got a259.srip1.akasrip.net.a9502240.1.cn.akasripcn.net
  - mhp 18y ago
    Your first traceroute should be to webnlbcopilot.fogcreek.com not webnlb.fogcreek.com
Andys 18y ago
So to cut a long story short, there are now two Internets. Or many Internets, if you like. There's the default basic one, which is the slowest and laggiest "consumer grade" Internet. Then you can pay more to a company like Akamai to have your traffic go over a different network that is properly designed and provisioned for less data contention.
- blasdel 18y ago
  You're right that there are many Internets, but there isn't a "default basic internet" any more than there's a "default basic airplane route" The distributions of Airlines and Bandwidth Providers are fairly similar in number and size, and the customer patterns are quite similar too. Very few people buy directly from the providers, they go through retail intermediaries to get the lowest prices (booking agents/ISPs). The people who do buy directly either have some sort of 'relationship' with the provider, or are massive enough to avoid buying anything at the retail level. The hub-and-spoke networks with routes merging mid-stream are also quite similar topographically (though not geographically). Akamai's network isn't any more "properly designed", they're just willing to pay the actual Bandwidth Providers more, and more importantly willing to pay up front for guaranteed bandwidth. Peering bandwidth is normally billed by volume used, with a fee based on your maximum thouroughput over the billing period.
  - Andys 18y ago
    Anyone could always buy point to point IP links with lower contention ratios, but this is the first time I've seen a company (Akamai) with enough of them internationally to have a serious attempt to have full Internet coverage.