5 ms·

I live in Kansas City and have enjoyed Google Fiber for the last three years. (First the free 5Mbps, then the $70/month Gigabit) The answer to this question, f

I live in Kansas City and have enjoyed Google Fiber for the last three years. (First the free 5Mbps, then the $70/month Gigabit) The answer to this question, for me, is "Doin' the same shit. Slightly faster." The speed test results are always fantastic, but real internet usage isn't quite as utopian as the hype makes it out to be. In some cases, I suspect there are bottlenecks between my house and whatever server I'm connecting to. In other cases, it almost feels like intentional throttling. Netflix and Hulu performance is outstanding, but Amazon Prime videos occasionally drop into that super-blocky low bandwidth mode for seemingly no reason. Most websites work great, but I've had super weird experiences with sites and servers that use IPV6 when I'm on my MacBook. It's these weird quirks that probably keep me from noticeably changing my internet habits. But, as usual, YMMV.

jonny_eh 11y ago
Not to mention that most devices these days connect over wifi, which almost always offers less than 1Gbps speed.
- noondip 11y ago
  Even using Ethernet and open source firmware on consumer routers (e.g., TomatoUSB on Broadcom-based ASUS), I wasn't able to get >100 Mbps. I guess many developers' own Internet connections don't have such bandwidth, so they're not incentivized to make improvements.
  - simoncion 11y ago
    Yeah, you have to do a bit of research to find routers that can route/firewall at ~1Gbps. The Ubiquiti EdgeRouter Lite is one such router, with a typical retail price of just south of 100 USD. (Though -for whatever reason- you do have to manually enable hardware offload to get 1Gbps perf out of the device.)
    - noondip 11y ago
      Here is the system I purchased - http://www.pcengines.ch/apu1d4.htm http://www.pcengines.ch/apu1d4.htm With both OpenBSD and Debian, I've had no problems routing at gigabit speed locally and the maximum 350 Mbps on WAN.
      - simoncion 11y ago
        That looks pretty neat. Can you route and firewall at gigabit rates? Also, it looks like it would cost ~150->300 USD to make a complete (board, case, storage, AC adaptor) router?
        noondip 11y ago
        That's right about the cost. I think it's a good deal compared to something like https://store.pfsense.org/SG-2440/ https://store.pfsense.org/SG-2440/ What do you mean by firewall, exactly? I do nothing extreme, just an pf or iptables ruleset, run dnsmasq, privoxy - those sorts of things. Glad to take some performance benchmarks you can recommend.
        simoncion 11y ago
        > What do you mean by firewall, exactly? The basic stuff: NAT translation, port forwarding, connection rejection and the like. For bonus points, doing stuff like traffic shaping/prioritizing (like CoDel) at gigabit speed would be rather interesting. At the moment the EdgeRouter Lite can do limited amounts of packet inspection using the offload hardware. As time goes on the Ubiquiti folks figure out how to better use the offload chip, but -for now- rate limiting and traffic shaping has to run through the thing's CPU, which -IIRC- gives you somewhere north of 100mbit/s of throughput. > I think it's a good deal compared to something like... Oh, for something that I would expect to be able to keep using for 10 years, I think it's quite a reasonable price. :D Edit: Yeah, I don't really have any perf benchmarks to recommend. I guess -if I had the time and the gumption- I'd do something like set up iperf (probably 3 so you can use TCP) on two machines (each in a different subnet so packets would pass through the router) and adjust the iperf listen port so that my various firewall rules triggered, and compare performance.
  - phinnaeus 11y ago
    My mid-range router simply didn't have a fast enough processor to handle Gigabit speeds, I ended up upgrading shortly after having the service installed in my apartment to a much nicer device. It was well worth it.
  - jeff_marshall 11y ago
    For me, it's less about the throughput than the total experience given the services I use. I have a less than stellar connection for the US (~25/5-ish?), and getting my voip to work took some real tuning in PFsense. The move to PFsense was basically motivated by the fact that I couldn't even do the things I wanted to prioritize voip via the gui provided by the mainstream combo wifi/router I had before. Without tuning (firewall, qos settings), it would frequently sound terrible as soon as someone fired up something bandwidth intensive. I wonder how much is HW vs. SW, since the router I'm using these days is a low-end single core (2x hyper thread) atom box from Soekris that rarely has >10% cpu load. There appear to be much faster HW options available from various vendors these days at a lower price point than I paid for the soekris when it was new, but PFsense wasn't available on my cheap MIPS wifi-router.
  - riebschlager 11y ago
    Google provided both modems and routers, which was absolutely essential. Very few people had/have the hardware to take advantage of the fiber speed. Anyone who wants to roll out a fast service should certainly take that cue.
    - wtallis 11y ago
      Anyone looking to roll out fast service should try to get Google to sell them the routers, because Google actually bothered to do the routers and modems right in ways that almost no retail modems and routers do, and few ISPs would be able to even develop detailed product requirements for.
  - wtallis 11y ago
    Using open-source firmware won't get you much in the way of performance benefits if it's not using recent Linux kernels and well-maintained in-tree drivers. And it was only recently that any routers started including CPUs that are powerful enough for high-speed packet processing. Prior to 802.11ac, basically everything was based off '90s-era single-core MIPS and the only hope of 100+Mbps throughput was to use hardware offloads that severely curtail the kinds of packet processing you can accomplish (and usually isn't supported by any open-source driver). Now we've got 1+GHz multi-core ARM processors in wireless router SoCs, but no mature drivers for them.
  - casylum 11y ago
    You should check out the Edge Router series from Ubiquity. They are cheap ($50-$350) and can route traffic at gigabit speeds. Here is the Edge Router Lite http://amzn.to/1UhB9SF http://amzn.to/1UhB9SF It's not a 'lazy developer' issue, but hardware really is the limiting factor. You would be surprised how much a router can impact your speed once you get above 50Mbps.
    - anon4 11y ago
      Another brand to recommend is MikroTik. Depending on your needs they have routers that can scale from small home use to medium - large office.
xirdstl 11y ago
I have that same experience with Amazon Prime videos. I instinctively want to blame Amazon, but who knows?
- jlgaddis 11y ago
  "Me three" re: Amazon Prime videos. Netflix and Hulu work wonderfully. We only occasionally watch something on Amazon Prime and I don't think there's been one time that we didn't have an issue with it (and I have fiber at home, 50x25 Mbps, and my ISP is well-connected network-wise).
  - ohitsdom 11y ago
    I also blame Amazon. I only have 15mbps down, but Netflix never has a problem after it initially buffers. Amazon almost always has a few buffering periods throughout a video.
cm2187 11y ago
To me the main difference is upload speed. Suddenly you can start using the internet as a LAN, and then only it starts to make sense to work in the cloud, save all your files there, not worry about the house burning down or being burglarised. Probably even more useful for smaller businesses.
- riebschlager 11y ago
  I totally agree with that. I can throw a 100MB ZIP into my Dropbox folder without a second thought. Box and Google Drive are equally snappy. With Google Fiber "Free" and Time Warner before it, I was capped at 1 Mbps upstream and cloud services were much more or a chore.
  - revelation 11y ago
    In light of this, would you now revise your original comment?
    - riebschlager 11y ago
      Not really. Before or after gigabit, I'd throw that file in my Dropbox. It just sucks less now. A fast connection doesn't mean that I'm deciding to throw more stuff online or make more use of cloud services. It's just nicer that the stuff I have to transfer moves a helluva lot faster now. So I haven't noticed a significant difference in my usage.
auganov 11y ago
1gbit/s is just under 120MB/s - the average read/write speed of an HDD. So yea, maybe you can backup your personal PC or two, but you still won't be using it for frequent reads/writes (latency issues aside). I have 1GbE LAN cabling at home and I'd really want 10GbE. Also to put things in perspective - HDMI 2.0 does 18gbit/s. PCIE v 4.0 x 16 rounds down to 260 gbit/s of bandwidth. Now please forgive me for being ridiculous and quoting these figures. :) But I don't think you need to be that crazy to imagine a few cool use cases for such bandwidths. Or the kind of resource wastefulness (in a positive sense) it would allow :-D My point is - 1gbit/s - not as utopian as it first sounds.
- infogulch 11y ago
  So a 1gbit/s connection like google fiber provides is around the same bandwidth as a typical hdd. A typical hdd has around 13ms of latency, which is also easily in the realm of latency to a regional datacenter. So any technical barriers to making all hdds obsolete are evaporating. I guess with the rise of ssds the only draw of hdds is now price, and I suppose that feature won't be supplanted by some service provider any time soon.
goda90 11y ago
I had Google fiber in an apartment with roommates in Provo Utah. In the same way, individual tasks didn't seem like they were amazingly faster(bottlenecks and device speed could have often been the culprit for sure) but the parallelism was great. Netflix, Clash of Clans and Battlefield 4 all running through the same pipe without a hitch. We never felt like we were stepping on each other's toes bandwidth wise. And I rarely felt the need to turn off my backup utility or stop large Steam downloads. Fun story though, it was fast enough that I got kicked for an hour from my school's network because I was downloading way too much output data, way too quickly over SCP from the school supercomputer.

jonny_eh 11y ago

Not to mention that most devices these days connect over wifi, which almost always offers less than 1Gbps speed.

noondip 11y ago
Even using Ethernet and open source firmware on consumer routers (e.g., TomatoUSB on Broadcom-based ASUS), I wasn't able to get >100 Mbps. I guess many developers' own Internet connections don't have such bandwidth, so they're not incentivized to make improvements.
- simoncion 11y ago
  Yeah, you have to do a bit of research to find routers that can route/firewall at ~1Gbps. The Ubiquiti EdgeRouter Lite is one such router, with a typical retail price of just south of 100 USD. (Though -for whatever reason- you do have to manually enable hardware offload to get 1Gbps perf out of the device.)
  - noondip 11y ago
    Here is the system I purchased - http://www.pcengines.ch/apu1d4.htm http://www.pcengines.ch/apu1d4.htm With both OpenBSD and Debian, I've had no problems routing at gigabit speed locally and the maximum 350 Mbps on WAN.
    - simoncion 11y ago
      That looks pretty neat. Can you route and firewall at gigabit rates? Also, it looks like it would cost ~150->300 USD to make a complete (board, case, storage, AC adaptor) router?
      - noondip 11y ago
        That's right about the cost. I think it's a good deal compared to something like https://store.pfsense.org/SG-2440/ https://store.pfsense.org/SG-2440/ What do you mean by firewall, exactly? I do nothing extreme, just an pf or iptables ruleset, run dnsmasq, privoxy - those sorts of things. Glad to take some performance benchmarks you can recommend.
        simoncion 11y ago
        > What do you mean by firewall, exactly? The basic stuff: NAT translation, port forwarding, connection rejection and the like. For bonus points, doing stuff like traffic shaping/prioritizing (like CoDel) at gigabit speed would be rather interesting. At the moment the EdgeRouter Lite can do limited amounts of packet inspection using the offload hardware. As time goes on the Ubiquiti folks figure out how to better use the offload chip, but -for now- rate limiting and traffic shaping has to run through the thing's CPU, which -IIRC- gives you somewhere north of 100mbit/s of throughput. > I think it's a good deal compared to something like... Oh, for something that I would expect to be able to keep using for 10 years, I think it's quite a reasonable price. :D Edit: Yeah, I don't really have any perf benchmarks to recommend. I guess -if I had the time and the gumption- I'd do something like set up iperf (probably 3 so you can use TCP) on two machines (each in a different subnet so packets would pass through the router) and adjust the iperf listen port so that my various firewall rules triggered, and compare performance.
- phinnaeus 11y ago
  My mid-range router simply didn't have a fast enough processor to handle Gigabit speeds, I ended up upgrading shortly after having the service installed in my apartment to a much nicer device. It was well worth it.
- jeff_marshall 11y ago
  For me, it's less about the throughput than the total experience given the services I use. I have a less than stellar connection for the US (~25/5-ish?), and getting my voip to work took some real tuning in PFsense. The move to PFsense was basically motivated by the fact that I couldn't even do the things I wanted to prioritize voip via the gui provided by the mainstream combo wifi/router I had before. Without tuning (firewall, qos settings), it would frequently sound terrible as soon as someone fired up something bandwidth intensive. I wonder how much is HW vs. SW, since the router I'm using these days is a low-end single core (2x hyper thread) atom box from Soekris that rarely has >10% cpu load. There appear to be much faster HW options available from various vendors these days at a lower price point than I paid for the soekris when it was new, but PFsense wasn't available on my cheap MIPS wifi-router.
- riebschlager 11y ago
  Google provided both modems and routers, which was absolutely essential. Very few people had/have the hardware to take advantage of the fiber speed. Anyone who wants to roll out a fast service should certainly take that cue.
  - wtallis 11y ago
    Anyone looking to roll out fast service should try to get Google to sell them the routers, because Google actually bothered to do the routers and modems right in ways that almost no retail modems and routers do, and few ISPs would be able to even develop detailed product requirements for.
- wtallis 11y ago
  Using open-source firmware won't get you much in the way of performance benefits if it's not using recent Linux kernels and well-maintained in-tree drivers. And it was only recently that any routers started including CPUs that are powerful enough for high-speed packet processing. Prior to 802.11ac, basically everything was based off '90s-era single-core MIPS and the only hope of 100+Mbps throughput was to use hardware offloads that severely curtail the kinds of packet processing you can accomplish (and usually isn't supported by any open-source driver). Now we've got 1+GHz multi-core ARM processors in wireless router SoCs, but no mature drivers for them.
- casylum 11y ago
  You should check out the Edge Router series from Ubiquity. They are cheap ($50-$350) and can route traffic at gigabit speeds. Here is the Edge Router Lite http://amzn.to/1UhB9SF http://amzn.to/1UhB9SF It's not a 'lazy developer' issue, but hardware really is the limiting factor. You would be surprised how much a router can impact your speed once you get above 50Mbps.
  - anon4 11y ago
    Another brand to recommend is MikroTik. Depending on your needs they have routers that can scale from small home use to medium - large office.

xirdstl 11y ago

I have that same experience with Amazon Prime videos. I instinctively want to blame Amazon, but who knows?

jlgaddis 11y ago
"Me three" re: Amazon Prime videos. Netflix and Hulu work wonderfully. We only occasionally watch something on Amazon Prime and I don't think there's been one time that we didn't have an issue with it (and I have fiber at home, 50x25 Mbps, and my ISP is well-connected network-wise).
- ohitsdom 11y ago
  I also blame Amazon. I only have 15mbps down, but Netflix never has a problem after it initially buffers. Amazon almost always has a few buffering periods throughout a video.

cm2187 11y ago

To me the main difference is upload speed. Suddenly you can start using the internet as a LAN, and then only it starts to make sense to work in the cloud, save all your files there, not worry about the house burning down or being burglarised. Probably even more useful for smaller businesses.

riebschlager 11y ago
I totally agree with that. I can throw a 100MB ZIP into my Dropbox folder without a second thought. Box and Google Drive are equally snappy. With Google Fiber "Free" and Time Warner before it, I was capped at 1 Mbps upstream and cloud services were much more or a chore.
- revelation 11y ago
  In light of this, would you now revise your original comment?
  - riebschlager 11y ago
    Not really. Before or after gigabit, I'd throw that file in my Dropbox. It just sucks less now. A fast connection doesn't mean that I'm deciding to throw more stuff online or make more use of cloud services. It's just nicer that the stuff I have to transfer moves a helluva lot faster now. So I haven't noticed a significant difference in my usage.

auganov 11y ago

1gbit/s is just under 120MB/s - the average read/write speed of an HDD. So yea, maybe you can backup your personal PC or two, but you still won't be using it for frequent reads/writes (latency issues aside). I have 1GbE LAN cabling at home and I'd really want 10GbE. Also to put things in perspective - HDMI 2.0 does 18gbit/s. PCIE v 4.0 x 16 rounds down to 260 gbit/s of bandwidth. Now please forgive me for being ridiculous and quoting these figures. :) But I don't think you need to be that crazy to imagine a few cool use cases for such bandwidths. Or the kind of resource wastefulness (in a positive sense) it would allow :-D My point is - 1gbit/s - not as utopian as it first sounds.

infogulch 11y ago
So a 1gbit/s connection like google fiber provides is around the same bandwidth as a typical hdd. A typical hdd has around 13ms of latency, which is also easily in the realm of latency to a regional datacenter. So any technical barriers to making all hdds obsolete are evaporating. I guess with the rise of ssds the only draw of hdds is now price, and I suppose that feature won't be supplanted by some service provider any time soon.

goda90 11y ago

I had Google fiber in an apartment with roommates in Provo Utah. In the same way, individual tasks didn't seem like they were amazingly faster(bottlenecks and device speed could have often been the culprit for sure) but the parallelism was great. Netflix, Clash of Clans and Battlefield 4 all running through the same pipe without a hitch. We never felt like we were stepping on each other's toes bandwidth wise. And I rarely felt the need to turn off my backup utility or stop large Steam downloads. Fun story though, it was fast enough that I got kicked for an hour from my school's network because I was downloading way too much output data, way too quickly over SCP from the school supercomputer.