5 ms·

Wow. What kind of networking hardware do you push to/through? I have a hard time visualizing where/how you would "put" 200 gbps worth of data over (any) network

by e12e 7y ago

Wow. What kind of networking hardware do you push to/through? I have a hard time visualizing where/how you would "put" 200 gbps worth of data over (any) network...

joaovictortr 7y ago
There is a presentation from EuroBSDCon 2019, that describes the techniques and hardware used to achieve that figure [0]. There is also a thread in HN if you're interested [1]. [0] https://people.freebsd.org/~gallatin/talks/euro2019.pdf https://people.freebsd.org/~gallatin/talks/euro2019.pdf [1] https://news.ycombinator.com/item?id=21043765 https://news.ycombinator.com/item?id=21043765
drewg123 7y ago
Multiple Mellanox or Chelsio 100GbE NICs in an LACP bundle. The 220g is from a prototype with a Rome CPU in a Naples-designed motherboard, with 4 Mellanox ConnectX-4 100GbE NICs. 2 of them are limited to Gen 3 x8 (since it is a Naples generation mobo).
- Teknoman117 7y ago
  I'm curious about you being limited to 220 Gb/s on an AMD Rome box. The maximum theoretical DDR4 bandwidth on such a system with 3200 MT DDR4 is 204 GiB/s, so you'd only be using ~15% of the DRAM bandwidth assuming the block/record/file size of the cached data you're serving is enough to mask the DRAM latency.
rasz 7y ago
as to where, is internet infrastructure this bad in US? Even something like 35C3 had 500gbit pipe last year https://media.ccc.de/v/35c3-9576-35c3_infrastructure_review https://media.ccc.de/v/35c3-9576-35c3_infrastructure_review
- kev009 7y ago
  No, the US is one of the most densely connected countries in the world. Fiber was over-provisioned by the functional Bell monopoly, railroads, and a smattering of upstarts in the 1980s and massive infrastructure capital bubble in the 2000 dotcom crash which was more about this infrastructure than software or computer hardware. If you hear people in the US complaining about their internet connection, it is because the local duopoly of telephone and cable company is failing them. In terms of industrial fiber, the competition is fierce and you can light up Nx100G to almost anywhere in the country.
- Teknoman117 7y ago
  The industrial fiber capacity is fantastic here. What sucks is the consumer internet service. In most places in the US, there is a monopoly/duopoly for consumer ISPs and they certainly abuse their positions.

by e12e 7y ago

Wow. What kind of networking hardware do you push to/through? I have a hard time visualizing where/how you would "put" 200 gbps worth of data over (any) network...

joaovictortr 7y ago

There is a presentation from EuroBSDCon 2019, that describes the techniques and hardware used to achieve that figure [0]. There is also a thread in HN if you're interested [1]. [0] https://people.freebsd.org/~gallatin/talks/euro2019.pdf https://people.freebsd.org/~gallatin/talks/euro2019.pdf [1] https://news.ycombinator.com/item?id=21043765 https://news.ycombinator.com/item?id=21043765

drewg123 7y ago

Multiple Mellanox or Chelsio 100GbE NICs in an LACP bundle. The 220g is from a prototype with a Rome CPU in a Naples-designed motherboard, with 4 Mellanox ConnectX-4 100GbE NICs. 2 of them are limited to Gen 3 x8 (since it is a Naples generation mobo).

Teknoman117 7y ago
I'm curious about you being limited to 220 Gb/s on an AMD Rome box. The maximum theoretical DDR4 bandwidth on such a system with 3200 MT DDR4 is 204 GiB/s, so you'd only be using ~15% of the DRAM bandwidth assuming the block/record/file size of the cached data you're serving is enough to mask the DRAM latency.

rasz 7y ago

as to where, is internet infrastructure this bad in US? Even something like 35C3 had 500gbit pipe last year https://media.ccc.de/v/35c3-9576-35c3_infrastructure_review https://media.ccc.de/v/35c3-9576-35c3_infrastructure_review

kev009 7y ago
No, the US is one of the most densely connected countries in the world. Fiber was over-provisioned by the functional Bell monopoly, railroads, and a smattering of upstarts in the 1980s and massive infrastructure capital bubble in the 2000 dotcom crash which was more about this infrastructure than software or computer hardware. If you hear people in the US complaining about their internet connection, it is because the local duopoly of telephone and cable company is failing them. In terms of industrial fiber, the competition is fierce and you can light up Nx100G to almost anywhere in the country.
Teknoman117 7y ago
The industrial fiber capacity is fantastic here. What sucks is the consumer internet service. In most places in the US, there is a monopoly/duopoly for consumer ISPs and they certainly abuse their positions.