11 ms·

25 Gbit/s at home, part 1

131 points by pavel_odintsov 3y ago

hinkley 3y ago
What is the state of the art in leveraging your home network to have a presence (website or otherwise) on the internet? Fiber just became available in our neighborhood recently and I'm about to have fewer people sharing our network, so the idea of being able to host my own files is becoming attractive again. We are about to be due for this pendulum (centralized vs distributed) to swing back again and I like to be prepared.
- ricw 3y ago
  As long as you have a static IP address, this is trivial and typically very reliable. Throw an ATS battery into the mix and the internet will be super reliant, even when the power goes down. Am doing this at home with a fibre line and it's been super reliable.
  - chaxor 3y ago
    Dynamic dns is a thing?
    - xur17 3y ago
      My ISP uses dynamic dns, but it never changes unless my modem is down for several minutes. My router has a script that updates a dns entry if it changes, but realistically this rarely happens.
    - rjsw 3y ago
      The choice for my fibre ISP is between static IPv4 or carrier-grade NAT. Both options get IPv6.
- organsnyder 3y ago
  Cloudflare tunnels are easy to set up. Otherwise, serving directly can be an option if you have a public static IP from your ISP or a public dynamic ISP and are okay with doing dynamic DNS. IMHO the key is to have a good router/firewall that gives you the flexibility you need—ideally a custom box like OP. My software preference is opnsense, but there are plenty of other good options.
  - ozarker 3y ago
    I'm hosting my blog on machines running at home accessible via a Cloudflare tunnel. It works very well in my experience. And it's nice not having ports open to the internet or having to mess with dynamic dns.
- candiddevmike 3y ago
  You can run a website off a cellular connection, you don't need fiber for that. The pendulum has never left decentralized from a tech standpoint, folks are just too lazy. Home computer ownership is tanking, millennials were probably the last "power user" generation, and there's just no major interest in self hosting with the convenience of cloud. Even among tech professionals like HN.
  - quaintdev 3y ago
    r/selfhosted would like to have word with you. That sub is increasing steadily. https://www.reddit.com/r/selfhosted https://www.reddit.com/r/selfhosted
    - candiddevmike 3y ago
      I'm a member, I even built a self hosted home organizer (https://homechart.app https://homechart.app). It's a niche community with a very high cost of entry (time/money/skill).
      - nirav72 3y ago
        Regarding the money aspects of the high cost of entry - you can source cheap hardware from secondary markets. I just helped a friend setup a cheap selfhosting/homelab for his home. He bought couple of HP EliteDesk 800 G4 minis for $99/each with i5-8500T CPUs. Each came with 16gb ram. Both units had NVMe drives. So we just had to add couple of 2.5 SSDs 2tb ($50/each) for media storage. Running Ubuntu server with CasaOS ontop with handful of containers for media consumption (e.g Jellyfin) on one unit. The i5-8500T is new enough and supports Intel quicksync. So no GPU needed for transcoding. The second unit is running cloudflare tunnel daemon for things like his MicroBin(like pastebin), Wireguard VPN and NextCloud for external access. He still has plenty of room to add more applications to both servers.
        candiddevmike 3y ago
        Where'd you find those, ebay? That sounds like a great deal.
- vollmond 3y ago
  > Just then, a bandwidth load as heavy as a pregnant elephant sits down on Manfred's head and sends clumps of humongous pixilation flickering across his sensorium: Around the world, five million or so geeks are bouncing on his home site, a digital flash crowd alerted by a posting from the other side of the bar. Manfred winces. "I really came here to talk about the economic exploitation of space travel, but I've just been slashdotted. Mind if I just sit and drink until it wears off?" - Charlie Stross, "Accelerando"
  - V1ndaar 3y ago
    Damn, I read that book as a teenager in 2006 or 07! I've been thinking about a reread whenever I remember it. It was the book that really made me appreciate the whole technological singularity idea back then.
    - vollmond 3y ago
      I've been trying to find my paper copy all week so I can reread it. Might just give up and read it on his site: https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando.html https://www.antipope.org/charlie/blog-static/fiction/acceler...
- Symbiote 3y ago
  I just run a normal website through a normal (major company) connection with their supplied router. I have 10 minutes or so of power outage per year, otherwise it works fine. Maintenance effort is close to zero, security update apply automatically.
- h2odragon 3y ago
  I'm tinkering with a Raspi to do that sometime soon. Its going to be on an isolated network segment off an openwrt router that'll forward ports and hold the public ip. Technically it's ready to go but I'm enjoying tinkering with it still and no rush to put anything up. its not "state of the art" but I expect it to serve my needs.
- bombcar 3y ago
  For me the state of the art is realizing that you do NOT need 99.99999% uptime for personal stuff. Once there, do whatever you want. Get a DMZ or even a separate connection if you really want to go wild, but it's not terribly complicated. There are even "in a box" solutions - such as https://mailinabox.email https://mailinabox.email or https://owncloud.com https://owncloud.com
- nickspacek 3y ago
  I don't want to open my home network to just anybody, so I have a "jumpbox" that is the lowest-end shared VM at Hetzner. It runs nginx, dnsmasq, and Wireguard; my home servers connect to it, I add other peers as I need to, and dnsmasq resolves the hostnames using Zeroguard IPs for the home network. I have 3 sets of DNS entries for the home lab servers: 1. "internal"/home network addresses (e.g. your 192.168.x.x) 2. Wireguard addresses (e.g. 10.0.x.x) 3. public DNS entries that all resolve to the jumpbox The purpose of #3 is to support simple Letsencrypt setup: nginx on jumpbox forwards Letsencrypt requests to the internal servers over the Wireguard connection. Internally, I use a https://github.com/nginx-proxy/nginx-proxy https://github.com/nginx-proxy/nginx-proxy setup, so that any time I want a new service running inside the home lab I just have to: 1. Pick a hostname and add it to public DNS 2. Configure its Docker container to add the environment variables that nginx-proxy looks for 3. Add the hostname to the jumpbox /etc/hosts 4. Add the hostname to internal LAN DNS It's a little much but I like how it works. It's not so bad to get setup.
- tgtweak 3y ago
  reverse dns to catch your dynamic IP changes, and port forwarding/dmz on your modem/router should be all you need if your ISP doesn't block port 80/443. If your ISP does port block (or you don't want to port forward or expose your IP) you can also set up a cloudflare tunnel (warp?) on a server, and point it to your local webserver's port, and set your cloudflare DNS for your domain so that it points to your warp tunnel. This way requests for the website go to cloudflare then to your machine and on to your local webserver.
- fulafel 3y ago
  It depends, if you have real IP connectivity on your box you just run a web server and bob's your uncle. If you're stuck behind a nat for v4, try v6. Or port forwarding or VPN to a cloud provider or vps with better IP connectivity.
gorbypark 3y ago
I can upgrade from 1Gbit to 10Gbit for only 5 euros extra per month but I don't really have anything to take advantage of it. I don't think my new MacBook Air M2 even supports 6E, and wiring the apartment would be a bit of a hassle.
- bombcar 3y ago
  If you only have a few people or clients in the house, the upgrade is likely not worth it, though 10Gb would be nice to flaunt. At that price I'd take it, but that's because I have a 10G network to my NAS, though I'd have to upgrade my router.
- civiloid 3y ago
  With the size of recent games and patches, 1 Gbit starts to feel a bit too low. :) And about Macbook... In my current setup, I'm using Thunderbolt -> PCIe and one of the Mellanox cards; it gives excellent and stable 16 Gbps with stock firmware and DriverKit-based drivers (that comes with OSX), though it feels that DriverKit driver doesn't use any offloading that it supports (while running iperf between Macbook and router, it consumes roughly 3 cores entirely on Macbook, while the router doesn't even ramp up the single core to its boost frequency). But you are right; rewiring is a hassle, though Cat5e usually works with 10Gbit over reasonable lengths (10-30 meters, maybe a bit more if it is perfect quality cable). Also, even if I decided not to go for fibre, I'd still switch to at least 10Gbps because I'm not alone in that network :)
londons_explore 3y ago
How many hours can you saturate that 25 Gbit connection up and down before support calls you and tells you to knock it off?
- tgtweak 3y ago
  I ran about 8TB on my fiber in one month, and no ISP notices to speak of. That's only ~25mbps sustained. You could easily push 500+TB on a 25gbps connection if you were doing something naughty with it, and I'm sure you'd hear from them about it if you did!
  - aidenn0 3y ago
    I used 2TB on my cable modem once and got called with them trying to upsell me to Unlimited data for $75/mo more.
  - xur17 3y ago
    I regularly use 5 - 10 TB on my fiber each month, and have never been contacted.
- chaxor 3y ago
  You absolutely should not ever get hassled for using a product that you already paid for, that's incredibly stupid. Unfortunately the US just treats anything that isn't a corporation like trash.
  - zamadatix 3y ago
    It's not necessarily the premise of a service with disjoint bandwidth and transfer amounts that's the problem IMO. Separating bandwidth and usage provides great value to most people who would otherwise have to pay for a much more expensive service if the two were always locked together literally. Where we could use better regulation is making the fair-use policies more upfront and useful so you don't have to dig for them or try to guess what they mean.
- civiloid 3y ago
vannucci 3y ago
I'm so jealous of this. I live in a major metro area of NYC and there's still no fiber out to my town. This is so cool.
tgtweak 3y ago
Recently moved up to 3gbps bidirectional and had a hell of a time getting 3gbps even with 10gbps network card directly into the 10gbps port on the ISP-provided modem. In addition to this, there are very few services that will actually feed you data at 3gbps. Steam - ~280MB/s (2.2gbps) Battle.net launcher - ~140MB/s (1.1gbps) 25GB Torrent with 1000+ seeders - ~70MB/s (~560mbps) 2GB iso from github - ~80MB/s (~650mbps) fast.com - 1.4gbps speedtest.net - 2.7gbps (using ISPs endpoint 2ms away) Using a download manager like IDM or jDownloader will help for http downloads, but most hosts will limit your speed even with 16 connections open. I've managed to see 2gbps moving data to/from servers (scp) with softether configured to use 16 connections. The reality is with a single connection (majority of ssl transfers) you'll be limited by the sending side in almost all cases. Overall it seems that while you can get connected and run an iperf to your ISP or multi-connection speedtest to a server hosted by your ISP or peered with your ISP, you'll be pretty much limited to <1gbps speeds regardless of your home network throughput. Knowing this I would have simply went multi-gig (2.5g) for all in-home networking and saved a good chunk of change on networking equipment.
- Phelinofist 3y ago
  280MB/s steam? On a good day I get 25MB/s (Vodafone 400/25) on any German mirror
  - tgtweak 3y ago
    Speeds seem to vary based on the game and based on releases, interestingly. Some larger new-releases (Baldur's gate) were only hitting <100MB/s. I imagine it comes down to how loaded up the local steam POP is and maybe even the specific server hosting that game's files. Aside from that release, almost everything that is a good size (>10GB) has gotten north of 100MB/s pretty regularly.
  - tgtweak 3y ago
    Your max theoretical would be 50MB/s on that connection, so 25MB is not terrible. I think there is some notable overhead on steam (check summing or encryption) as with my 1gbps connection previously I would never get past 100MB/s (800mbps).
tiffanyh 3y ago
WiFi While I love that ISP are continuing to ramp up the speeds beyond 5 Gbps, keep in mind that 99.99% of home user connect on WiFi which taps out at like 1-2 Gbps.
robbiet480 3y ago
I've got AT&T Fiber's 5 gigabit symmetric service (with a Ubiquiti equipment stack that handles 40 gigabit without issue) and while its been very stable and I win all nerd fights with my speed tests, I've been thinking about dropping back down to 1 gigabit because I don't find much value out of 5. If it was cheaper ($180/mo for 5 gigabit) maybe I'd keep it but Astound/WaveG is offering gigabit symmetric for $50/mo with 4 free months so tough to swallow.
- Aqua_Geek 3y ago
  What Ubiquiti equipment are you using? Doesn’t their router top out at 3.5Gbps?
  - robbiet480 3y ago
    UDM Pro USW-Pro-Aggregation USW-Pro-48-PoE US-XG-6PoE Recently bypassed AT&T's residential gateway by getting a XGS-PON on a SFP+ stick, let me shave off 2ms and gain a couple hundred mbps each way, in addition to removing their crappy gateway.
    - zamadatix 3y ago
      Which stick did you go with?
      - robbiet480 3y ago
        This one [0] via a group buy on the 8311 Discord [1]. The instructions for use are at [2]. [0]: https://ecin.ca/custom-xgs-pon-sfp-stick-module-xgspon-ont-w-t-mac-function-mounted-on-sfp-package/ https://ecin.ca/custom-xgs-pon-sfp-stick-module-xgspon-ont-w... [1]: https://discord.gg/8311-886329492438671420 https://discord.gg/8311-886329492438671420 [2]: https://docs.google.com/document/u/0/d/1UIAgtxkImgFRwyaGDGtISD0JXnxWNvuuNDrnRac6wGc/mobilebasic https://docs.google.com/document/u/0/d/1UIAgtxkImgFRwyaGDGtI...
        zamadatix 3y ago
zamadatix 3y ago
For home runs I'd recommend using duplex instead of simplex (the cable out to the provider has to stay to whatever they require of course). For a 100M cable it's $37.47 for the duplex and $24.48 for the simplex, the price difference of $13 for a really long run. The price difference of 2x 25BASE-LR (duplex) vs 2x 25BASE-BX (simplex) transceivers will be more like $60. Even if you want to stick with simplex optics everywhere adding <$10 more to know you have an extra run already made should you ever need it is pretty handy. If your concern is the physical size of the cable just make sure you're getting a non-splittable duplex cable. Some people prefer splittable because then if they need a simplex cable they can just pull it apart but that's what requires the cable to be larger as you get 2 individually shielded cables. The actual fiber core is measured in microns, you can pack 100s in roughly the same sized jacket. Bonus note: If you're ever looking at doing a bunch of runs between two spots you can look into MTP/MPO trunk cables. These put 12 strands in the same protective sheath so it's still a really tiny cable and then you can break out via a pigtail cable at the other end.
- tgtweak 3y ago
  The price of 25GbaseT (rj45) is still very high. 10GbaseT (rj45) is coming down very nicely though and some motherboards, routers and high end access points have 10g ports onboard. I think it's actually cheaper to go 40gbps QSFP and run MPO cables (4 pairs internally) than it is to go 25g fiber. This is pretty much the case in datacenter right now for anything that doesn't need to go over 300M.
  - wmf 3y ago
    I think it's actually cheaper to go 40gbps QSFP and run MPO cables (4 pairs internally) than it is to go 25g fiber. It's not. 25G and 40G transceivers cost the same and MPO cables are more expensive.
- civiloid 3y ago
  True, in terms of price, duplex is cheaper (though compared to overall router cost that is negligible). In my case, I wanted to be able to swap SFPs if input one dies (for any reason), and it doesn't make a lot of sense to buy different modules for different purposes, even if that would save a bit of money. There is another difference - if I want to go for field connectors and cut it myself, the Simplex connector would cost about 4.15$, and the Duplex would be 6.16$, which would further increase the price difference. But thanks for the advice, it is indeed a viable option.
ivan4th 3y ago
> The top shows that single-core performance is the bottleneck, and all CPU is consumed by ksoftirqd. Trying out VPP instead of Linux kernel networking might make some sense: https://fd.io https://fd.io It will likely reach the target performance, although the NIC requirements need to be rechecked. But... well, this software is not for the faint of heart ;)
- tgtweak 3y ago
  You can/should also use hardware offloaded queues wherever possible, or look into kernel-bypass networking.
- civiloid 3y ago
  Well... I did more than I've described in the post (I'm the article's author). It just became too large, so I've decided to stop and publish a second part once I run more tests and fix some problems. So far, I have yet to figure out all the problems and settings with Netronome (or give up on it). Still, it is Netronome-specific as I can easily get line rate over TCP single stream with Mellanox cards, and the CPU doesn't even break a sweat (however, I have relatively simple routing rules). I want to try out programmable NIC, and as far as I understand, Netronome has some support there; I just wanted to get a simple setup running first and then optimise/improve it. I tend to perfect the setup, and if I let myself do more and more and more - I will always need more time to finish it.
satvikpendem 3y ago
These kinds of speeds are available in Switzerland because their laws state that any fiber laid down must be usable by competitors, so that ISPs must compete on service alone. I wish we had that in other countries as well.
- ipqk 3y ago
  But then who pays for laying down the fiber? What company would want to pay that cost when others can use it?
  - satvikpendem 3y ago
    The company pays but then it can also use other companies' fiber, so sometimes it's worth it to do so in order to show that they have coverage in a certain area where there might not be fiber (that they then lay down), and also that they cover everywhere else. It's quite a clever system that uses market greed to incentivize building. It's the same system as railroads, imagine if railroads could not share tracks with each other. https://en.wikipedia.org/wiki/Arrangements_between_railroads#Trackage_rights https://en.wikipedia.org/wiki/Arrangements_between_railroads...
  - sschueller 3y ago
    Grants from the government. Some places also vote to have the city or town connect with Fiber which then puts tax money aside for it. In many places it is the power companies that run fiber and they are mostly partially or fully state owned. The lagest ISP also has the state as the majority share holder.
    - theultdev 3y ago
      The government doesn't pay for anything. It's whether you want to pay through it via tax dollars or directly.
      - sschueller 3y ago
        By government I mean the tax payer. The government made the decision to allocate tax payer money for such things. As it stimulates the economy similar to how laying new train track does.
tombert 3y ago
Man, and I thought I was cool for getting my home configured for 10 gigabits. It wasn't too hard or expensive to get used 10 gigabit cards for my router and server. It was a bit pricier to get a used Thunderbolt 10 gigabit adapter for my Mac, but still not prohibitive. I thought I was really future-proofing by making my house 10 gigabit compatible when I only have 2 gigabit internet, but if I have to worry about 25 gigabit internet everywhere I think I think I give up!
- civiloid 3y ago
  The cool thing about fibre is that it is way more future-proof than good old copper. By the way, I also went for a Thunderbolt enclosure and network card for Mac. It is a bit pricy (the enclosure costs way more than yet another ConnectX).
sschueller 3y ago
If you want to know more about init7s network and how they can provide 25gbits see this presentation they did https://youtu.be/wXmJCzMeIBo https://youtu.be/wXmJCzMeIBo
- civiloid 3y ago
  After reading your article, I've started to think if I want to get myself a splicer :D Before, I felt that field connectors would work just fine and splices were too expensive...
- denysvitali 3y ago
  The most beautiful part of it is that no matter what fiber plan you pick, the price is always 777 CHF/year. This effectively means that the only thing stopping you from getting a 25 Gbps connection is, well, the fiber connection and the hardware
bunabhucan 3y ago
This seems like a bonkers solution to be putting packets through a 150w CPU, especially with the price of electrity in Switzerland.
- zamadatix 3y ago
  It's putting a packet through a 150w CPU like weighing a brick is using using a 500 lb scale. I.e. just because it can do that much and you're using that device doesn't mean it's the relevant number to look at. When you're not pushing things it'll be near idle wattage, when you are pushing things like mad it'll be around <4 core wattage (which will still be a lot less than 150W) in this workload. The important thing is it's a CPU which has a great single thread performance and, as a result, it has a high peak multicore wattage you won't actually care about. Overall I'd bet all of the NICs and transceivers are the majority of the power usage in a given year.
- civiloid 3y ago
  I still need to measure and optimise power consumption correctly, but it drew about 70W from the socket during the tests, according to my simple power meter. CPU never ran at boost frequency, and a lot of other stuff is in a low power state.
neals 3y ago
I'm a proud owner of a 1 Gbit/s fiber connection with a 100 Mbit (wired) network in my house. I want to upgrade it to match, but I don't want to go through the effort because I never run into download times for anything.
- zamadatix 3y ago
  OG Cat 5 (non-e) I take it? That is such a pain to redo, I rewired some of that myself and it was a drag. If my job weren't being the literal network guy I probably would have stuck with just using Wi-Fi too!
- avidiax 3y ago
  I have 10gbit service, 1gb internal network. So like you, but 1 generation forward. I don't usually saturate 1gbps, but I definitely saturate 100mbps with XBOX downloads, any cloud provider, torrents, etc. You could probably pull your old cables fairly easily. Attach two wire pull tapes or strings to the old cable, pull the old cable out with plenty of wire pulling lubricant, and then use one tape/string to pull a Cat 7, leaving a spare for mistakes or additional pulls.
  - denysvitali 3y ago
    Do yourself a favor: buy some fiber cables, two switches and connect both ends like that. This is the only way to "future proof" it (past 1 Gbps) and will avoid you pulling cables for a while - I hope.
    - civiloid 3y ago
      You can make it work with 2.5GbE and maybe even 10 GbE with some luck. But looking at its current state and price I personally would agree that fiber is more future proof.
      - denysvitali 3y ago
        I also think that fiber (per meter) is more cost effective than copper anyways for home use, so for me it's a no-brainer
newhotelowner 3y ago
Crying here with my 200 Mbps fiber. Paying $600 a month even though we have 24-port fiber coming in the premise. Spectrum sucks
- zsazsak 3y ago
  I would like to know more about this. What town? Why so expensive?
mrkeen 3y ago
I have 1Gb/s at home. Somehow it doesn't translate into a reliable streaming service experience. I'm currently subscribed to Disney Plus, and trying to watch tv shows via Chromecast rarely works. I even upgraded to the latest Chromecast to try to fix the problem, but it didn't make it much better. I'd trade for a 20mbit/s connection if the streaming services would send me content instead of "an unknown error occurred"
deleted 3y ago
[deleted]
cameronh90 3y ago
I get about 200Mb/s on my 1Gb/s connection because my office is at the other side of the house to the router so I use mesh WiFi. Given that all my walls are solid brick, that's not too bad, really.
e40 3y ago
I have 10Gbps via Sonic.net and n Unifi UDM SE. I don't have 10Gbps internally, though, but the speed test on the UDM SE got about 3.5Gbps. I get 900+Mbps internally and torrents can do about 80-90MB/s. It is glorious. An interesting note about the torrenting: I was shocked that my Mac Mini (Intel) running 12.$latest absolutely could not keep up with more than 20MB/s. The network stack just broke and the machine would actually stop responding to pings. I moved it to a Linux box and ran Transmission (same software) in a container and it is where the 80-90MB/s numbers came from. Doesn't phase the box it's running on. Seriously disappointed in the Apple networking stack.
bitsandboots 3y ago
As an enthusiast but not an expert nor a person with unlimited time and money, I found that while hitting 1gb/s LAN on ethernet is easy, moving up to 10gb/s has been hard. I have 2 networking cards and a router in between them, all 10gb capable. Yet, it's really hard to get them to do that. I started with reliably getting 2.5gb/s and managed to move up to 5gb/s through some tweaks, but actually getting the full value of the hardware seems hard. The bottleneck could be anywhere as far as I know, and I don't have the hardware to swap out in experiments. I don't think it's a drive bottleneck, but even then its not clear. So, I'm all for faster, but it seems faster comes with some headaches if you don't know what you're doing. It's just surprising that consumers are still pretty much limited to 1gb/s after all these years, and my attempt to do otherwise hasn't fully paid off yet.
- civiloid 3y ago
  I think about these headaches as extra fun :) But about your problem - cards manage to negotiate 5 Gbps, or even negotiation fails? If it's the latter, then it might be cables. Overall, for me, it works to simplify the network as much as possible and first achieve results on a direct connection to the router when it's nearby and then add more and more. Oh, and also measure as much as possible (that's why I'm sad about the state of system metrics on OSX, for example)
thefz 3y ago
I have been on 7Mbps / 1Mbps until 2021. Moved in with GF, she had 100Mbps ADSL which blew my mind. Now we have 2.5Gbps FTTH and I feel blessed. I'm moving to a slightly remote place soon which has 200Mbps FWA, and still it blows my mind. I can't imagine having 25Gbps at home. Insane. Downloads matter up to an extent. What still gives me day 1 shivers is 600Mbps upload after years at 1, and sub-10ms latency.
- civiloid 3y ago
  At some point in my life, I was forced to use 7/0.7 Mbps ADSL after having 50/50 Mbps and that ADSL was very oversubscribed (high packet loss and huge speed drops to <2 Mbps during weekends or evenings). Later on, I had FTTH, but I couldn't control the router (fibre was terminating at the landlord's house, and I had cat5e from the router to the place where I lived, and the landlord kept the router in the worst place possible so it was overheating during the summer). Since then, I have tried to be in control of network hardware, and I now avoid *DSL, even though VDSL is decent nowadays. But the reason for 25 Gbps for me is "just because", as ISP charges the same monthly amount as I pay now (there is a setup fee for 25, though).
- tgtweak 3y ago
  I remember going from 56kbps dial-up (getting about 5-6KB/s) to a T1 and downloading an mp3 went from 20 minutes to 30 seconds. Diminishing returns past that point for sure :)
GuB-42 3y ago
The hardest part, I think, is making use of that speed. 25 GBps is about the sequential speed of a decent NVMe SSD, and much faster than anything SATA. It is also faster than what a CPU can handle for all but the most basic of processing. Working at GBytes/s speeds require serious performance considerations when coding, often involving working with assembly, SIMD, etc... Things like the TCP stack start being a bottleneck at this point. There is also a limit on how fast you can consume content. 25 GBit/s is about 4k@120Hz uncompressed video. And remember that both ends need to handle the bandwidth. Doing something useful with the full bandwidth is a challenge in itself. Running a server for many users is the most probable use case, but you need a beefy machine, maybe several to be able to serve at such speed. You need to find enough users too. I am not saying that getting 25 Gbit/s at home is a bad thing, it is a technical achievement and I am all for it. But I feel like finding an application that can saturate that kind of bandwidth is like a continuation to the challenge. Maybe something to do with gaming or VR.
- helf 3y ago
  [dead]
- civiloid 3y ago
  The main application here is to force myself into solving those problems :) I have plans on what I'll do after. As with a 25 Gbps router comes a 25 Gbps home NAS, then I would want to have decent latency for packet processing and I hope all of that would force me to do something interesting :)
- AndroTux 3y ago
  Honestly, for most applications at home, I struggle to justify even 1 Gbit/s. At my location, I have the option to upgrade to a 1 Gbit/s connection at the click of a button. I currently get 500/500 Mbit/s for ~45€ per month. 1 Gbit/s would cost around 100€ per month. I often think about how I could justify the increase in price for myself, but I always come to the conclusion that I just don't need it. And I am a heavy internet user. I download a lot, and I upload a lot (which is why I'm very grateful for my 500 Mbit/s in upload speed). But even with "just" 500 Mbit/s, downloading a 100 GB game takes about 30 minutes. And that's provided I have an endpoint available that can supply the full 500 Mbit/s, which is surprisingly often not the case. If I happen to need 100 GB of data, I can wait 30 minutes. Disclaimer: I'm talking about uplink here. Internally, between my PCs, my home server and my NAS, I'm currently looking into getting 10 Gbit/s set up. And yes, I'm sure there are some use cases out there that benefit from 1 Gbit/s uplink. But I, as a power internet user, have decided for myself that I just don't need it, which means that the average Joe definitely doesn't need it. Still, I think it's great that it's available here and I'm all for getting that expanded further.

What is the state of the art in leveraging your home network to have a presence (website or otherwise) on the internet? Fiber just became available in our neighborhood recently and I'm about to have fewer people sharing our network, so the idea of being able to host my own files is becoming attractive again. We are about to be due for this pendulum (centralized vs distributed) to swing back again and I like to be prepared.

ricw 3y ago
As long as you have a static IP address, this is trivial and typically very reliable. Throw an ATS battery into the mix and the internet will be super reliant, even when the power goes down. Am doing this at home with a fibre line and it's been super reliable.
- chaxor 3y ago
  Dynamic dns is a thing?
  - xur17 3y ago
    My ISP uses dynamic dns, but it never changes unless my modem is down for several minutes. My router has a script that updates a dns entry if it changes, but realistically this rarely happens.
  - rjsw 3y ago
    The choice for my fibre ISP is between static IPv4 or carrier-grade NAT. Both options get IPv6.
organsnyder 3y ago
Cloudflare tunnels are easy to set up. Otherwise, serving directly can be an option if you have a public static IP from your ISP or a public dynamic ISP and are okay with doing dynamic DNS. IMHO the key is to have a good router/firewall that gives you the flexibility you need—ideally a custom box like OP. My software preference is opnsense, but there are plenty of other good options.
- ozarker 3y ago
  I'm hosting my blog on machines running at home accessible via a Cloudflare tunnel. It works very well in my experience. And it's nice not having ports open to the internet or having to mess with dynamic dns.
candiddevmike 3y ago
You can run a website off a cellular connection, you don't need fiber for that. The pendulum has never left decentralized from a tech standpoint, folks are just too lazy. Home computer ownership is tanking, millennials were probably the last "power user" generation, and there's just no major interest in self hosting with the convenience of cloud. Even among tech professionals like HN.
- quaintdev 3y ago
  r/selfhosted would like to have word with you. That sub is increasing steadily. https://www.reddit.com/r/selfhosted https://www.reddit.com/r/selfhosted
  - candiddevmike 3y ago
    I'm a member, I even built a self hosted home organizer (https://homechart.app https://homechart.app). It's a niche community with a very high cost of entry (time/money/skill).
    - nirav72 3y ago
      Regarding the money aspects of the high cost of entry - you can source cheap hardware from secondary markets. I just helped a friend setup a cheap selfhosting/homelab for his home. He bought couple of HP EliteDesk 800 G4 minis for $99/each with i5-8500T CPUs. Each came with 16gb ram. Both units had NVMe drives. So we just had to add couple of 2.5 SSDs 2tb ($50/each) for media storage. Running Ubuntu server with CasaOS ontop with handful of containers for media consumption (e.g Jellyfin) on one unit. The i5-8500T is new enough and supports Intel quicksync. So no GPU needed for transcoding. The second unit is running cloudflare tunnel daemon for things like his MicroBin(like pastebin), Wireguard VPN and NextCloud for external access. He still has plenty of room to add more applications to both servers.
      - candiddevmike 3y ago
        Where'd you find those, ebay? That sounds like a great deal.
vollmond 3y ago
> Just then, a bandwidth load as heavy as a pregnant elephant sits down on Manfred's head and sends clumps of humongous pixilation flickering across his sensorium: Around the world, five million or so geeks are bouncing on his home site, a digital flash crowd alerted by a posting from the other side of the bar. Manfred winces. "I really came here to talk about the economic exploitation of space travel, but I've just been slashdotted. Mind if I just sit and drink until it wears off?" - Charlie Stross, "Accelerando"
- V1ndaar 3y ago
  Damn, I read that book as a teenager in 2006 or 07! I've been thinking about a reread whenever I remember it. It was the book that really made me appreciate the whole technological singularity idea back then.
  - vollmond 3y ago
    I've been trying to find my paper copy all week so I can reread it. Might just give up and read it on his site: https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando.html https://www.antipope.org/charlie/blog-static/fiction/acceler...
Symbiote 3y ago
I just run a normal website through a normal (major company) connection with their supplied router. I have 10 minutes or so of power outage per year, otherwise it works fine. Maintenance effort is close to zero, security update apply automatically.
h2odragon 3y ago
I'm tinkering with a Raspi to do that sometime soon. Its going to be on an isolated network segment off an openwrt router that'll forward ports and hold the public ip. Technically it's ready to go but I'm enjoying tinkering with it still and no rush to put anything up. its not "state of the art" but I expect it to serve my needs.
bombcar 3y ago
For me the state of the art is realizing that you do NOT need 99.99999% uptime for personal stuff. Once there, do whatever you want. Get a DMZ or even a separate connection if you really want to go wild, but it's not terribly complicated. There are even "in a box" solutions - such as https://mailinabox.email https://mailinabox.email or https://owncloud.com https://owncloud.com
nickspacek 3y ago
I don't want to open my home network to just anybody, so I have a "jumpbox" that is the lowest-end shared VM at Hetzner. It runs nginx, dnsmasq, and Wireguard; my home servers connect to it, I add other peers as I need to, and dnsmasq resolves the hostnames using Zeroguard IPs for the home network. I have 3 sets of DNS entries for the home lab servers: 1. "internal"/home network addresses (e.g. your 192.168.x.x) 2. Wireguard addresses (e.g. 10.0.x.x) 3. public DNS entries that all resolve to the jumpbox The purpose of #3 is to support simple Letsencrypt setup: nginx on jumpbox forwards Letsencrypt requests to the internal servers over the Wireguard connection. Internally, I use a https://github.com/nginx-proxy/nginx-proxy https://github.com/nginx-proxy/nginx-proxy setup, so that any time I want a new service running inside the home lab I just have to: 1. Pick a hostname and add it to public DNS 2. Configure its Docker container to add the environment variables that nginx-proxy looks for 3. Add the hostname to the jumpbox /etc/hosts 4. Add the hostname to internal LAN DNS It's a little much but I like how it works. It's not so bad to get setup.
tgtweak 3y ago
reverse dns to catch your dynamic IP changes, and port forwarding/dmz on your modem/router should be all you need if your ISP doesn't block port 80/443. If your ISP does port block (or you don't want to port forward or expose your IP) you can also set up a cloudflare tunnel (warp?) on a server, and point it to your local webserver's port, and set your cloudflare DNS for your domain so that it points to your warp tunnel. This way requests for the website go to cloudflare then to your machine and on to your local webserver.
fulafel 3y ago
It depends, if you have real IP connectivity on your box you just run a web server and bob's your uncle. If you're stuck behind a nat for v4, try v6. Or port forwarding or VPN to a cloud provider or vps with better IP connectivity.

gorbypark 3y ago

I can upgrade from 1Gbit to 10Gbit for only 5 euros extra per month but I don't really have anything to take advantage of it. I don't think my new MacBook Air M2 even supports 6E, and wiring the apartment would be a bit of a hassle.

bombcar 3y ago
If you only have a few people or clients in the house, the upgrade is likely not worth it, though 10Gb would be nice to flaunt. At that price I'd take it, but that's because I have a 10G network to my NAS, though I'd have to upgrade my router.
civiloid 3y ago
With the size of recent games and patches, 1 Gbit starts to feel a bit too low. :) And about Macbook... In my current setup, I'm using Thunderbolt -> PCIe and one of the Mellanox cards; it gives excellent and stable 16 Gbps with stock firmware and DriverKit-based drivers (that comes with OSX), though it feels that DriverKit driver doesn't use any offloading that it supports (while running iperf between Macbook and router, it consumes roughly 3 cores entirely on Macbook, while the router doesn't even ramp up the single core to its boost frequency). But you are right; rewiring is a hassle, though Cat5e usually works with 10Gbit over reasonable lengths (10-30 meters, maybe a bit more if it is perfect quality cable). Also, even if I decided not to go for fibre, I'd still switch to at least 10Gbps because I'm not alone in that network :)

londons_explore 3y ago

How many hours can you saturate that 25 Gbit connection up and down before support calls you and tells you to knock it off?

tgtweak 3y ago
I ran about 8TB on my fiber in one month, and no ISP notices to speak of. That's only ~25mbps sustained. You could easily push 500+TB on a 25gbps connection if you were doing something naughty with it, and I'm sure you'd hear from them about it if you did!
- aidenn0 3y ago
  I used 2TB on my cable modem once and got called with them trying to upsell me to Unlimited data for $75/mo more.
- xur17 3y ago
  I regularly use 5 - 10 TB on my fiber each month, and have never been contacted.
chaxor 3y ago
You absolutely should not ever get hassled for using a product that you already paid for, that's incredibly stupid. Unfortunately the US just treats anything that isn't a corporation like trash.
- zamadatix 3y ago
  It's not necessarily the premise of a service with disjoint bandwidth and transfer amounts that's the problem IMO. Separating bandwidth and usage provides great value to most people who would otherwise have to pay for a much more expensive service if the two were always locked together literally. Where we could use better regulation is making the fair-use policies more upfront and useful so you don't have to dig for them or try to guess what they mean.
civiloid 3y ago

vannucci 3y ago

I'm so jealous of this. I live in a major metro area of NYC and there's still no fiber out to my town. This is so cool.

tgtweak 3y ago

Recently moved up to 3gbps bidirectional and had a hell of a time getting 3gbps even with 10gbps network card directly into the 10gbps port on the ISP-provided modem. In addition to this, there are very few services that will actually feed you data at 3gbps. Steam - ~280MB/s (2.2gbps) Battle.net launcher - ~140MB/s (1.1gbps) 25GB Torrent with 1000+ seeders - ~70MB/s (~560mbps) 2GB iso from github - ~80MB/s (~650mbps) fast.com - 1.4gbps speedtest.net - 2.7gbps (using ISPs endpoint 2ms away) Using a download manager like IDM or jDownloader will help for http downloads, but most hosts will limit your speed even with 16 connections open. I've managed to see 2gbps moving data to/from servers (scp) with softether configured to use 16 connections. The reality is with a single connection (majority of ssl transfers) you'll be limited by the sending side in almost all cases. Overall it seems that while you can get connected and run an iperf to your ISP or multi-connection speedtest to a server hosted by your ISP or peered with your ISP, you'll be pretty much limited to <1gbps speeds regardless of your home network throughput. Knowing this I would have simply went multi-gig (2.5g) for all in-home networking and saved a good chunk of change on networking equipment.

Phelinofist 3y ago
280MB/s steam? On a good day I get 25MB/s (Vodafone 400/25) on any German mirror
- tgtweak 3y ago
  Speeds seem to vary based on the game and based on releases, interestingly. Some larger new-releases (Baldur's gate) were only hitting <100MB/s. I imagine it comes down to how loaded up the local steam POP is and maybe even the specific server hosting that game's files. Aside from that release, almost everything that is a good size (>10GB) has gotten north of 100MB/s pretty regularly.
- tgtweak 3y ago
  Your max theoretical would be 50MB/s on that connection, so 25MB is not terrible. I think there is some notable overhead on steam (check summing or encryption) as with my 1gbps connection previously I would never get past 100MB/s (800mbps).

tiffanyh 3y ago

WiFi While I love that ISP are continuing to ramp up the speeds beyond 5 Gbps, keep in mind that 99.99% of home user connect on WiFi which taps out at like 1-2 Gbps.

robbiet480 3y ago

I've got AT&T Fiber's 5 gigabit symmetric service (with a Ubiquiti equipment stack that handles 40 gigabit without issue) and while its been very stable and I win all nerd fights with my speed tests, I've been thinking about dropping back down to 1 gigabit because I don't find much value out of 5. If it was cheaper ($180/mo for 5 gigabit) maybe I'd keep it but Astound/WaveG is offering gigabit symmetric for $50/mo with 4 free months so tough to swallow.

Aqua_Geek 3y ago
What Ubiquiti equipment are you using? Doesn’t their router top out at 3.5Gbps?
- robbiet480 3y ago
  UDM Pro USW-Pro-Aggregation USW-Pro-48-PoE US-XG-6PoE Recently bypassed AT&T's residential gateway by getting a XGS-PON on a SFP+ stick, let me shave off 2ms and gain a couple hundred mbps each way, in addition to removing their crappy gateway.
  - zamadatix 3y ago
    Which stick did you go with?
    - robbiet480 3y ago
      This one [0] via a group buy on the 8311 Discord [1]. The instructions for use are at [2]. [0]: https://ecin.ca/custom-xgs-pon-sfp-stick-module-xgspon-ont-w-t-mac-function-mounted-on-sfp-package/ https://ecin.ca/custom-xgs-pon-sfp-stick-module-xgspon-ont-w... [1]: https://discord.gg/8311-886329492438671420 https://discord.gg/8311-886329492438671420 [2]: https://docs.google.com/document/u/0/d/1UIAgtxkImgFRwyaGDGtISD0JXnxWNvuuNDrnRac6wGc/mobilebasic https://docs.google.com/document/u/0/d/1UIAgtxkImgFRwyaGDGtI...
      - zamadatix 3y ago

zamadatix 3y ago

For home runs I'd recommend using duplex instead of simplex (the cable out to the provider has to stay to whatever they require of course). For a 100M cable it's $37.47 for the duplex and $24.48 for the simplex, the price difference of $13 for a really long run. The price difference of 2x 25BASE-LR (duplex) vs 2x 25BASE-BX (simplex) transceivers will be more like $60. Even if you want to stick with simplex optics everywhere adding <$10 more to know you have an extra run already made should you ever need it is pretty handy. If your concern is the physical size of the cable just make sure you're getting a non-splittable duplex cable. Some people prefer splittable because then if they need a simplex cable they can just pull it apart but that's what requires the cable to be larger as you get 2 individually shielded cables. The actual fiber core is measured in microns, you can pack 100s in roughly the same sized jacket. Bonus note: If you're ever looking at doing a bunch of runs between two spots you can look into MTP/MPO trunk cables. These put 12 strands in the same protective sheath so it's still a really tiny cable and then you can break out via a pigtail cable at the other end.

tgtweak 3y ago
The price of 25GbaseT (rj45) is still very high. 10GbaseT (rj45) is coming down very nicely though and some motherboards, routers and high end access points have 10g ports onboard. I think it's actually cheaper to go 40gbps QSFP and run MPO cables (4 pairs internally) than it is to go 25g fiber. This is pretty much the case in datacenter right now for anything that doesn't need to go over 300M.
- wmf 3y ago
  I think it's actually cheaper to go 40gbps QSFP and run MPO cables (4 pairs internally) than it is to go 25g fiber. It's not. 25G and 40G transceivers cost the same and MPO cables are more expensive.
civiloid 3y ago
True, in terms of price, duplex is cheaper (though compared to overall router cost that is negligible). In my case, I wanted to be able to swap SFPs if input one dies (for any reason), and it doesn't make a lot of sense to buy different modules for different purposes, even if that would save a bit of money. There is another difference - if I want to go for field connectors and cut it myself, the Simplex connector would cost about 4.15$, and the Duplex would be 6.16$, which would further increase the price difference. But thanks for the advice, it is indeed a viable option.

ivan4th 3y ago

> The top shows that single-core performance is the bottleneck, and all CPU is consumed by ksoftirqd. Trying out VPP instead of Linux kernel networking might make some sense: https://fd.io https://fd.io It will likely reach the target performance, although the NIC requirements need to be rechecked. But... well, this software is not for the faint of heart ;)

tgtweak 3y ago
You can/should also use hardware offloaded queues wherever possible, or look into kernel-bypass networking.
civiloid 3y ago
Well... I did more than I've described in the post (I'm the article's author). It just became too large, so I've decided to stop and publish a second part once I run more tests and fix some problems. So far, I have yet to figure out all the problems and settings with Netronome (or give up on it). Still, it is Netronome-specific as I can easily get line rate over TCP single stream with Mellanox cards, and the CPU doesn't even break a sweat (however, I have relatively simple routing rules). I want to try out programmable NIC, and as far as I understand, Netronome has some support there; I just wanted to get a simple setup running first and then optimise/improve it. I tend to perfect the setup, and if I let myself do more and more and more - I will always need more time to finish it.

satvikpendem 3y ago

These kinds of speeds are available in Switzerland because their laws state that any fiber laid down must be usable by competitors, so that ISPs must compete on service alone. I wish we had that in other countries as well.

ipqk 3y ago
But then who pays for laying down the fiber? What company would want to pay that cost when others can use it?
- satvikpendem 3y ago
  The company pays but then it can also use other companies' fiber, so sometimes it's worth it to do so in order to show that they have coverage in a certain area where there might not be fiber (that they then lay down), and also that they cover everywhere else. It's quite a clever system that uses market greed to incentivize building. It's the same system as railroads, imagine if railroads could not share tracks with each other. https://en.wikipedia.org/wiki/Arrangements_between_railroads#Trackage_rights https://en.wikipedia.org/wiki/Arrangements_between_railroads...
- sschueller 3y ago
  Grants from the government. Some places also vote to have the city or town connect with Fiber which then puts tax money aside for it. In many places it is the power companies that run fiber and they are mostly partially or fully state owned. The lagest ISP also has the state as the majority share holder.
  - theultdev 3y ago
    The government doesn't pay for anything. It's whether you want to pay through it via tax dollars or directly.
    - sschueller 3y ago
      By government I mean the tax payer. The government made the decision to allocate tax payer money for such things. As it stimulates the economy similar to how laying new train track does.

tombert 3y ago

Man, and I thought I was cool for getting my home configured for 10 gigabits. It wasn't too hard or expensive to get used 10 gigabit cards for my router and server. It was a bit pricier to get a used Thunderbolt 10 gigabit adapter for my Mac, but still not prohibitive. I thought I was really future-proofing by making my house 10 gigabit compatible when I only have 2 gigabit internet, but if I have to worry about 25 gigabit internet everywhere I think I think I give up!

civiloid 3y ago
The cool thing about fibre is that it is way more future-proof than good old copper. By the way, I also went for a Thunderbolt enclosure and network card for Mac. It is a bit pricy (the enclosure costs way more than yet another ConnectX).

sschueller 3y ago

If you want to know more about init7s network and how they can provide 25gbits see this presentation they did https://youtu.be/wXmJCzMeIBo https://youtu.be/wXmJCzMeIBo

civiloid 3y ago
After reading your article, I've started to think if I want to get myself a splicer :D Before, I felt that field connectors would work just fine and splices were too expensive...
denysvitali 3y ago
The most beautiful part of it is that no matter what fiber plan you pick, the price is always 777 CHF/year. This effectively means that the only thing stopping you from getting a 25 Gbps connection is, well, the fiber connection and the hardware

bunabhucan 3y ago

This seems like a bonkers solution to be putting packets through a 150w CPU, especially with the price of electrity in Switzerland.

zamadatix 3y ago
It's putting a packet through a 150w CPU like weighing a brick is using using a 500 lb scale. I.e. just because it can do that much and you're using that device doesn't mean it's the relevant number to look at. When you're not pushing things it'll be near idle wattage, when you are pushing things like mad it'll be around <4 core wattage (which will still be a lot less than 150W) in this workload. The important thing is it's a CPU which has a great single thread performance and, as a result, it has a high peak multicore wattage you won't actually care about. Overall I'd bet all of the NICs and transceivers are the majority of the power usage in a given year.
civiloid 3y ago
I still need to measure and optimise power consumption correctly, but it drew about 70W from the socket during the tests, according to my simple power meter. CPU never ran at boost frequency, and a lot of other stuff is in a low power state.

neals 3y ago

I'm a proud owner of a 1 Gbit/s fiber connection with a 100 Mbit (wired) network in my house. I want to upgrade it to match, but I don't want to go through the effort because I never run into download times for anything.

zamadatix 3y ago
OG Cat 5 (non-e) I take it? That is such a pain to redo, I rewired some of that myself and it was a drag. If my job weren't being the literal network guy I probably would have stuck with just using Wi-Fi too!
avidiax 3y ago
I have 10gbit service, 1gb internal network. So like you, but 1 generation forward. I don't usually saturate 1gbps, but I definitely saturate 100mbps with XBOX downloads, any cloud provider, torrents, etc. You could probably pull your old cables fairly easily. Attach two wire pull tapes or strings to the old cable, pull the old cable out with plenty of wire pulling lubricant, and then use one tape/string to pull a Cat 7, leaving a spare for mistakes or additional pulls.
- denysvitali 3y ago
  Do yourself a favor: buy some fiber cables, two switches and connect both ends like that. This is the only way to "future proof" it (past 1 Gbps) and will avoid you pulling cables for a while - I hope.
  - civiloid 3y ago
    You can make it work with 2.5GbE and maybe even 10 GbE with some luck. But looking at its current state and price I personally would agree that fiber is more future proof.
    - denysvitali 3y ago
      I also think that fiber (per meter) is more cost effective than copper anyways for home use, so for me it's a no-brainer

newhotelowner 3y ago

Crying here with my 200 Mbps fiber. Paying $600 a month even though we have 24-port fiber coming in the premise. Spectrum sucks

zsazsak 3y ago
I would like to know more about this. What town? Why so expensive?

mrkeen 3y ago

I have 1Gb/s at home. Somehow it doesn't translate into a reliable streaming service experience. I'm currently subscribed to Disney Plus, and trying to watch tv shows via Chromecast rarely works. I even upgraded to the latest Chromecast to try to fix the problem, but it didn't make it much better. I'd trade for a 20mbit/s connection if the streaming services would send me content instead of "an unknown error occurred"

deleted 3y ago

[deleted]

cameronh90 3y ago

I get about 200Mb/s on my 1Gb/s connection because my office is at the other side of the house to the router so I use mesh WiFi. Given that all my walls are solid brick, that's not too bad, really.

e40 3y ago

I have 10Gbps via Sonic.net and n Unifi UDM SE. I don't have 10Gbps internally, though, but the speed test on the UDM SE got about 3.5Gbps. I get 900+Mbps internally and torrents can do about 80-90MB/s. It is glorious. An interesting note about the torrenting: I was shocked that my Mac Mini (Intel) running 12.$latest absolutely could not keep up with more than 20MB/s. The network stack just broke and the machine would actually stop responding to pings. I moved it to a Linux box and ran Transmission (same software) in a container and it is where the 80-90MB/s numbers came from. Doesn't phase the box it's running on. Seriously disappointed in the Apple networking stack.

bitsandboots 3y ago

As an enthusiast but not an expert nor a person with unlimited time and money, I found that while hitting 1gb/s LAN on ethernet is easy, moving up to 10gb/s has been hard. I have 2 networking cards and a router in between them, all 10gb capable. Yet, it's really hard to get them to do that. I started with reliably getting 2.5gb/s and managed to move up to 5gb/s through some tweaks, but actually getting the full value of the hardware seems hard. The bottleneck could be anywhere as far as I know, and I don't have the hardware to swap out in experiments. I don't think it's a drive bottleneck, but even then its not clear. So, I'm all for faster, but it seems faster comes with some headaches if you don't know what you're doing. It's just surprising that consumers are still pretty much limited to 1gb/s after all these years, and my attempt to do otherwise hasn't fully paid off yet.

civiloid 3y ago
I think about these headaches as extra fun :) But about your problem - cards manage to negotiate 5 Gbps, or even negotiation fails? If it's the latter, then it might be cables. Overall, for me, it works to simplify the network as much as possible and first achieve results on a direct connection to the router when it's nearby and then add more and more. Oh, and also measure as much as possible (that's why I'm sad about the state of system metrics on OSX, for example)

thefz 3y ago

I have been on 7Mbps / 1Mbps until 2021. Moved in with GF, she had 100Mbps ADSL which blew my mind. Now we have 2.5Gbps FTTH and I feel blessed. I'm moving to a slightly remote place soon which has 200Mbps FWA, and still it blows my mind. I can't imagine having 25Gbps at home. Insane. Downloads matter up to an extent. What still gives me day 1 shivers is 600Mbps upload after years at 1, and sub-10ms latency.

civiloid 3y ago
At some point in my life, I was forced to use 7/0.7 Mbps ADSL after having 50/50 Mbps and that ADSL was very oversubscribed (high packet loss and huge speed drops to <2 Mbps during weekends or evenings). Later on, I had FTTH, but I couldn't control the router (fibre was terminating at the landlord's house, and I had cat5e from the router to the place where I lived, and the landlord kept the router in the worst place possible so it was overheating during the summer). Since then, I have tried to be in control of network hardware, and I now avoid *DSL, even though VDSL is decent nowadays. But the reason for 25 Gbps for me is "just because", as ISP charges the same monthly amount as I pay now (there is a setup fee for 25, though).
tgtweak 3y ago
I remember going from 56kbps dial-up (getting about 5-6KB/s) to a T1 and downloading an mp3 went from 20 minutes to 30 seconds. Diminishing returns past that point for sure :)

GuB-42 3y ago

The hardest part, I think, is making use of that speed. 25 GBps is about the sequential speed of a decent NVMe SSD, and much faster than anything SATA. It is also faster than what a CPU can handle for all but the most basic of processing. Working at GBytes/s speeds require serious performance considerations when coding, often involving working with assembly, SIMD, etc... Things like the TCP stack start being a bottleneck at this point. There is also a limit on how fast you can consume content. 25 GBit/s is about 4k@120Hz uncompressed video. And remember that both ends need to handle the bandwidth. Doing something useful with the full bandwidth is a challenge in itself. Running a server for many users is the most probable use case, but you need a beefy machine, maybe several to be able to serve at such speed. You need to find enough users too. I am not saying that getting 25 Gbit/s at home is a bad thing, it is a technical achievement and I am all for it. But I feel like finding an application that can saturate that kind of bandwidth is like a continuation to the challenge. Maybe something to do with gaming or VR.

helf 3y ago
[dead]
civiloid 3y ago
The main application here is to force myself into solving those problems :) I have plans on what I'll do after. As with a 25 Gbps router comes a 25 Gbps home NAS, then I would want to have decent latency for packet processing and I hope all of that would force me to do something interesting :)
AndroTux 3y ago
Honestly, for most applications at home, I struggle to justify even 1 Gbit/s. At my location, I have the option to upgrade to a 1 Gbit/s connection at the click of a button. I currently get 500/500 Mbit/s for ~45€ per month. 1 Gbit/s would cost around 100€ per month. I often think about how I could justify the increase in price for myself, but I always come to the conclusion that I just don't need it. And I am a heavy internet user. I download a lot, and I upload a lot (which is why I'm very grateful for my 500 Mbit/s in upload speed). But even with "just" 500 Mbit/s, downloading a 100 GB game takes about 30 minutes. And that's provided I have an endpoint available that can supply the full 500 Mbit/s, which is surprisingly often not the case. If I happen to need 100 GB of data, I can wait 30 minutes. Disclaimer: I'm talking about uplink here. Internally, between my PCs, my home server and my NAS, I'm currently looking into getting 10 Gbit/s set up. And yes, I'm sure there are some use cases out there that benefit from 1 Gbit/s uplink. But I, as a power internet user, have decided for myself that I just don't need it, which means that the average Joe definitely doesn't need it. Still, I think it's great that it's available here and I'm all for getting that expanded further.