4 ms·

The major decision to make is, what kind of connector do you want? Ethernet or SFP? Ethernet is more costly, SFP is the more approachable-but-unusual way. Le

The major decision to make is, what kind of connector do you want? Ethernet or SFP? Ethernet is more costly, SFP is the more approachable-but-unusual way. Lengths of the runs and how many clients changes the decision making a bit, so keep that in mind. I didn't mind cost so much but I wanted to stick with typical Ethernet. Mainly a comfort thing, I don't know the limits of SFP well. I went with this switch: Mikrotik CRS312-4C+8XG ... and these cards: ASUS XG-C100C I upgraded my home lab to 10GbE around a year ago mainly to speed up my SSD-based backups. All told I think it was about $800 to upgrade my storage cluster and a couple clients.

OJFord 5y ago
Ethernet is more costly?! Over what distance? And surely ignoring the 'endpoint hardware', switches or whatever, (or equivalently considering only stuff high enough end that it has SFP 'for free' anyway) whatever the distance?
- Sebb767 5y ago
  You can get a lot of cheap, used SFP+ stuff on eBay and the likes. For short range (up to ~3m) you can go with DACs, which are 5-10€ a piece; for longer ranges, you can go to fiber, where you'll pay 20-30€ per cable. Ethernet cables are a bit cheaper, but you need high quality ones (cat6 shielded and up) and the receivers are a lot more expensive, plus they get hotter and use more power. Unless you get a really great deal on ethernet hardware or need backwards compatibility to 2.5/1 gbit, SFP+ will be a lot cheaper.
- zamadatix 5y ago
  In the case of 10G sure, over any distance. Servers use cheap and simple passive copper DACs in racks and switches use optical transceivers for uplinks (usually of a higher speed RJ45 doesn't support anyways). 10G RJ45 is more complicated to implement for both the NIC and switch and requires more power, meanwhile it's not mass produced because of the aforementioned DACs and optical uplinks and lack of consumer penetration.
- icelancer 5y ago
  Ethernet is quite a bit more costly for 10G. In our deployment we use SFP+ with 3m DACs which are downright cheap, plus long fiber runs which are more feasible and less expensive for connecting sub-sites. We support 10G-BaseT as well for devices that are 10G-PoE-BaseT but otherwise standard here is SFP+. Gets even cheaper when you consider power consumption, heat dissipation, and multi-port cards (2xSFP+ NICs are extremely affordable and nice when you have segregated networks or VM hosts). EDIT: 10G-BaseT also has ~2 orders of magnitude higher latency compared to SFP+ Fiber, and ~1 order of magnitude higher latency than SFP+ DAC. The numbers are small across the board, but it's relevant. 5-10W draw for 10GbE compared to 1-2W draw for SFP+ Fiber, for power consumption numbers.
ComputerGuru 5y ago
Note: SFP/SFP+ is just the transceiver format. LR/SR is fiber and T is Ethernet.
hiptobecubic 5y ago
I recently discovered /r/homelab and I found it extremely confusing. People are throwing thousands of dollars and thousands of watts at servers that could power a medium-sized business and then putting them in their basements and running local DNS servers, a NAS and maybe a mail server handling personal mail. Most of the compute power is spent reinstalling various virtualization technologies over and over again and repeatedly backing up a bunch of torrented movies and music without compression or dedup to flex their 30T NAS. What am I missing?
- bravetraveler 5y ago
  Couldn't say for sure, I've noticed similar. I feel like I have to be overlooking something... even though I may appear to be in the same wasteful-spending boat. Redundancy is an easy way to spend a ton of cash poorly, but if done well I recommend it for anyone! The virtualization aspect for example, that's just a bullet-point for how I use KVM and some of the fancier gear to prove out certain implementations... then automate using them. Lately, SR-IOV. For those like me, it's often fairly practical but the trivial stuff may be what gets shared

OJFord 5y ago

Ethernet is more costly?! Over what distance? And surely ignoring the 'endpoint hardware', switches or whatever, (or equivalently considering only stuff high enough end that it has SFP 'for free' anyway) whatever the distance?

Sebb767 5y ago
You can get a lot of cheap, used SFP+ stuff on eBay and the likes. For short range (up to ~3m) you can go with DACs, which are 5-10€ a piece; for longer ranges, you can go to fiber, where you'll pay 20-30€ per cable. Ethernet cables are a bit cheaper, but you need high quality ones (cat6 shielded and up) and the receivers are a lot more expensive, plus they get hotter and use more power. Unless you get a really great deal on ethernet hardware or need backwards compatibility to 2.5/1 gbit, SFP+ will be a lot cheaper.
zamadatix 5y ago
In the case of 10G sure, over any distance. Servers use cheap and simple passive copper DACs in racks and switches use optical transceivers for uplinks (usually of a higher speed RJ45 doesn't support anyways). 10G RJ45 is more complicated to implement for both the NIC and switch and requires more power, meanwhile it's not mass produced because of the aforementioned DACs and optical uplinks and lack of consumer penetration.
icelancer 5y ago
Ethernet is quite a bit more costly for 10G. In our deployment we use SFP+ with 3m DACs which are downright cheap, plus long fiber runs which are more feasible and less expensive for connecting sub-sites. We support 10G-BaseT as well for devices that are 10G-PoE-BaseT but otherwise standard here is SFP+. Gets even cheaper when you consider power consumption, heat dissipation, and multi-port cards (2xSFP+ NICs are extremely affordable and nice when you have segregated networks or VM hosts). EDIT: 10G-BaseT also has ~2 orders of magnitude higher latency compared to SFP+ Fiber, and ~1 order of magnitude higher latency than SFP+ DAC. The numbers are small across the board, but it's relevant. 5-10W draw for 10GbE compared to 1-2W draw for SFP+ Fiber, for power consumption numbers.

ComputerGuru 5y ago

Note: SFP/SFP+ is just the transceiver format. LR/SR is fiber and T is Ethernet.

hiptobecubic 5y ago

I recently discovered /r/homelab and I found it extremely confusing. People are throwing thousands of dollars and thousands of watts at servers that could power a medium-sized business and then putting them in their basements and running local DNS servers, a NAS and maybe a mail server handling personal mail. Most of the compute power is spent reinstalling various virtualization technologies over and over again and repeatedly backing up a bunch of torrented movies and music without compression or dedup to flex their 30T NAS. What am I missing?

bravetraveler 5y ago
Couldn't say for sure, I've noticed similar. I feel like I have to be overlooking something... even though I may appear to be in the same wasteful-spending boat. Redundancy is an easy way to spend a ton of cash poorly, but if done well I recommend it for anyone! The virtualization aspect for example, that's just a bullet-point for how I use KVM and some of the fancier gear to prove out certain implementations... then automate using them. Lately, SR-IOV. For those like me, it's often fairly practical but the trivial stuff may be what gets shared