4 ms·
I'm right in the middle of a huge home network/storage upgrade myself, although I'm making a slightly bigger jump. I've got a fairly extensive home lab network
by jlgaddis 6y ago
I'm right in the middle of a huge home network/storage upgrade myself, although I'm making a slightly bigger jump.
I've got a fairly extensive home lab network but I've been "stuck" at 1 GbE for several years now, mostly because I couldn't justify the cost of upgrading to 10 GbE. See, I'd need a switch with a minimum of five 10 GbE ports -- just for the FreeNAS box, my workstation, and three of the VMware nodes; if I were gonna make the jump to 10 GbE, I'd want to upgrade those machines, at the least.
A couple years ago I bought a pair of cheap, old Mellanox 10 GbE cards and a short direct-attach cable (DAC) to directly connect the FreeNAS machine and the "giant" VMware box (which has roughly the same "horsepower" of three of the others combined!) that I ran most things on. For the most part, everything else has a single 1 GbE link, except for my workstation (2 x 1 GbE) and the other VMware servers (4 x 1 GbE).
Anyways, I recently started thinking about upgrading and/or replacing some things. While doing my research, I came across recommendations for some specific (enterprise) network gear that -- having been mostly a Cisco / Juniper guy for the last 15 years or so -- I wouldn't have normally considered (and, no, MikroTik is NOT an option, for several reasons). I also learned a few "interesting" pieces of information, like how to magically turn some specific (relatively cheap) 10 GbE NICs into (not so cheap!) 40 GbE NICs just by flashing different firmware on them.
I also ended up getting a set of server rails and a small box full of dual-port 10 GbE NICs "free" when I went to buy the set of rails from a guy off Craigslist. They were Broadcom NICs but they'll work just fine in the other VMware servers!
So, by the time I'm done with everything (I'm also in the process of building a new room in the basement just for the lab), my FreeNAS box and workstation will both be connected at 40 GbE and the half-dozen or so other servers will all be connected at 10 GbE (although, to be honest, I will very likely end up connecting the second 10 GbE ports on them all, too; I'll have enough 10 GbE ports left on the switch.).
Before any asks, I'm not sure how much I've spent on the upgrade (and I'm halfway afraid to total it all up, honestly!). I've almost certainly spent more than I had originally planned when first starting to think about doing all of this but: 1) the additional cost to go to 40 GbE on the storage box and my workstation was really that much more so it was easy to justify, 2) once I'm all done with everything, I'm gonna get rid of the "giant" VMware box and, quite likely, a few of the others -- and maybe even some of the other things I've forgotten I have and/or don't use anymore; that should make the total overall cost fairly neglible, and 3) I've got 40 gigabit at my house!
Of course, at this point, the storage server won't be able to come close to saturating the 40 GbE link... which means I'll need to add more, faster storage... rinse and repeat, it never ends! Just say no to home labs, kids!
- c0nsumer 6y agoWhy isn't Mikrotik an option for you? (I don't know much about this space and feature-wise they seem pretty useful for me.)