12 ms·
I built a $5k Raspberry Pi server (yes, it's ridiculous)
- DeathArrow 5y agoA better fitted title would be: "I built a $5,000 Raspberry Pi server so you don't have to do it".
- ishikawa 5y agoAccess to the page is denied
- ishikawa 5y agoAll fine now. That is a nice job!
- anotheraccount9 5y agoIt's probably hosted on his pie.
- geerlingguy 5y agoHeh, not yet. I'm planning on moving my site to my homelab in the next month, at least for a short-term experiment.
- PragmaticPulp 5y agoThe headline buries the lede: It’s a NAS build that uses expensive SSDs. I’m looking forward to the day that I can retire my spinning-disk NAS, but at $700 per 8TB SSD I think it’s going to be a while before they’re cheap enough that I want to take that leap. Like this article concludes: Using a Raspberry Pi for an SSD NAS is a huge waste. There are plenty of low-power Mini-ITX or smaller x86 boards with substantially more compute power than the Pi.
- hinkley 5y agoIt's starting to look like we'll get spinning rust that speaks NVME, which will let people transition to SSD more gracefully.
- dragontamer 5y agoHard drives can easily saturate a 1Gbps link. And RAIDed hard drives can saturate 5Gbps or even 10Gbps with enough parallelism. I'm not sure if any NAS is cost-effective when you use SSDs, aside from maybe a cost-effective SSD cache (Maybe a $100 to $250 1TB L2ARC cache, depending on the speed of the SSD you get).
- geerlingguy 5y agoIndeed, the board in the video (Radxa Taco, hopefully for sale next month) would actually make a decent 1 Gbps NAS (with a 2nd 2.5G port if you want to also use it as a router or have higher network storage speed for short bursts) for HDDs. Most of the average NAS HDDs top out at speeds that are within the Pi's performance realm, though software RAID and ZFS both bottleneck a bit, so you can't get a full 2.5 Gbps (2.3 Gbps real-world) through the Pi to the disks.
- Bluecobra 5y agoInteresting, I have an old HP MicroServer laying around gathering dust. It only has a dual core AMD Turion (like Intel Atom) and 4Gb of memory. It should be easy to retrofit this with a Pi, in fact there's an empty 5.25" drive bay on this case.
- hdjjhhvvhga 5y ago> I’m looking forward to the day that I can retire my spinning-disk NAS, but at $700 per 8TB SSD I think it’s going to be a while before they’re cheap enough that I want to take that leap. It's not just the price - SSDs are not a good choice for long-term storage, especially if they're not powered up continuously.
- 5y ago
- lp0_on_fire 5y agoYou mean you built a $35 raspberry pi server with $4900 worth of SSDs.
- timbit42 5y agoSo... a $4935 Raspberry Pi server.
- rym_ 5y agoI get so frustrated with the idiotic thumbnails on youtube video these days. I am sure it's all optimized for the maximum number of clicks/views but if I see a still that looks the way it does at the bottom of this page I generally just don't watch it on principle.
- jonathankoren 5y agoIt’s not just the thumbnail faces. Streamers constantly overreact to even the most mundane things. My son and a friend of his were watching some pair of Minecraft streamers, and I couldn’t get follow what was happening. Both men were constantly yelling “WHAT? WHAT? OH MY GOD! AHHHHHHHHH!” As far as I could tell, they were both yelling independent things, and just walking around sheep.
- amelius 5y agoAs someone who constantly underreacts to everything, even when I receive a nice gift, I must confess that I am somewhat jealous of the streamer generation.
- deleted 5y ago[deleted]
- MichaelBurge 5y agoI think it's because kids watch a lot of Youtube, and kids are drawn to people making exaggerated faces.
- wmanley 5y agoI think it's because the thumbnails are small, so if the faces aren't exaggerated you won't perceive any particular emotion at all. It's a bit like the difference between theatre and film acting.
- Mordisquitos 5y agoBut that starts from the premise that you even care about the "emotion" expressed in the video. While I'm sure that there are many genres of videos where the emotions expressed by the authors are relevant to the topic at hand, there are many topics in which an expression of emotion, let alone an exaggerated one, is completely out of place.
- deleted 5y ago[deleted]
- nzach 5y agoI was the only one expecting to see a cluster with 100+ units?
- theandrewbailey 5y agoI expected a beefy GPU somewhere.
- geerlingguy 5y agoStill working on that... https://pipci.jeffgeerling.com/#gpus-graphics-cards https://pipci.jeffgeerling.com/#gpus-graphics-cards
- Aeolun 5y agoNope, that was what caused me to click. If someone told me they stuck 4 $1000 disks in a PI I wouldn’t have bothered :/
- rasz 5y agonot even with a stupid face on a thumbnail? cmon its so clickable!!1
- deleted 5y ago[deleted]
- JKCalhoun 5y agoSeeing what a Raspberry Pi can do still has me clicking through.
- Nextgrid 5y agoI mean, it could always do that. You can connect a ton of drives over USB (I think the device limit is 127). The question when it comes to RPi-powered NASes is whether it can do that well and the answer is as everyone expected: not really; the CPU is the bottleneck regardless of the underlying interface the drives connect over.
- ulzeraj 5y agoI have a question about this. I don't use unencrypted storage and RPis have no hardware accelerated extensions. But modern SSDs do, and IMO it is good enough for some stuff even considering the issues. They call it Self Encrypted Disks but I don't know if this feature requires some BIOS specific instructions or special devices like TMPs or if there are special parameters one could pass to GEOM or LUKS to use those encryption capabilities. Anyone could give me a brief explanation on SED and if they could be used on RPi?
- deleted 5y ago[deleted]
- tw04 5y agoSED requires a key server to function or a utility to manually enter a password. sedutil appears to be the go-to on linux assuming it's not your boot drive: https://wiki.archlinux.org/title/Self-encrypting_drives#Encrypting_a_non-root_drive https://wiki.archlinux.org/title/Self-encrypting_drives#Encr... In the enterprise we use external keyservers like the ones Thales sells.
- ZiiS 5y agoThe common standard for SED is TCG OPAL. This has a pre-boot image allowing you to enter your password. Once the drive is decrypted it is completly transparent with your unmodified bootloader and kernel loaded with normal commands. I don't think the RasPi can trigger this unlocking nor could be modified to.
- ZiiS 5y ago* on boot. You can trigger the decryption of additional drives after boot.
- theandrewbailey 5y agoI noticed that OPAL features are becoming less used in encryption software, because the keys to some drives can be dumped through their JTAG ports.
- glitchc 5y agoThis is an idea akin to "you can drive a car with your feet if you wanted to!"
- tiahura 5y agoThe clickbait buries the lede. The Radxa Taco is an upcoming Pi breakout that come with support for 5 SSDs and 2 Ethernet ports for $100 bucks. Looks cool.
- amelius 5y agoTL;DR: Guy buys underspec'd hardware and is surprised it doesn't perform well.
- JKCalhoun 5y agoThat's what he does though: push the Raspberry Pi to its limits (and this allows us to keep our wallets safe from thinking we might want to throw expensive SSD drives at it).
- MisterTea 5y agoBooo. Its just a pi with expensive ssd disks attached. Nothing interesting here.
- geerlingguy 5y agoThe difference is the Radxa Taco board, which interfaces a CM4 directly with 5x SATA III slots, 1x native NVMe M.2 (2280), and 1x 2.5G NIC. Most typical Pi 'server' builds slap all that IO on top of the Pi 4 model B's USB 3.0 bus.
- MisterTea 5y agoAnd the Pi4 SoC has only 1 pcie 2.0 lane so there's little to gain over USB 3.0 save for some latency.
- geerlingguy 5y agoIn my testing over the past year, almost everything gets a 5-20% speed boost by cutting out the USB 3.0 layer, except for large file sequential copies. It adds up, and is also more reliable and practical for things like ZFS.
- szszrk 5y agoDisagree. Plain Pi won't be able to attach 4 sata drives natively and is not a real alternative for actual NAS. This might be. Jeff does a lot of Pi-NAS testing and I'm thankful for that, as I find of the shelf solutions lacking a lot. Mostly software-wise. There is a huge demand for a simple NAS device that is not much larger then drives it uses, but doesn't have Qnap OS installed on it. It has good network connectivity. 2.5G is a nice option for homelabs and actuall advanced home use.
- MisterTea 5y ago> There is a huge demand for a simple NAS device that is not much larger then drives it uses, but doesn't have Qnap OS installed on it. So how did this article enlighten you on software alternatives to Qnap? Not saying the board is uninteresting, I do like these new PCIe breakout Pi "motherboards". But the whole I built a $5k server is the wrong way to sell it.
- whalesalad 5y ago$5,000 for 152Mb/s read performance - incredible. All sarcasm aside, love this project.
- opheliate 5y agoReally sad to see so many people dismiss this just based on the YouTube thumbnail at the bottom. Jeff consistently makes very high quality content, I really enjoyed his series on Kubernetes, I would urge you to look past any preconceptions there. The fact is, if YouTube makes up a sizeable portion of your income, and that can increase by a factor of 10% just by making a dumb face, you’d be silly not to do so. It’s unfortunate that this is the way things work, but I can personally look past that to ensure that creators who provide value to me are able to put food on the table.
- MeinBlutIstBlau 5y agoThat expression alone tells me "I'm not really an expert, just someone with time, money and trying to appeal to morons." YouTube was cool back in the day cause people didn't make it their livelihoods. It was something to do for fun or to showoff. You just wanted to get popular or well known, not get rich.
- geerlingguy 5y ago> and that can increase by a factor of 10% just by making a dumb face Think more like 20-30% — and not doing it leads to a much quicker falloff from YouTube recommendations, where over 90% of video views come from. I overlook dumb thumbnails when the content is actually enjoyable/entertaining/educational.
- 0des 5y agoNoting the username, are you the guy in the video?
- geerlingguy 5y agoYes.
- AnIdiotOnTheNet 5y agoThere's a thread not too far from this one where people are complaining about the difficulty of cancelling subscriptions to their gyms. I'm sure the gyms use similar reasoning.
- alimbada 5y agoAny chance we can see performance with a more realistic build with say 5 x 1TB/2TB spinning rust disks?
- Nextgrid 5y agoPower may be a concern when it comes to spinning disks.
- geerlingguy 5y agoThe Taco is able to supply enough power for Seagate IronWolf NAS drives. I tested 4 of them, the one concern I have is it doesn't stagger the spinup, so if you did 5 plus a high-power NVMe you might be in trouble right at system boot. But I was using a decent 12v 8A power supply.
- geerlingguy 5y agoI tested that last year in a couple videos comparing an ASUSTOR to a Raspberry Pi: https://www.youtube.com/watch?v=vBccak8f-VY https://www.youtube.com/watch?v=vBccak8f-VY (it works fine, though network copies are still 20-30% slower on the Pi compared to the other NAS). I may do another video on a different NAS that has a CPU closer to the Pi's.
- MR4D 5y agoJeff - you mentioned "Pi's bus being only Gen 2 and x1". Do you see a linear scaling if those interfaces on the Pi are improved - i.e. is there anything else that looks like a near-term bottleneck if those are improved? Great article - I love watching your youtube channel as well
- geerlingguy 5y agoRight now the CPU itself is also a bottleneck quite often. I'm often puzzled by why certain things slow down and find often the Pi's CPU architecture is slightly 'weird', especially in how the PCIe bus works. I hope whatever comes next will have a more standard PCIe bus, and if nothing else Gen 3.1 or Gen 4, even if only x1 (that bandwidth would be a great boost).
- cyberge99 5y agoYeah don’t get all the gruff towards this. It’s an experiment and he took the time to put together a compendium and video about it. I’ve learned quite a few thongs from Jeff and not every creation is going to be an award winner, but at least he did something.
- pmelendez 5y agoIt makes me sad that ~75% of the discussion is around YouTube thumbnails and not the post. A $5k RPi NAS is indeed ridiculous, and was hoping to see some good insights in here.
- II2II 5y agoThe only reason why I followed the link to the article is because of the author and it appeared on HN. If I had seen that thumbnail on YouTube, I would have scrolled past it immediately since it looked ridiculous and like the sort of video that would not offer meaningful insight for the amount time invested into watching it. Thankfully the blog post was upfront about the limitations. (I trust the video would be as well, but the blog post allowed me to reach that conclusion faster.)
- flyinghamster 5y agoFor all the I/O bottlenecks it has, I've still been rather surprised at the Pi 4/400's raw number-crunching power. It compares well to a 3.3 GHz i3-2120 in a clustered number-crunching task I sometimes run (via MPICH) - which seriously surprised me. I suspect that if it were too big to fit in-cache, the Pi would probably fall flat on its face, but still it just floors me that this itsy-bitsy SoC can do so much on seven watts at the wall socket.
- nielsole 5y agoi3-2120 is a 2011 32nm CPU with a TDP of 65W https://www.intel.com/content/www/us/en/products/sku/53426/intel-core-i32120-processor-3m-cache-3-30-ghz/specifications.html https://www.intel.com/content/www/us/en/products/sku/53426/i...
- tejohnso 5y agoWhat's so bad about it? The title is accurate and the actual device is shown, isn't it? At a glance you can see that it's a pi with a lot of attached storage, so the thumbnail actually conveys just about all you'd need to know to decide whether to watch or not. No deception. Seems pretty reasonable. Now if instead of the guy's face and his own hands, there was a bikini model doing yoga...
- ggm 5y agoI have the Radxa 4 port SATA card[1], same JMicron chip, but for a pi4 via USB bridging "clip" in an aluminium heatshink chassis. Its working ok but the fan goes on a lot, I guess driving real spinning rust disks (I couldnt afford SSD) burns heat. Also running ZFS. I love doing backups via zfs snapshot to an external drive. I will probably buy this CM4 based unit. Just came here to say "the gear works, does what it says on the label" based on my experience with a prior card. (now unavailable I believe) I am quite disappointed how slowly 2TB SSD are dropping in price. I use 2TB shucked SATA HDD, they were <$100 AUD each. I actually did think a 2TB unit would by now have dropped to =~ $120 AUD by now. I guess I didn't predict supply chain shock or the stickyness of NAND pricing. [1] https://wiki.radxa.com/Dual_Quad_SATA_HAT https://wiki.radxa.com/Dual_Quad_SATA_HAT
- computershit 5y agoI always enjoy your videos, Jeff -- keep up the good work!