4 ms·

Interesting device! There's more details linked in the CNX Soft post [1], in particular, a thread on ODROID.com showing benchmarks [2]. The two disks are connec

by zxcvgm 6y ago

Interesting device! There's more details linked in the CNX Soft post [1], in particular, a thread on ODROID.com showing benchmarks [2]. The two disks are connected to the CPU via the ASM1061 SATA to PCIe x1 controller, meaning it can't fully handle max throughput from 2 SSDs, but I guess spinning drives should be fine. [1]: https://www.cnx-software.com/2020/10/20/odroid-hc4-low-cost-dual-nas-comes-with-4gb-ram-supports-2-5-inch-and-3-5-inch-sata-drives/ https://www.cnx-software.com/2020/10/20/odroid-hc4-low-cost-... [2]: https://forum.odroid.com/viewtopic.php?f=29&t=40609 https://forum.odroid.com/viewtopic.php?f=29&t=40609

MisterTea 6y ago
> The two disks are connected to the CPU via the ASM1061 SATA to PCIe x1 controller, meaning it can't fully handle max throughput from 2 SSDs... The network interface is a single gigabit port so this does not matter one bit.
- BlackLotus89 6y ago
  Depending on your use case I would argue. If you use it as a pure nas it wouldn't matter. If you want to do fast encoding or something similar and get bottlenecked by your storage it could be annoying. I do post processing on my NAS and use my SSD as swap so I like every mb/sec I can get :)
  - escardin 6y ago
    Then you're using something bigger than a Cortex-A55?
    - BlackLotus89 6y ago
      Edit: arg thought the comment was responding to a different comment from me. (Where I talked about my home ceph setup that was another reason why I need fast swap) Yes I build something using a standard x86_64 (celeron) setup. I heavily experimented with an all arm setup thought. I still find it pretty hard to balance a cheap build that is expandable. I recently lost an OSD and my setup had a consistently high load while recovering. I don't really like the power consumption of my current setup and am thinking of moving to arm. (I use multiple small HDDs between 1tb and 10tb and have different pools with different redundancys. Am thinking of replacing it with a simple raid or something and 3-4 12tb HDDs with a simple redundancy. Could use minio as s3 replacement and btrfs as fs [I won't use zfs for different reasons] btrfs has snapshots, compression and deduplication so everything I want and need. Never did use rbd over network) Edit: This is my home setup I'm talking about. At work we got a real setup.
      - myrandomcomment 6y ago
        Just buy a FreeNAS mini from IXSystems. It just works. I have had one for 6 years now with no issue. Yes it cost money, but if you can pay for 4x12TB then I assume you can pay for the mini. :) https://www.truenas.com/truenas-mini/ https://www.truenas.com/truenas-mini/
        JAlexoid 6y ago
        Why not just buy a low power desktop or one of those thin clients on sale and DIY? You can even 3D print a nice enclosure, these days. If you're DIYing, then DIY.
        myrandomcomment 6y ago
        When I was younger and still an everyday networking guy I had a rack with tons of kit, ran my own DNS, firewall was OpenBSD, etc. Today I have a family and a sr. role at a startup. My network is 100% Ubiquiti, storage is a FreeNAS Mini and every TV has an AppleTV with the same apps and setup. Why because I do not have time to play anymore. Why because if I get killed in a car accident my non-techy wife can understand the tools I left.
      - rektide 6y ago
        My Celeron boxes spend most of their life at ~6W, burst to 15W. Drives are all attached via USB, but since these are Intel Broadwell based Celerons (5 years old now! wow, long in the tooth. time flies), there are 4x USB3.0 root complexes that can run pretty much full tilt. Trying to go lower power doesn't feel worth it, particularly when the hard drives drink so much power. I feel like I'm spilling the beans sharing this, but ServeTheHome has has an excellent "TinyMiniMicro" series on ~1L size mini-PCs, which are used in businesses a lot. They are much beefier boxes than my little celeron, now often 6 core with way higher clocks. They come with 35W and 65W cpus, but even the couple generations old models still tend to idle at ~10W & otherwise sip power only as demanded[1]. If power is your concern, get rid of your small hard drives. Switch to ARM? Meh. [1] https://www.servethehome.com/hp-elitedesk-800-g4-mini-tinyminimicro-guide-review/3/ https://www.servethehome.com/hp-elitedesk-800-g4-mini-tinymi...
bserge 6y ago
Oh... ASMedia... Judging by their older controllers, they're going to need a heatsink, and even then they can fail out of the blue... At least it's SATA. This is exactly what I've been looking for, a cheap Ethernet connected device that can saturate 1GbE. It's perfect for backups.
- rektide 6y ago
  Well, cheap may rule this out... for now, but... Skip the middleman and directly connect your hard drive[1]! I think you may have trouble saturating the hard drive's dual 25Gbe links, and a decent switch is gonna be pricey, but that backup should be going very VERY fast indeed when you're done. Kioxia (nee Toshiba) is not the only one doing this, thankfully, cause I think it's interesting as heck. A lot of hard drives have a good amount of speedy Cortex R processors on them already, why use those to talk NVMe or SATA when you could have the hard drive talking ethernet directly? [1] https://business.kioxia.com/en-us/news/2020/ssd-20200922-2.html https://business.kioxia.com/en-us/news/2020/ssd-20200922-2.h...
  - JAlexoid 6y ago
    When you're looking for durable long term backup solution - it's either high TBW values for NVMe drives or, practically, hard drives. Paying about $1000 for 20TB of fault tolerant extra long term storage isn't a problem. I'm even starting to consider magnetic tapes for archives. What's the price of a TB of high TBW NVMe storage?
    - rektide 6y ago
      I really wish we had better understanding about the failure modes of NVMe past wearing out the flash. Lets say we're writing a vibration sensor's data to the drive, once an hour. We leave 50% of the drive unpartitioned to give the drive controller plenty of headroom for whatever it wants. It might take centuries to get to the expected TBW. Yet there is still some kind of MTBF that is likely to happen. This above workload scenario is 100% write based. What about drives in data-centers that have lots of user images that they are unlikely/not-incentivized to prune. The drive could, hypothetically, only need to be written once, then forever read. When do sectors start going bad here? Or is the failure mode otherwise, in other componenets? Will this drive last a century? More? What other modes of failure should we expect, how probable are they? I love your challenge here @JAlexoid. Switching from thinking in terms of TB/$ to TB/y/$ is a great twist.

by zxcvgm 6y ago

MisterTea 6y ago

> The two disks are connected to the CPU via the ASM1061 SATA to PCIe x1 controller, meaning it can't fully handle max throughput from 2 SSDs... The network interface is a single gigabit port so this does not matter one bit.

BlackLotus89 6y ago
Depending on your use case I would argue. If you use it as a pure nas it wouldn't matter. If you want to do fast encoding or something similar and get bottlenecked by your storage it could be annoying. I do post processing on my NAS and use my SSD as swap so I like every mb/sec I can get :)
- escardin 6y ago
  Then you're using something bigger than a Cortex-A55?
  - BlackLotus89 6y ago
    Edit: arg thought the comment was responding to a different comment from me. (Where I talked about my home ceph setup that was another reason why I need fast swap) Yes I build something using a standard x86_64 (celeron) setup. I heavily experimented with an all arm setup thought. I still find it pretty hard to balance a cheap build that is expandable. I recently lost an OSD and my setup had a consistently high load while recovering. I don't really like the power consumption of my current setup and am thinking of moving to arm. (I use multiple small HDDs between 1tb and 10tb and have different pools with different redundancys. Am thinking of replacing it with a simple raid or something and 3-4 12tb HDDs with a simple redundancy. Could use minio as s3 replacement and btrfs as fs [I won't use zfs for different reasons] btrfs has snapshots, compression and deduplication so everything I want and need. Never did use rbd over network) Edit: This is my home setup I'm talking about. At work we got a real setup.
    - myrandomcomment 6y ago
      Just buy a FreeNAS mini from IXSystems. It just works. I have had one for 6 years now with no issue. Yes it cost money, but if you can pay for 4x12TB then I assume you can pay for the mini. :) https://www.truenas.com/truenas-mini/ https://www.truenas.com/truenas-mini/
      - JAlexoid 6y ago
        Why not just buy a low power desktop or one of those thin clients on sale and DIY? You can even 3D print a nice enclosure, these days. If you're DIYing, then DIY.
        myrandomcomment 6y ago
        When I was younger and still an everyday networking guy I had a rack with tons of kit, ran my own DNS, firewall was OpenBSD, etc. Today I have a family and a sr. role at a startup. My network is 100% Ubiquiti, storage is a FreeNAS Mini and every TV has an AppleTV with the same apps and setup. Why because I do not have time to play anymore. Why because if I get killed in a car accident my non-techy wife can understand the tools I left.
    - rektide 6y ago
      My Celeron boxes spend most of their life at ~6W, burst to 15W. Drives are all attached via USB, but since these are Intel Broadwell based Celerons (5 years old now! wow, long in the tooth. time flies), there are 4x USB3.0 root complexes that can run pretty much full tilt. Trying to go lower power doesn't feel worth it, particularly when the hard drives drink so much power. I feel like I'm spilling the beans sharing this, but ServeTheHome has has an excellent "TinyMiniMicro" series on ~1L size mini-PCs, which are used in businesses a lot. They are much beefier boxes than my little celeron, now often 6 core with way higher clocks. They come with 35W and 65W cpus, but even the couple generations old models still tend to idle at ~10W & otherwise sip power only as demanded[1]. If power is your concern, get rid of your small hard drives. Switch to ARM? Meh. [1] https://www.servethehome.com/hp-elitedesk-800-g4-mini-tinyminimicro-guide-review/3/ https://www.servethehome.com/hp-elitedesk-800-g4-mini-tinymi...

bserge 6y ago

Oh... ASMedia... Judging by their older controllers, they're going to need a heatsink, and even then they can fail out of the blue... At least it's SATA. This is exactly what I've been looking for, a cheap Ethernet connected device that can saturate 1GbE. It's perfect for backups.

rektide 6y ago
Well, cheap may rule this out... for now, but... Skip the middleman and directly connect your hard drive[1]! I think you may have trouble saturating the hard drive's dual 25Gbe links, and a decent switch is gonna be pricey, but that backup should be going very VERY fast indeed when you're done. Kioxia (nee Toshiba) is not the only one doing this, thankfully, cause I think it's interesting as heck. A lot of hard drives have a good amount of speedy Cortex R processors on them already, why use those to talk NVMe or SATA when you could have the hard drive talking ethernet directly? [1] https://business.kioxia.com/en-us/news/2020/ssd-20200922-2.html https://business.kioxia.com/en-us/news/2020/ssd-20200922-2.h...
- JAlexoid 6y ago
  When you're looking for durable long term backup solution - it's either high TBW values for NVMe drives or, practically, hard drives. Paying about $1000 for 20TB of fault tolerant extra long term storage isn't a problem. I'm even starting to consider magnetic tapes for archives. What's the price of a TB of high TBW NVMe storage?
  - rektide 6y ago
    I really wish we had better understanding about the failure modes of NVMe past wearing out the flash. Lets say we're writing a vibration sensor's data to the drive, once an hour. We leave 50% of the drive unpartitioned to give the drive controller plenty of headroom for whatever it wants. It might take centuries to get to the expected TBW. Yet there is still some kind of MTBF that is likely to happen. This above workload scenario is 100% write based. What about drives in data-centers that have lots of user images that they are unlikely/not-incentivized to prune. The drive could, hypothetically, only need to be written once, then forever read. When do sectors start going bad here? Or is the failure mode otherwise, in other componenets? Will this drive last a century? More? What other modes of failure should we expect, how probable are they? I love your challenge here @JAlexoid. Switching from thinking in terms of TB/$ to TB/y/$ is a great twist.