3 ms·

I have always wanted to set up a ceph system with one drive per node. The ideal form factor would be a drive with a couple network interfaces built in. western

by somat 3y ago

I have always wanted to set up a ceph system with one drive per node. The ideal form factor would be a drive with a couple network interfaces built in. western digital had a press release about an experiment they did that was exactly this, but it never ended up with drive you could buy. The hardkernel HC2 SOC was a nearly ideal form factor for this, and I still have a stack of them laying around that I bought to make a ceph cluster, but I ran out of steam when I figured out they were 32bit. not to say it would be impossible I just never did it.

progval 3y ago
I used to use Ceph Luminous (v12) on these, they worked fine. Unfortunately, a bug in Nautilus (v14) prevented 32-bits and 64-bits archs from talking to each other. Pacific (v16) allegedly solves this, but I didn't try it: https://ceph.com/en/news/blog/2021/v16-2-5-pacific-released/ https://ceph.com/en/news/blog/2021/v16-2-5-pacific-released/ If you want to try it with a more modern (and 64-bits) device, the hardkernel HC4 might do it for you. It's conceptually similar to the HC2 but has two drives. Unfortunately it only has double the RAM (4GB), which is probably not enough anymore.
- eurekin 3y ago
  Looks so good, wish for a > 1gbit version, since HDDs alone can saturate that
  - progval 3y ago
    Did you look at their H3? It's pricier but it has two 2.5Gbits ports (along with a NVMe slot and an Intel CPU)
    - eurekin 3y ago
      I have one and love it! It bravely holds together my intranet dev services :) For a ceph node would still consider a version with 10gbit eth
narism 3y ago
Sounds like the Seagate Kinetic HDD: https://www.seagate.com/www-content/product-content/hdd-fam/kinetic-hdd/_shared/docs/kinetic-ds1835-2-1503us.pdf https://www.seagate.com/www-content/product-content/hdd-fam/...
- somat 3y ago
  That would be perfect. Unfortunately, going by the data sheet it would not run ceph you would have to work with seagate's proprietary object store. I will note that as far as I can tell it is unobtainium. none of the usual vendors stock them, you probably have to prove to seagate that you are a "serious enterprise customer" and commit to a thousand units before they will let you buy some.

by somat 3y ago

progval 3y ago

I used to use Ceph Luminous (v12) on these, they worked fine. Unfortunately, a bug in Nautilus (v14) prevented 32-bits and 64-bits archs from talking to each other. Pacific (v16) allegedly solves this, but I didn't try it: https://ceph.com/en/news/blog/2021/v16-2-5-pacific-released/ https://ceph.com/en/news/blog/2021/v16-2-5-pacific-released/ If you want to try it with a more modern (and 64-bits) device, the hardkernel HC4 might do it for you. It's conceptually similar to the HC2 but has two drives. Unfortunately it only has double the RAM (4GB), which is probably not enough anymore.

eurekin 3y ago
Looks so good, wish for a > 1gbit version, since HDDs alone can saturate that
- progval 3y ago
  Did you look at their H3? It's pricier but it has two 2.5Gbits ports (along with a NVMe slot and an Intel CPU)
  - eurekin 3y ago
    I have one and love it! It bravely holds together my intranet dev services :) For a ceph node would still consider a version with 10gbit eth

narism 3y ago

Sounds like the Seagate Kinetic HDD: https://www.seagate.com/www-content/product-content/hdd-fam/kinetic-hdd/_shared/docs/kinetic-ds1835-2-1503us.pdf https://www.seagate.com/www-content/product-content/hdd-fam/...

somat 3y ago
That would be perfect. Unfortunately, going by the data sheet it would not run ceph you would have to work with seagate's proprietary object store. I will note that as far as I can tell it is unobtainium. none of the usual vendors stock them, you probably have to prove to seagate that you are a "serious enterprise customer" and commit to a thousand units before they will let you buy some.