3 ms·

Glad to see Moosefs mentioned, I've used it a fee years ago to reuse old PCs for storage. If you aren't hosting data with strict SLA, a master and standby nodes

by a012 6y ago

Glad to see Moosefs mentioned, I've used it a fee years ago to reuse old PCs for storage. If you aren't hosting data with strict SLA, a master and standby nodes work great.

FilingTrader 6y ago
MooseFS is great. I actually tried LizardFS first, and it was painfully slow. Only later, after settling on MooseFS did I read that erasure coding can slow things down, and erasure coding was the only thing I tried with LizardFS. So, I still can't say for sure that MooseFS is faster. Anyway, it appears to be better supported at the present time, although I did like that LizardFS can have multiple hot masters without paying for the pro version.
- gravypod 6y ago
  Have you looked at Ceph ad a storage backend? It allows per-file erasure coding configuration and has a lot of cool features. It can also manage raw block devices from what I'm told. It's really awesome you've gotten a chance to build up this cluster. I wish more startups realized that, despite the initial price tag, some of this is way better than what you'll get from AWS. Your NVMe is likely 10x to 100x faster than premium ssd storage at a fraction of the price. If you ever use GPUs you'll be saving loads of money. It's hard to convince people though that bare metal isn't scary.
  - FilingTrader 6y ago
    I haven't tried Ceph. It didn't seem as appealing, judging from comments I found (mostly on Hacker News): http://www.regressionist.com/2020/06/20/reviews-of-distributed-filesystems/ http://www.regressionist.com/2020/06/20/reviews-of-distribut... But it's also hard to tell exactly what people are reviewing. Evidently CephFS is less mature than Ceph object store. But, yeah. This just all seems so much cheaper than AWS. There are other advantages, too. Like, a lot of stuff is better with the low latency that comes with having the cluster on-location. Python plots are more pleasant to interact with (ssh -X). NoMachine remote desktop is faster, etc. Also, it's a bit like an all-you-can-eat buffet. You pay once, and then you can just relax and enjoy. I have enough storage now that I don't need to stress about costs before deciding to cache intermediate data. And I don't need to worry about whether I really need those older results that I still have saved on disk that I'm paying each minute to store on AWS. Those kind of thoughts are just gone. I do have one GPU now, a 1080 Ti ($450 on eBay). I am ready to house a total of 6, but I'm waiting to buy more until I really need them.

by a012 6y ago

Glad to see Moosefs mentioned, I've used it a fee years ago to reuse old PCs for storage. If you aren't hosting data with strict SLA, a master and standby nodes work great.

FilingTrader 6y ago

MooseFS is great. I actually tried LizardFS first, and it was painfully slow. Only later, after settling on MooseFS did I read that erasure coding can slow things down, and erasure coding was the only thing I tried with LizardFS. So, I still can't say for sure that MooseFS is faster. Anyway, it appears to be better supported at the present time, although I did like that LizardFS can have multiple hot masters without paying for the pro version.

gravypod 6y ago
Have you looked at Ceph ad a storage backend? It allows per-file erasure coding configuration and has a lot of cool features. It can also manage raw block devices from what I'm told. It's really awesome you've gotten a chance to build up this cluster. I wish more startups realized that, despite the initial price tag, some of this is way better than what you'll get from AWS. Your NVMe is likely 10x to 100x faster than premium ssd storage at a fraction of the price. If you ever use GPUs you'll be saving loads of money. It's hard to convince people though that bare metal isn't scary.
- FilingTrader 6y ago
  I haven't tried Ceph. It didn't seem as appealing, judging from comments I found (mostly on Hacker News): http://www.regressionist.com/2020/06/20/reviews-of-distributed-filesystems/ http://www.regressionist.com/2020/06/20/reviews-of-distribut... But it's also hard to tell exactly what people are reviewing. Evidently CephFS is less mature than Ceph object store. But, yeah. This just all seems so much cheaper than AWS. There are other advantages, too. Like, a lot of stuff is better with the low latency that comes with having the cluster on-location. Python plots are more pleasant to interact with (ssh -X). NoMachine remote desktop is faster, etc. Also, it's a bit like an all-you-can-eat buffet. You pay once, and then you can just relax and enjoy. I have enough storage now that I don't need to stress about costs before deciding to cache intermediate data. And I don't need to worry about whether I really need those older results that I still have saved on disk that I'm paying each minute to store on AWS. Those kind of thoughts are just gone. I do have one GPU now, a 1080 Ti ($450 on eBay). I am ready to house a total of 6, but I'm waiting to buy more until I really need them.