3 ms·

For a less ad hoc approach to reducing storage costs, I suggest looking into the ZFS filesystem. Compression is completely transparent in ZFS. Once you enable c

For a less ad hoc approach to reducing storage costs, I suggest looking into the ZFS filesystem. Compression is completely transparent in ZFS. Once you enable compression in ZFS, all of your files will automatically be compressed when written, and decompressed when read. I am currently managing a Postgres cluster with a petabyte of data in it. We found ZFS to be a great way to reduce overall storage costs. We just switched our machines to machines running ZFS, and we were suddenly using 1/3rd the amount of disk space. Although it took us a while to learn all of the gotchas of ZFS, it wound up saving us a huge amount of $$$. (As I understand it, ZFS would not have helped in Flickr's case. Since JPEGs are already compressed, ZFS would not have provided any benefit. Flickr was able to save storage by using an ad hoc compression algorithm.)

devty 10y ago
The article suggested that they had to carefully balance the processing cost of compressing and decompressing data and the storage cost. Did the use of ZFS in your system incur noticeable processing cost in your system? Any noticeable increase in latency on your system?
- malisper 10y ago
  When we first switched to ZFS, we had some ingestion issues. It turned out the problems were caused by poor architecture decisions we made and was merely exaggerated by ZFS. So yes, there was a noticeable increase in latency, but ZFS was only a tiny piece of it.
brokenmachine 10y ago
How much RAM do you need for a Petabyte ZFS cluster? I investigated ZFS for my home server and it was recommended to have 1Gb RAM for every Tb, and much more if you enabled deduplication/compression (I forget which).
- olavgg 10y ago
  The 1GB ram per TB is wrong. More memory definitely helps, especially if you add an L2ARC cache which can use a lot of memory just for mapping. But in most cases you are probably fine with just 1-2GB of ram. I would never use deduplication on ZFS, it is very slow even if you have enough RAM. And in most cases, savings are less than 10%.
deleted 10y ago
[deleted]

devty 10y ago

The article suggested that they had to carefully balance the processing cost of compressing and decompressing data and the storage cost. Did the use of ZFS in your system incur noticeable processing cost in your system? Any noticeable increase in latency on your system?

malisper 10y ago
When we first switched to ZFS, we had some ingestion issues. It turned out the problems were caused by poor architecture decisions we made and was merely exaggerated by ZFS. So yes, there was a noticeable increase in latency, but ZFS was only a tiny piece of it.

brokenmachine 10y ago

How much RAM do you need for a Petabyte ZFS cluster? I investigated ZFS for my home server and it was recommended to have 1Gb RAM for every Tb, and much more if you enabled deduplication/compression (I forget which).

olavgg 10y ago
The 1GB ram per TB is wrong. More memory definitely helps, especially if you add an L2ARC cache which can use a lot of memory just for mapping. But in most cases you are probably fine with just 1-2GB of ram. I would never use deduplication on ZFS, it is very slow even if you have enough RAM. And in most cases, savings are less than 10%.

deleted 10y ago

[deleted]