7 ms·

Q - assuming the NAS was strictly used as NAS and not as a server with VMs, is there a point in having a large amount of RAM? (large as in >8GB) I'm not sure w

by speff 10mo ago

Q - assuming the NAS was strictly used as NAS and not as a server with VMs, is there a point in having a large amount of RAM? (large as in >8GB) I'm not sure what the benefit would be since all it's doing is moving information from the drives over to the network.

PikachuEXE 10mo ago
If you use ZFS you might need more RAM for performance?
loloquwowndueo 10mo ago
Caching files in ram means they can be moved to the network faster - right?
- speff 10mo ago
  Makes sense. I didn't know if the FS used RAM for this purpose without some specialized software. PikachuEXE and Mewse mentioned ZFS. Looks like it has native support for caching frequent reads [0]. Good to know [0]: https://www.truenas.com/docs/references/l2arc/ https://www.truenas.com/docs/references/l2arc/
- ac29 10mo ago
  Depends on the network speed. At 1Gbps a single HDD can easily saturate the network with sequential reads. A pair of HDD could do the same at 2.5Gbps. At 10Gbps or more, you would definitely see the benefits of caching in memory.
  - butvacuum 10mo ago
    Not as much as expected. I have several toy ZFS pools out of ancient 3tb wd reds, and anything remotely home-grade (stripped mirrors, 4,6,8 wide raidz1/2) saturates the disks before 10gig networking. As long as it's sequential, 8gb or 128gb doesn't matter.
mewse-hn 10mo ago
ZFS uses a large amount of ram, i think the old rule of thumb was 1GB ram per 1TB of storage
- 01HNNWZ0MV43FF 10mo ago
  Huh. More than just the normal page cache on other filesystems?
  - tekla 10mo ago
    ZFS will eat up as much RAM as you give it as it caches files in memory as accessed.
    - ac29 10mo ago
      All filesystems do this (at least all modern ones, on linux)
  - WarOnPrivacy 10mo ago
    Yes. Parent's comment matches everything I've heard. 32GB is a common recommendation for home lab setups. I run 32 in my TrueNAS builds (36TB and 60TB).
    - magicalhippo 10mo ago
      You can run it with much less. I don't recall the bare minimum but with a bit of tweaking 2GB should be plenty[1]. I recall reading some running it on a 512MB system, but that was a while ago so not sure if you can still go that low. Performance can suffer though, for example low memory will limit the size of the transaction groups. So for decent performance you will want 8GB or more depending on workloads. [1]: https://openzfs.github.io/openzfs-docs/Project%20and%20Community/FAQ.html#hardware-requirements https://openzfs.github.io/openzfs-docs/Project%20and%20Commu...
- yjftsjthsd-h 10mo ago
  That's only for deduplication. https://superuser.com/a/993019 https://superuser.com/a/993019
justsomehnguy 10mo ago
As the other said already if you have more RAM you can have more cache. Honestly it's not that needed but if you would really use the 10Gbit+ networking then 1 second is ~125Mbytes. So depending on your usage you can never even more than 15% utilization or have it almost all if you constantly running something on it ie torrents or using it a SAN/NAS for VM on some other machine. But for a rare occasional home usage nor 32Gb nor this monstrosity and complexity doesn't make sense - just buy some 1-2 bay Synology and forget about it.
- ekropotin 10mo ago
  I won’t be able to sleep having my data just on 1 disk
  - kalleboo 10mo ago
    No matter what you should have an off-site backup as well in case of lightning, flood, fire, virus, etc.
firecall 10mo ago
I am not at all an expert, I can only share my anecdotal unscientific observations! I'm running a TrueNAS box with 3x cheap shucked Seagate drives.* The TrueNAS box has 48GB RAM, is using ZFS and is sharing the drives as a Time Machine destination to a couple of Macs in my office. I can un-confidently say that it feels like the fastest TM device I've ever used! TrueNAS with ZFS feels faster than Open Media Vault(OMV) did on the same hardware. I originally setup OMV on this old gaming PC, as OMV is easy. OMV was reliable, but felt slow compared to how I remembered TrueNAS and ZFS feeling the last time I setup a NAS. So I scrubbed OMV and installed TrueNAS, and purely based on seat-of-pants metrics, ZFS felt faster. And I can confirm that it soaks up most of the 48GB of RAM! TrueNAS reports ZFS Cache currently at 36.4 GiB. I dont know why or how it works, and it's only a Time Machine destination, but there we are those are my metrics and that's what I know LOL * I don't recommend this. They seem unreliable and report errors all the time. But it's just what I had sitting around :-) I'd hoped by now to be able to afford to stick 3x 4TB/8TB SSDs of some sort in the case, but prices are tracking up on SSDs...
thefz 10mo ago
ZFS cache.
aunty_helen 10mo ago
People get carried away with their home lab setups. There's a distinct type of person that thinks they need 100tb of storage in their own house. If you're running a NAS for a company that has many users and multi disc access at the same time, sure. But then you're probably then not buying hdds to shuck and cheap components off ebay.
- some-guy 10mo ago
  My NAS: 1) A refurbished Dell Wyse 5070 (8GB of RAM) with a cheap 64GB SSD from 2013 2) An 8-bay USB-C hard drive enclosure 3) 4 used 12TB hard drives from eBay, 4 3TB drives from 2010 that still somehow haven’t died. 4) A headless Debian with various Linux ISO trackers / downloaders in Docker / Plex (the CPU, though slow, has decent hardware encoding) 5) No RAID, but an rsync script for important Linux ISOs and important data that runs weekly across different drives. I also have cold storage backup by purchasing “Lot of X number” 500GB hard drives on eBay from time to time which store things like photos, music, etc over two drives each. The whole setup didn’t cost me much, and is more than enough for what I need it to do.
  - machomaster 10mo ago
    No Raid? Have you considered using SnapRAID (+ mergerfs if you fancy)? Seems like an ideal use case for the solution.
Aurornis 10mo ago
It depends on your file workload. The RAM can be used as a read cache. I have some workloads where I have to go through a lot of files multiple times and the extra RAM cache makes a huge difference. You can tell when the NAS is pulling from cache or when it has a cache miss.

by speff 10mo ago

PikachuEXE 10mo ago

If you use ZFS you might need more RAM for performance?

loloquwowndueo 10mo ago

Caching files in ram means they can be moved to the network faster - right?

speff 10mo ago
Makes sense. I didn't know if the FS used RAM for this purpose without some specialized software. PikachuEXE and Mewse mentioned ZFS. Looks like it has native support for caching frequent reads [0]. Good to know [0]: https://www.truenas.com/docs/references/l2arc/ https://www.truenas.com/docs/references/l2arc/
ac29 10mo ago
Depends on the network speed. At 1Gbps a single HDD can easily saturate the network with sequential reads. A pair of HDD could do the same at 2.5Gbps. At 10Gbps or more, you would definitely see the benefits of caching in memory.
- butvacuum 10mo ago
  Not as much as expected. I have several toy ZFS pools out of ancient 3tb wd reds, and anything remotely home-grade (stripped mirrors, 4,6,8 wide raidz1/2) saturates the disks before 10gig networking. As long as it's sequential, 8gb or 128gb doesn't matter.

mewse-hn 10mo ago

ZFS uses a large amount of ram, i think the old rule of thumb was 1GB ram per 1TB of storage

01HNNWZ0MV43FF 10mo ago
Huh. More than just the normal page cache on other filesystems?
- tekla 10mo ago
  ZFS will eat up as much RAM as you give it as it caches files in memory as accessed.
  - ac29 10mo ago
    All filesystems do this (at least all modern ones, on linux)
- WarOnPrivacy 10mo ago
  Yes. Parent's comment matches everything I've heard. 32GB is a common recommendation for home lab setups. I run 32 in my TrueNAS builds (36TB and 60TB).
  - magicalhippo 10mo ago
    You can run it with much less. I don't recall the bare minimum but with a bit of tweaking 2GB should be plenty[1]. I recall reading some running it on a 512MB system, but that was a while ago so not sure if you can still go that low. Performance can suffer though, for example low memory will limit the size of the transaction groups. So for decent performance you will want 8GB or more depending on workloads. [1]: https://openzfs.github.io/openzfs-docs/Project%20and%20Community/FAQ.html#hardware-requirements https://openzfs.github.io/openzfs-docs/Project%20and%20Commu...
yjftsjthsd-h 10mo ago
That's only for deduplication. https://superuser.com/a/993019 https://superuser.com/a/993019

justsomehnguy 10mo ago

As the other said already if you have more RAM you can have more cache. Honestly it's not that needed but if you would really use the 10Gbit+ networking then 1 second is ~125Mbytes. So depending on your usage you can never even more than 15% utilization or have it almost all if you constantly running something on it ie torrents or using it a SAN/NAS for VM on some other machine. But for a rare occasional home usage nor 32Gb nor this monstrosity and complexity doesn't make sense - just buy some 1-2 bay Synology and forget about it.

ekropotin 10mo ago
I won’t be able to sleep having my data just on 1 disk
- kalleboo 10mo ago
  No matter what you should have an off-site backup as well in case of lightning, flood, fire, virus, etc.

firecall 10mo ago

I am not at all an expert, I can only share my anecdotal unscientific observations! I'm running a TrueNAS box with 3x cheap shucked Seagate drives.* The TrueNAS box has 48GB RAM, is using ZFS and is sharing the drives as a Time Machine destination to a couple of Macs in my office. I can un-confidently say that it feels like the fastest TM device I've ever used! TrueNAS with ZFS feels faster than Open Media Vault(OMV) did on the same hardware. I originally setup OMV on this old gaming PC, as OMV is easy. OMV was reliable, but felt slow compared to how I remembered TrueNAS and ZFS feeling the last time I setup a NAS. So I scrubbed OMV and installed TrueNAS, and purely based on seat-of-pants metrics, ZFS felt faster. And I can confirm that it soaks up most of the 48GB of RAM! TrueNAS reports ZFS Cache currently at 36.4 GiB. I dont know why or how it works, and it's only a Time Machine destination, but there we are those are my metrics and that's what I know LOL * I don't recommend this. They seem unreliable and report errors all the time. But it's just what I had sitting around :-) I'd hoped by now to be able to afford to stick 3x 4TB/8TB SSDs of some sort in the case, but prices are tracking up on SSDs...

thefz 10mo ago

ZFS cache.

aunty_helen 10mo ago

People get carried away with their home lab setups. There's a distinct type of person that thinks they need 100tb of storage in their own house. If you're running a NAS for a company that has many users and multi disc access at the same time, sure. But then you're probably then not buying hdds to shuck and cheap components off ebay.

some-guy 10mo ago
My NAS: 1) A refurbished Dell Wyse 5070 (8GB of RAM) with a cheap 64GB SSD from 2013 2) An 8-bay USB-C hard drive enclosure 3) 4 used 12TB hard drives from eBay, 4 3TB drives from 2010 that still somehow haven’t died. 4) A headless Debian with various Linux ISO trackers / downloaders in Docker / Plex (the CPU, though slow, has decent hardware encoding) 5) No RAID, but an rsync script for important Linux ISOs and important data that runs weekly across different drives. I also have cold storage backup by purchasing “Lot of X number” 500GB hard drives on eBay from time to time which store things like photos, music, etc over two drives each. The whole setup didn’t cost me much, and is more than enough for what I need it to do.
- machomaster 10mo ago
  No Raid? Have you considered using SnapRAID (+ mergerfs if you fancy)? Seems like an ideal use case for the solution.

Aurornis 10mo ago

It depends on your file workload. The RAM can be used as a read cache. I have some workloads where I have to go through a lot of files multiple times and the extra RAM cache makes a huge difference. You can tell when the NAS is pulling from cache or when it has a cache miss.