5 ms·

TBH, not sure if spending $3500 on 40TB storage v.s. ~$800 with rotating disks at the same storage capacity. You can put $200 on top with a 2TB NVMe SSD as cach

by liuliu 4y ago

TBH, not sure if spending $3500 on 40TB storage v.s. ~$800 with rotating disks at the same storage capacity. You can put $200 on top with a 2TB NVMe SSD as cache. The reason to question this is that 40TB seems small if you want to have a NAS for small video editing studios. And for personal use, you probably not going to need more than 2TB work set paged in at any given moment.

walrus01 4y ago
personally if I had to do this I would go with rotating disk for bulk storage in a NAS, and something like two 2TB to 4TB size NVME SSDs in a proper video editing workstation motherboard directly attached to pci-e 4.0 bus. This will be considerably faster for working with "immediate" needs of video files rather than over a 10GbE network. like, a difference of 900MBps over network vs 2500MBps with local sequential read/writes on NVME SSD on same motherboard.
- TheDudeMan 4y ago
  "I find myself needing a central storage solution disconnected from my main workstation."
  - walrus01 4y ago
    there would still be such in what I described, just that it would have a much higher capacity composed of 16-18TB each spinning disks.
    - geerlingguy 4y ago
      In this case the storage is used directly for the editing, though. I am able to close a project on one computer, hop over to another and within seconds start editing at full speed (without requiring proxy footage). HDDs can't provide the latency that I need, unless I was running a dozen or two to overcome the horrible seek speeds (or had at least double the RAM so the entire project would be cached in RAM).
      - derefr 4y ago
        What's wrong with proxy footage, though? Especially if the NAS was hyperconverged-in-the-small (i.e. had the spare compute to render out proxy footage for uploaded files in the background.)
        geerlingguy 4y ago
        Nothing really, but it's just one extra layer of inconvenience. If you can edit full res, it's nice to be able to (especially for color grading and cropping purposes).
jjcm 4y ago
Possibly for smaller projects, for anything remotely sizable, 2TB is likely not going to cut it. 5k prores is 1TB for every 30min of footage, which means you're only getting an hour out of a 2TB drive. Storage needs for any pro video workflow get very large, very quick.
- tiernano 4y ago
  That is true... But in that case, double or quadruple your SSD...
- Dylan16807 4y ago
  Well you're talking about 60fps ProRes 4444 (XQ?) footage, and the article is talking about 30fps ProRes 422 LT footage. That's 6.5x (10x?) less data for the same size project, and not much difference in how "pro" it is.
rektide 4y ago
For compare $16/TB is pretty awesome[1], which for that budget would be about 217GB. ~5.5X. [1] https://diskprices.com/?locale=us&condition=new&capacity=12-&disk_types=internal_hdd https://diskprices.com/?locale=us&condition=new&capacity=12-...
sorenjan 4y ago
I'm not familiar with NAS file systems. Is it fairly straight forward to use hard drives with SSDs as transparent cache, and make it look like a single file share?
- js2 4y ago
  I had written a long comment about ZFS caching and still didn't do a good job, so let me link you to this instead: https://www.truenas.com/docs/references/slog/ https://www.truenas.com/docs/references/slog/ The short version is: - ZFS caches reads into RAM. Gobs of RAM helps. You can also add a secondary read cache device (L2ARC). - ZFS can caches synchronous writes to an SLOG device, but if you've disabled synchronous writes it won't make a difference. An SLOG device makes no difference for asynchronous writes. I use ZFS for my Time Machine backups (among other things) and have synchronous writes disabled for the Time Machine datasets. Before we had ZFS, the traditional way to speed up NAS was a raid controller with a battery-backed RAM cache.
  - pdimitar 4y ago
    How do you use your ZFS NAS for Time Machine backups? Very curious, I'd like to do the same.
    - js2 4y ago
      I'm using TrueNAS which makes it pretty easy. I'm doing it via SMB (Samba). Previously I was doing it via AFP (Netatalk). This looks more or less correct: https://www.reddit.com/r/MacOS/comments/lh0yjc/configure_a_truenas_core_share_as_a_time_machine/ https://www.reddit.com/r/MacOS/comments/lh0yjc/configure_a_t... Basically it creates a dataset and a Samba share, and the configures Samba with a couple plugins so that: 1) a dataset is created for each user that connects to the share, so there's per-user datasets; 2) a ZFS snapshot is created automatically whenever Time Machine disconnects from the share. (2) is so that you can roll back if need be. In the past, I've had Time Machine decide that the sparse image it uses for a backup is corrupt and it wants to zero it out and start over. The ZFS snapshots lets you recover from that, though I haven't had it occur under macOS Monterey and/or since I switched to SMB from AFP. On the parent dataset I disabled sync. If you google for "TrueNAS Time Machine" you'll find some discussions of all this in the TrueNAS forums.
mattalex 4y ago
He also has a hard drive based NAS, this is an additional local buffer for current work.

by liuliu 4y ago

walrus01 4y ago

personally if I had to do this I would go with rotating disk for bulk storage in a NAS, and something like two 2TB to 4TB size NVME SSDs in a proper video editing workstation motherboard directly attached to pci-e 4.0 bus. This will be considerably faster for working with "immediate" needs of video files rather than over a 10GbE network. like, a difference of 900MBps over network vs 2500MBps with local sequential read/writes on NVME SSD on same motherboard.

TheDudeMan 4y ago
"I find myself needing a central storage solution disconnected from my main workstation."
- walrus01 4y ago
  there would still be such in what I described, just that it would have a much higher capacity composed of 16-18TB each spinning disks.
  - geerlingguy 4y ago
    In this case the storage is used directly for the editing, though. I am able to close a project on one computer, hop over to another and within seconds start editing at full speed (without requiring proxy footage). HDDs can't provide the latency that I need, unless I was running a dozen or two to overcome the horrible seek speeds (or had at least double the RAM so the entire project would be cached in RAM).
    - derefr 4y ago
      What's wrong with proxy footage, though? Especially if the NAS was hyperconverged-in-the-small (i.e. had the spare compute to render out proxy footage for uploaded files in the background.)
      - geerlingguy 4y ago
        Nothing really, but it's just one extra layer of inconvenience. If you can edit full res, it's nice to be able to (especially for color grading and cropping purposes).

jjcm 4y ago

Possibly for smaller projects, for anything remotely sizable, 2TB is likely not going to cut it. 5k prores is 1TB for every 30min of footage, which means you're only getting an hour out of a 2TB drive. Storage needs for any pro video workflow get very large, very quick.

tiernano 4y ago
That is true... But in that case, double or quadruple your SSD...
Dylan16807 4y ago
Well you're talking about 60fps ProRes 4444 (XQ?) footage, and the article is talking about 30fps ProRes 422 LT footage. That's 6.5x (10x?) less data for the same size project, and not much difference in how "pro" it is.

rektide 4y ago

For compare $16/TB is pretty awesome[1], which for that budget would be about 217GB. ~5.5X. [1] https://diskprices.com/?locale=us&condition=new&capacity=12-&disk_types=internal_hdd https://diskprices.com/?locale=us&condition=new&capacity=12-...

sorenjan 4y ago

I'm not familiar with NAS file systems. Is it fairly straight forward to use hard drives with SSDs as transparent cache, and make it look like a single file share?

js2 4y ago
I had written a long comment about ZFS caching and still didn't do a good job, so let me link you to this instead: https://www.truenas.com/docs/references/slog/ https://www.truenas.com/docs/references/slog/ The short version is: - ZFS caches reads into RAM. Gobs of RAM helps. You can also add a secondary read cache device (L2ARC). - ZFS can caches synchronous writes to an SLOG device, but if you've disabled synchronous writes it won't make a difference. An SLOG device makes no difference for asynchronous writes. I use ZFS for my Time Machine backups (among other things) and have synchronous writes disabled for the Time Machine datasets. Before we had ZFS, the traditional way to speed up NAS was a raid controller with a battery-backed RAM cache.
- pdimitar 4y ago
  How do you use your ZFS NAS for Time Machine backups? Very curious, I'd like to do the same.
  - js2 4y ago
    I'm using TrueNAS which makes it pretty easy. I'm doing it via SMB (Samba). Previously I was doing it via AFP (Netatalk). This looks more or less correct: https://www.reddit.com/r/MacOS/comments/lh0yjc/configure_a_truenas_core_share_as_a_time_machine/ https://www.reddit.com/r/MacOS/comments/lh0yjc/configure_a_t... Basically it creates a dataset and a Samba share, and the configures Samba with a couple plugins so that: 1) a dataset is created for each user that connects to the share, so there's per-user datasets; 2) a ZFS snapshot is created automatically whenever Time Machine disconnects from the share. (2) is so that you can roll back if need be. In the past, I've had Time Machine decide that the sparse image it uses for a backup is corrupt and it wants to zero it out and start over. The ZFS snapshots lets you recover from that, though I haven't had it occur under macOS Monterey and/or since I switched to SMB from AFP. On the parent dataset I disabled sync. If you google for "TrueNAS Time Machine" you'll find some discussions of all this in the TrueNAS forums.

mattalex 4y ago

He also has a hard drive based NAS, this is an additional local buffer for current work.