3 ms·
> That's a solved problem ... compute farm with a clustered or distributed file system, a copy data management solution, or cachefs, for example. Thank you for
by jpeloquin 9y ago
> That's a solved problem ... compute farm with a clustered or distributed file system, a copy data management solution, or cachefs, for example.
Thank you for suggesting some potential solutions to my data management problems. From googling them, I get the impression that you are thinking primarily of enterprise scale data management (i.e., organizations with server farms and an IT department), whereas I'm primarily thinking of organizations with < 30 employees (who mostly use desktop software) and a single file server. My particular situation is an academic lab. However, I think these solutions can can still work with a little adaptation:
Copy data management seems to be the use of block level data deduplication or virtual disks on a server in order to decrease the disk utilization per VM or per container. I'm not completely sure; I found mostly marketing documents, and there's no wikipedia entry. This, as well as clustered/distributed file systems, would apply if we turned the file server into a VM server and had each employee work on a VM via remote desktop. In principle, this could work, and would let temporary employees (e.g., summer students and visiting scholars) get started quickly with a standard OS environment. I will experiment with this when our last batch of desktop PCs hit end of life.
CacheFS looks useful if we start using NFS to connect to our file server instead of Samba. It looks like Windows 10 (Enterprise) supports NFS caching too (https://technet.microsoft.com/en-us/library/cc976862.aspx https://technet.microsoft.com/en-us/library/cc976862.aspx). I will try this.