3 ms·

Show HN: GnuRAMage – A Bash tool to sync files between HDDs and RAM disks

2 points by mokulanis 1y ago

I've developed GnuRAMage, a Bash script to manage large RAM disks by synchronizing data with persistent storage (HDDs/SSDs). It's for anyone wanting to use the speed of RAM disks for frequently accessed files without risking data loss due to RAM's volatility. The script works by: Initializing from a config file (GnuRAMage.ini), checking paths and dependencies (rsync). Performing an initial copy of specified files/directories from a source (e.g., HDD) to the RAM disk. Entering a monitoring loop to periodically synchronize changes from the RAM disk back to the source using rsync. Handling interruptions (like Ctrl+C) gracefully with a final sync attempt before exiting. Key features: Simple INI configuration. Rsync-style exclusion patterns. Dry-run mode for testing. Comprehensive logging. One-time sync mode. Optional checksum verification. Can generate standalone scripts for initial copy and periodic sync, useful for cron or systemd. It's built with Bash and rsync, aiming for simplicity. Installation is straightforward: Bash git clone git@github.com:FPGArtktic/GnuRAMage.git cd GnuRAMage chmod +x gramage.sh # Then copy GnuRAMage.ini.example to GnuRAMage.ini and configure I created GnuRAMage to solve my own challenges with making large RAM disks practical and safe. I'm looking for feedback, suggestions, and any use cases I might have overlooked. GitHub (includes full README with detailed config, options, testing, etc.): https://github.com/FPGArtktic/GnuRAMage https://github.com/FPGArtktic/GnuRAMage Thanks for checking it out!

atmanactive 1y ago
I thought Linux buffering holds everything important in RAM anyway, no? If I had 1GiB of files served by Apache/Nginx and changed only once every 24 hours, then, stuffing it all in a RAM disk wouldn't make any difference on performance. Or would it?
- compressedgas 1y ago
  Yes, but there is no means to pin files and directories into memory so that they never get evicted from the file system cache. If there were, one could use that instead.
  - atmanactive 1y ago
    As long as they are used, they are there.
- mokulanis 1y ago
  No i would not. I am using a RAM disk for compiling large projects, especially with a size of several terabytes, is a brilliant solution for your company. It totally makes sense given how compilation cycles can just chew through SSDs while HDDs are too slow.
  - atmanactive 1y ago
    That's a valid use case, thank you. Writing being the key here.
  - nolist_policy 1y ago
    I have found that temporary files are flushed much less by btrfs than ext4. ext4 will do a disk write io for every created file, even if it's only a temporary file. For btrfs, new files that are deleted before write back will never hit the disk.

2 points by mokulanis 1y ago

atmanactive 1y ago

I thought Linux buffering holds everything important in RAM anyway, no? If I had 1GiB of files served by Apache/Nginx and changed only once every 24 hours, then, stuffing it all in a RAM disk wouldn't make any difference on performance. Or would it?

compressedgas 1y ago
Yes, but there is no means to pin files and directories into memory so that they never get evicted from the file system cache. If there were, one could use that instead.
- atmanactive 1y ago
  As long as they are used, they are there.
mokulanis 1y ago
No i would not. I am using a RAM disk for compiling large projects, especially with a size of several terabytes, is a brilliant solution for your company. It totally makes sense given how compilation cycles can just chew through SSDs while HDDs are too slow.
- atmanactive 1y ago
  That's a valid use case, thank you. Writing being the key here.
- nolist_policy 1y ago
  I have found that temporary files are flushed much less by btrfs than ext4. ext4 will do a disk write io for every created file, even if it's only a temporary file. For btrfs, new files that are deleted before write back will never hit the disk.