5 ms·
>If you're going to trash my /tmp with gigantic files please don't. Really I write statistical programs that often generate temporary intermediate files of mul
by RobAley 14y ago
>If you're going to trash my /tmp with gigantic files please don't. Really
I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session and my disks kept filling up, so I decided to use /tmp and let the OS manage the space.
Is there a better way to manage large tmp files than this that I'm missing?
- raverbashing 14y agoDepends on your use case and how much of that complexity you wish to manage In your case, what could work is having the new run use the same file names as the old run, or coming up with a rotation schedule (so you would keep the latest N runs) But if having them on /tmp as it is working for you, great, no need to complicate things. There are also speed considerations, if you're using these files as actual intermediate storage (as opposed to a log/debug dump) it may be interesting to have them written to a different disk (like an SSD)
- RobAley 14y agoIn my case, the runs are often run in parallel and in batches of varying sizes, so "same name" or rotation schedules wouldn't work well. They are a kind of hybrid log/storage file (they contain intermediate data sectioned/logged into the actions that created them, like an extremely verbose log). Dedicated SSDs would be very nice, but out of the price budget (i.e. its a zero cost project, run on existing scavanged hardware!). As you say, /tmp is working well for me, my example was to perhaps illustrate that there are (what I think are) legitimate uses for large files in /tmp.
- mpyne 14y agoGiven that they should persist, the better way would be /var/tmp I would think, at least if persistence means "should survive reboot". Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents of /tmp at startup (and I know Gentoo/OpenRC is). There might be an even more appropriate FHS-compliant directory (/var/run perhaps?)
- RobAley 14y agoApologies, by persist I meant just beyond the end of that run of the software (i.e. the software itself couldn't manage/delete the files), to see what went "wrong". Deleting at reboot is perfectly fine. /var/run "This directory contains system information data describing the system since it was booted" which doesn't seem to fit my use case that well. Not to say there isn't another directory that does fit, but I haven't found it.
- regularfry 14y agoSounds like a job for /var/cache.