4 ms·

In addition to `/tmp`, you can put your `~/.cache` on `tmpfs`. Laptops have ridiculously large amounts of RAM nowadays. My Linux laptop setup barely makes a d

by neilv 2y ago

In addition to `/tmp`, you can put your `~/.cache` on `tmpfs`. Laptops have ridiculously large amounts of RAM nowadays. My Linux laptop setup barely makes a dent in the RAM, even when running 3 different Web browsers and other gluttonous desktop programs. `tmpfs` is a great use for excess RAM, reducing wear on SSD. (I also disable swap.) I've also done things like build an entire large ecosystem of packages in `tmpfs`, when the build server happened to have mirrored 10krpms drives, and I didn't want the tons of intermediate files to eventually be synced to disk. (Even though, with disk, they would also probably hang around in Linux filesystem buffers, not reads hitting disk each time.)

deleted 2y ago
[deleted]
zbowling 2y ago
This would break a bunch of little things in annoying ways. Like I have shell script tools that store shell scrollback logs in .cache, and I want that past reboot.
- neilv 2y ago
  FWIW, I've been doing it for many years, and not noticed any problems. If something is supposed to persist past reboot, I wouldn't put it in `.cache`. Though I don't know offhand what the official documented behavior of `.cache` is, and I can't immediately find that documentation (maybe some open desktop cabal thing?).
  - mhitza 2y ago
    XDG_CACHE_HOME same text across bott resources https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html https://specifications.freedesktop.org/basedir-spec/basedir-... or https://wiki.archlinux.org/title/XDG_Base_Directory https://wiki.archlinux.org/title/XDG_Base_Directory > $XDG_CACHE_HOME defines the base directory relative to which user-specific non-essential data files should be stored. If $XDG_CACHE_HOME is either not set or empty, a default equal to $HOME/.cache should be used.
  - viraptor 2y ago
    That's silly. Cache is there for preserving things across many runs of some application. Applications certainly use it in a way that assumes long term storage. It's not for short term temporary things. It's a cache. It's not going to cause "problems". It's just going to massively slow down many use cases that rely on downloads.
  - arp242 2y ago
    It's fine to put ~/.cache on tmpfs, but doing it by default for the general case is going to cause a lot of hurt. My ~/.cache could be rm -rf'd without too much worry right now, but that doesn't mean that persisting it isn't useful. For example on my system right now: - Browser cache is useful to persist, especially with some larger sites. - I put my Go module and build cache in ~/.cache, and while that can be deleted it's useful to persist because it make builds shorter, and avoids having to (re)-download the same modules over and over again. Note that at the moment my internet is kind of crappy so this can take quite a while. - Some other download cache things in there, from luarocks, xlocate, few other things. - I store psql history per-database in ~/.cache/psql-dbname. It's useful to keep this around. - Vim backup files, persistent undo files, and swap files are stored in ~/.vim. The swap files especially are important because I want to keep them after an unexpected system crash. Some of this is solvable by moving stuff to other directories. Others are inherently unsolvable. I also have just 8G of RAM, which is fine but not fine for storing ~/.cache in RAM.
elpocko 2y ago
If my ~/.cache was on tmpfs, I would need to download tens of gigabytes of stuff every day. I wouldn't be able to work. There are lots of huge files stored in there, mostly required by Python packages for Stable Diffusion and other ML stuff that insists on downloading huge model files and putting them into ~/.cache. Edit: du -hc ~/.cache says it's 25G.
- person4268 2y ago
  You can set HF_HOME to move the folder it downloads those to elsewhere (or well, the HF transformers/diffusers stuff at least), iirc.
- neilv 2y ago
  Agreed, if you're storing 25GB+ in `~/.cache`, then probably you won't put it on `tmpfs` on a laptop. I didn't think of the massive ML models, because I store those in the home directory, where I can see them. (Incidentally, since you mention Stable Diffusion: for an earlier version of SD, I went to some care to make sure I didn't accidentally lose the checkpoint file, because I didn't know whether it would be pulled from distribution. Then there were regressions in the SD training data.)
  - elpocko 2y ago
    >I store those in the home directory, where I can see them. That works for most things, and that's how it should be. But sometimes the dev just doesn't provide links and you either have to read their code to figure out the model URLs, or simply allow the automatic download that ends up in .cache.
exe34 2y ago
how do you hibernate without swap? or do you just sleep?
- pantalaimon 2y ago
  Standby is usually enough and much faster anyway
tjoff 2y ago
No. Thinner laptops often max out at 32 or even (!) 16GB. And defaults at incomprehensible 8GB.
mort96 2y ago
8GiB is still very common, and it's not hard to find cheaper laptops with 4GiB... I don't think your average person's laptop has as ridiculously large amounts of RAM as you seem to think
- Aurornis 2y ago
  > I don't think your average person's laptop has as ridiculously large amounts of RAM as you seem to think Your average person isn’t running Linux or compiling things. The comment was in the context of developer laptops.
  - pantalaimon 2y ago
    Not every developer always has the biggest and latest laptop.
  - arp242 2y ago
    I run Linux and have 8G and compile things. Works fine. Currently ~/.cache is 2G, but that's because I cleared out ~/.cache/go just yesterday after compiling ~1000 Go modules for some testing, which bloated it to >50G.
    - nottorp 2y ago
      What kills your ram is not compiling things but the browser and those Electron apps. (Well and Yocto builds, but that's not your average 'compiling things'.)
  - mort96 2y ago
    Fair. I read the comment as, more or less, "the typical laptop has so much RAM these days that putting ~/.cache on tmpfs would be a reasonable default" or something like that. But I guess it could -- and should -- be read as, "if you spend a bunch extra on RAM, you can get laptops with so much RAM these days that you can consider putting ~/.cache on tmpfs", which is reasonable enough.
binkHN 2y ago
I do have a notebook with plenty of RAM, so I haven't bothered with swap, but I'm new to Linux. Without swap configured, what happens when tmpfs uses all the memory that's been allocated to it? Thanks.

by neilv 2y ago

deleted 2y ago

[deleted]

zbowling 2y ago

This would break a bunch of little things in annoying ways. Like I have shell script tools that store shell scrollback logs in .cache, and I want that past reboot.

neilv 2y ago
FWIW, I've been doing it for many years, and not noticed any problems. If something is supposed to persist past reboot, I wouldn't put it in `.cache`. Though I don't know offhand what the official documented behavior of `.cache` is, and I can't immediately find that documentation (maybe some open desktop cabal thing?).
- mhitza 2y ago
  XDG_CACHE_HOME same text across bott resources https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html https://specifications.freedesktop.org/basedir-spec/basedir-... or https://wiki.archlinux.org/title/XDG_Base_Directory https://wiki.archlinux.org/title/XDG_Base_Directory > $XDG_CACHE_HOME defines the base directory relative to which user-specific non-essential data files should be stored. If $XDG_CACHE_HOME is either not set or empty, a default equal to $HOME/.cache should be used.
- viraptor 2y ago
  That's silly. Cache is there for preserving things across many runs of some application. Applications certainly use it in a way that assumes long term storage. It's not for short term temporary things. It's a cache. It's not going to cause "problems". It's just going to massively slow down many use cases that rely on downloads.
- arp242 2y ago
  It's fine to put ~/.cache on tmpfs, but doing it by default for the general case is going to cause a lot of hurt. My ~/.cache could be rm -rf'd without too much worry right now, but that doesn't mean that persisting it isn't useful. For example on my system right now: - Browser cache is useful to persist, especially with some larger sites. - I put my Go module and build cache in ~/.cache, and while that can be deleted it's useful to persist because it make builds shorter, and avoids having to (re)-download the same modules over and over again. Note that at the moment my internet is kind of crappy so this can take quite a while. - Some other download cache things in there, from luarocks, xlocate, few other things. - I store psql history per-database in ~/.cache/psql-dbname. It's useful to keep this around. - Vim backup files, persistent undo files, and swap files are stored in ~/.vim. The swap files especially are important because I want to keep them after an unexpected system crash. Some of this is solvable by moving stuff to other directories. Others are inherently unsolvable. I also have just 8G of RAM, which is fine but not fine for storing ~/.cache in RAM.

elpocko 2y ago

If my ~/.cache was on tmpfs, I would need to download tens of gigabytes of stuff every day. I wouldn't be able to work. There are lots of huge files stored in there, mostly required by Python packages for Stable Diffusion and other ML stuff that insists on downloading huge model files and putting them into ~/.cache. Edit: du -hc ~/.cache says it's 25G.

person4268 2y ago
You can set HF_HOME to move the folder it downloads those to elsewhere (or well, the HF transformers/diffusers stuff at least), iirc.
neilv 2y ago
Agreed, if you're storing 25GB+ in `~/.cache`, then probably you won't put it on `tmpfs` on a laptop. I didn't think of the massive ML models, because I store those in the home directory, where I can see them. (Incidentally, since you mention Stable Diffusion: for an earlier version of SD, I went to some care to make sure I didn't accidentally lose the checkpoint file, because I didn't know whether it would be pulled from distribution. Then there were regressions in the SD training data.)
- elpocko 2y ago
  >I store those in the home directory, where I can see them. That works for most things, and that's how it should be. But sometimes the dev just doesn't provide links and you either have to read their code to figure out the model URLs, or simply allow the automatic download that ends up in .cache.

exe34 2y ago

how do you hibernate without swap? or do you just sleep?

pantalaimon 2y ago
Standby is usually enough and much faster anyway

tjoff 2y ago

No. Thinner laptops often max out at 32 or even (!) 16GB. And defaults at incomprehensible 8GB.

mort96 2y ago

8GiB is still very common, and it's not hard to find cheaper laptops with 4GiB... I don't think your average person's laptop has as ridiculously large amounts of RAM as you seem to think

Aurornis 2y ago
> I don't think your average person's laptop has as ridiculously large amounts of RAM as you seem to think Your average person isn’t running Linux or compiling things. The comment was in the context of developer laptops.
- pantalaimon 2y ago
  Not every developer always has the biggest and latest laptop.
- arp242 2y ago
  I run Linux and have 8G and compile things. Works fine. Currently ~/.cache is 2G, but that's because I cleared out ~/.cache/go just yesterday after compiling ~1000 Go modules for some testing, which bloated it to >50G.
  - nottorp 2y ago
    What kills your ram is not compiling things but the browser and those Electron apps. (Well and Yocto builds, but that's not your average 'compiling things'.)
- mort96 2y ago
  Fair. I read the comment as, more or less, "the typical laptop has so much RAM these days that putting ~/.cache on tmpfs would be a reasonable default" or something like that. But I guess it could -- and should -- be read as, "if you spend a bunch extra on RAM, you can get laptops with so much RAM these days that you can consider putting ~/.cache on tmpfs", which is reasonable enough.

binkHN 2y ago

I do have a notebook with plenty of RAM, so I haven't bothered with swap, but I'm new to Linux. Without swap configured, what happens when tmpfs uses all the memory that's been allocated to it? Thanks.