There is (for many, but not all large models). Specifically there is huggingface's accelerate library that let's you run the model partially on your gpu, partia

There is (for many, but not all large models). Specifically there is huggingface's accelerate library that let's you run the model partially on your gpu, partially on cpu/ram and what doesn't fit in ram is cached in nvme storage (a mirror of two fast drives recommended). I didn't have much luck with stock accelerate, but once gpu is disabled (so it runs only on cpu offloading to nvme storage where ram is insufficient) worked pretty well with me. (there is a small code change that has to be done as the stock software refuses to run without gpu-it is a simple change described in its github issues). My gpu is 8gb vram, but this way I managed to run 7b parameter models. In principle I could run a lot larger ones, but of course it takes a lot more time. The 7b bloom takes 90s for one inference and additional 60s to load the model (from a spinning disc array) initially.