3 ms·
1. turn the swap off or monitor it closely 2. try to load a big model, like 65b-q4 or 30b-f16 3. observe the OOM - It's not so hard to test this.
by dislikedtom2 4y ago
1. turn the swap off or monitor it closely
2. try to load a big model, like 65b-q4 or 30b-f16
3. observe the OOM
- It's not so hard to test this.
- gg82 4y agoUsing a memory mapped file doesn't use swap. The memory is backed by the file that is memory mapped!