4 ms·
If anyone is interested in running it on windows and a GPU (30b 4bit fits in a 3090), here's a guide: https://gist.github.com/lxe/82eb87db25fdb75b92fa18a6d494ee
by lxe 4y ago
If anyone is interested in running it on windows and a GPU (30b 4bit fits in a 3090), here's a guide: https://gist.github.com/lxe/82eb87db25fdb75b92fa18a6d494ee3c https://gist.github.com/lxe/82eb87db25fdb75b92fa18a6d494ee3c