3 ms·
Running it fully locally isn't easy because you need a complex setup with GPUs. Can you give some more color on exactly what you are trying to do (number of PDF
by neil77450 4y ago
Running it fully locally isn't easy because you need a complex setup with GPUs. Can you give some more color on exactly what you are trying to do (number of PDFs, types of queries, etc.)? With some more info we can give you some guidance.
- MacsHeadroom 4y agooobabooga's textgen webui has a tab for fine tuning now. You only need a single consumer GPU to fine tune up to 33B parameter models at a rate of about 200 epochs per hour, per GPU. There are also one-click finetuning projects which run on free Google Colab GPUs like https://github.com/lxe/simple-llama-finetuner https://github.com/lxe/simple-llama-finetuner It's easy and not complex at all.