4 ms·
how can I train a model on a bunch of pdf's and run it locally?
by bobwernstein1 4y ago
how can I train a model on a bunch of pdf's and run it locally?
- neil77450 4y agoRunning it fully locally isn't easy because you need a complex setup with GPUs. Can you give some more color on exactly what you are trying to do (number of PDFs, types of queries, etc.)? With some more info we can give you some guidance.
- MacsHeadroom 4y agooobabooga's textgen webui has a tab for fine tuning now. You only need a single consumer GPU to fine tune up to 33B parameter models at a rate of about 200 epochs per hour, per GPU. There are also one-click finetuning projects which run on free Google Colab GPUs like https://github.com/lxe/simple-llama-finetuner https://github.com/lxe/simple-llama-finetuner It's easy and not complex at all.
- MacsHeadroom 4y agoFine tuning a model like llama is super easy. You can fine tune models up to 33B parameters on a single consumer GPU at around 3 epochs per minute (less than 6 hours for 1000 epochs). You can even fine tune on a free Google Colab GPU. Start here: https://github.com/lxe/simple-llama-finetuner https://github.com/lxe/simple-llama-finetuner