4 ms·
It depends on what you're doing... Just for reference, here is a small showcase of the capabilities that I've trained on a 13 billion parameter llama2 fine tune
by Tostino 3y ago
It depends on what you're doing... Just for reference, here is a small showcase of the capabilities that I've trained on a 13 billion parameter llama2 fine tune (done with qlora).
https://old.reddit.com/r/LocalLLaMA/comments/186qq92/comment/kba2d6m/?utm_source=share&utm_medium=web2x&context=3 https://old.reddit.com/r/LocalLLaMA/comments/186qq92/comment...
Edit: Embed some of the content instead.
Inkbot can create knowledge graphs. The structure returned is proper YAML, and I got much better results with my fine-tune than using GPT4.
https://huggingface.co/Tostino/Inkbot-13B-8k-0.2 https://huggingface.co/Tostino/Inkbot-13B-8k-0.2
Simple prompt: https://gist.github.com/Tostino/c3541f3a01d420e771f66c62014e6a24 https://gist.github.com/Tostino/c3541f3a01d420e771f66c62014e...
Complex prompt: https://gist.github.com/Tostino/44bbc6a6321df5df23ba5b400a01e37d https://gist.github.com/Tostino/44bbc6a6321df5df23ba5b400a01...
It also does chunked summarization.
Here is an example of chunking:
Part 1: chunked summarization - https://gist.github.com/Tostino/cacb1cecdf2eb7386baf565d157f56a0 https://gist.github.com/Tostino/cacb1cecdf2eb7386baf565d157f...
Part 2: summary-of-summaries - https://gist.github.com/Tostino/81eeee9781e519044950332b4e64bef1 https://gist.github.com/Tostino/81eeee9781e519044950332b4e64...
Here is an example of a single-shot document that fits entirely within context: https://gist.github.com/Tostino/4ba4e7e7988348134a7256fd1cbbf4ff https://gist.github.com/Tostino/4ba4e7e7988348134a7256fd1cbb...
- cced 3y agoHave any references on how you fine tuned?
- Tostino 3y agoSure thing, I used axolotl, and my training parameters were: sequence_len: 6144 lora_r: 128 lora_alpha: 48 learning_rate: 0.00006 warmup_steps: 600 lr_scheduler: cosine gradient_accumulation_steps: 4 micro_batch_size: 1 num_epochs: 4 optimizer: paged_adamw_32bit flash_attention: true sample_packing: true
- kaycebasques 3y agoThanks for sharing all these details throughout the thread, Tostino. True open source spirit.
- reactive001 3y agoAmazing work, I've really wanted to get into knowledge graph generation with LLM's for the last year but haven't found the time. Glad to see someone making good progress on the idea! How are you going about generating training data?
- Tostino 3y agoLots and lots of manual review of very detailed instructions to "more powerful LLMs" with 2-4 prompts to generate the training data.
- thecal 3y agoI really like Inkbot! Are you working on a new version? How about one from Yi 34B?
- Tostino 3y agoYeah, I will be soon. I was busy adding `chat template` support to vLLM recently, so the model (and any others that implement it properly) will work seamlessly with a clone of the OpenAI chat/completions endpoint. https://github.com/vllm-project/vllm/pull/1756 https://github.com/vllm-project/vllm/pull/1756 Now that I have that out of the way, back to model training ;).
- thecal 3y agoVery cool, looking forward to it!
- anon373839 3y agoThis looks really impressive. Any chance a 7B Inkbot is in the works?
- Tostino 3y agoYeah, i'll be training a Mistral variant soon (and some of the larger, newer models as well). I had a few dataset issues I know about that I wanted to fix first.
- anon373839 3y agoThat’s great to hear. Thanks!