4 ms·
I did try, but got: ``` ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported. ```
by eurekin 4y ago
I did try, but got:
```
ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.
```
- superkuh 4y ago> Unfortunately there's a mismatch between the model generated by the delta patcher and the tokenizer (32001 vs 32000 tokens). There's a tool to fix this at llama-tools (https://github.com/Ronsor/llama-tools https://github.com/Ronsor/llama-tools). Add 1 token like (C controltoken), and then run the conversion script.
- DrSiemer 4y agoJust rename it in the tokenconfig.json
- eurekin 4y agoThanks, that indeed worked! This and using conda in wsl2, instead on bare windows