3 ms·
The 7GB runs great on a 3080Ti. I am getting a lot of 'ValueError: Max tokens + prompt length' errors with larger files. Can this Gitlab client also replace the
by daviding 4y ago
The 7GB runs great on a 3080Ti. I am getting a lot of 'ValueError: Max tokens + prompt length' errors with larger files. Can this Gitlab client also replace the vocab.bpe and tokenizer.json config like Copilots? Thanks for your work on Fauxpilot, really enjoying playing with it.
- moyix 4y agoI believe right now the VSCode extension just passes along the entire file up to your cursor [1] rather than trying to figure out how much will fit into the context limit – it's definitely still very early stages :) It would be pretty simple to run the contents through the tokenizer using e.g. this JS lib that wraps Huggingface Tokenizers [2] and then keep only the last (2048-requested_tokens) tokens in the prompt. If they don't get to it first I may try to throw this together soon. [1] https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/blob/main/src/completion/gitlab_code_completion_provider.ts#L57 https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/blob... [2] https://www.npmjs.com/package/tokenizers https://www.npmjs.com/package/tokenizers
- daviding 4y agoUnderstood - thanks again. (plus note to self; have individual source files with less in them ;)