3 ms·
Can't I use the max_tokens (set to 1) and logit_bias parameters? Not saying I want to do this. I just want to understand how this works.
by Kiro 3y ago
Can't I use the max_tokens (set to 1) and logit_bias parameters? Not saying I want to do this. I just want to understand how this works.
- baobabKoodaa 3y agoNot sure exactly what is logit_bias, but after Googling for 5 seconds it seems to be an OpenAI parameter that's not available in HuggingFace transformers? Anyway, if your idea is to make one API call per token, the biggest problem with that approach is that it would be really slow to do that.