2 ms·
"not exposing the probability distribution per generated token" can you elaborate on what that means?
by meghan_rain 4y ago
"not exposing the probability distribution per generated token"
can you elaborate on what that means?
- throwaway1851 4y agoA language model takes in a sequence of tokens and outputs a probability (0-1) for each token in the vocabulary (the set of all tokens the model knows). Based on this probability distribution, there are various sampling strategies that can be employed to choose which token to actually show to the user.
- wskish 4y agoOpenAI's previous completion endpoint for the davinci-003 and older models included a "logprob" return option: https://platform.openai.com/docs/api-reference/completions/create#completions/create-logprobs https://platform.openai.com/docs/api-reference/completions/c... Their newer chat style endpoint for the GPT-3.5-turbo and GPT-4 models no longer supports this. https://platform.openai.com/docs/api-reference/chat https://platform.openai.com/docs/api-reference/chat