2 ms·
Sounds like you're hitting rate limiting for tokens per minute. I'd suggest working on reducing the size of your prompt, which will also result in more accurate
by phillipcarter 3y ago
Sounds like you're hitting rate limiting for tokens per minute. I'd suggest working on reducing the size of your prompt, which will also result in more accurate responses (since LLMs do poorly when prompts get past a certain size).