3 ms·
> current 8k GPT-4 is already restricted to 25 prompts/3 hours I'm pretty sure they're using a 4k GPT-4 model for ChatGPT Plus, even though they only announced
by tempaccount420 3y ago
> current 8k GPT-4 is already restricted to 25 prompts/3 hours
I'm pretty sure they're using a 4k GPT-4 model for ChatGPT Plus, even though they only announced 8k and 32k... It can't handle more than 4k of tokens (actually a little below that, starts ignoring your last few sentences if you get close). If you check developer tools, the request to an API /models endpoint says the limit for GPT-4 is 4096. It's very unfortunate.
- reaperman 3y agoAh this explains a lot. I couldn't understand why I couldn't get close to the ~12 pages that everyone was saying 8,000 tokens implied.
- tempaccount420 3y agoAs far as I know it's not documented anywhere and there is no way to ask the team at ChatGPT questions. I sent them an email about it a few days after GPT-4 release and still haven't received a reply. Another thing that annoys me is how most updates don't get a changelog entry. For whatever reason, they keep little secrets like that.
- jiggawatts 3y agoTheir PR is terrible and I get the impression that they wish their own users would “just go away”. Every time I see a company act like this, more responsive and truly open competition eventually eats their lunch.
- int_19h 3y agoThe raw chat log has the system message on top, plus "user:" and "assistant:" for each message, and im_start/im_end tokens to separate messages, hence why the visible chat context is slightly under 4k.