3 ms·
FWIW, the 128k context window for GPT-4 is only for input. I believe the output content is still only 4k.
by pseudosavant 3y ago
FWIW, the 128k context window for GPT-4 is only for input. I believe the output content is still only 4k.
- moffkalast 3y agoHow does that make any sense on a decoder-only architecture?
- grey8 3y agoIt's not about the model. The model can output more - it's about the API. A better phrasing would be that they don't allow you to output more than 4k tokens per message. Same with Anthropic and Claude, sadly.
- ljhskyso 3y agoyou can always put the unfinished output as the input to continue forever until reaching the full 128k context window