3 ms·
No clue, but it wouldn't surprised me if they identified inputs and training that weren't actually helping. Just as human beings aren't necessarily more helpfu
by ravenstine 3y ago
No clue, but it wouldn't surprised me if they identified inputs and training that weren't actually helping. Just as human beings aren't necessarily more helpful by having more varied input, the same seems to apply to LLMs.
The interesting thing is it still has a 128k context length. This is awesome because GPT became way more useful to me once it reached this level of context.
- computerex 3y agoJust a caveat, this is 128k context length for ingestion, I believe the output is still constrained to just 4k tokens.
- ravenstine 3y agoYes, this is true. Nevertheless, that made a huge difference for my workflow. I haven't had to think about exceeding the total context in a long time.