4 ms·
How do you deal with context length degradation here? The harder questions will only arrive when the context is getting full.
by OtherShrezzing 4mo ago
How do you deal with context length degradation here?
The harder questions will only arrive when the context is getting full.
- danieljacksonno 4mo agoIf you have it question you for 1M tokens (aka the full length of the Wheel of Time series), I think your own context might get full before the LLMs.
- d4rkp4ttern 4mo agoRight, even a conservative 200k context length is on the order of 200 pages, which is more than enough context to arrive at an answer.
- mkl 4mo agoThe Wheel of Time is 4.3M words: https://www.reddit.com/r/WoT/comments/zs1ust/chart_of_wot_word_counts_books_and_chapters/ https://www.reddit.com/r/WoT/comments/zs1ust/chart_of_wot_wo.... Some of its individual books are near 400k.
- raincole 4mo agoIf it's a well-known concept (like pretty much anything you can find from undergraduate textbook), the LLM doesn't need the whole context to teach you. If it's something actually novel, no matter how much context you provide it'll still hallucinate.