4 ms·
> With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it
by IanCal 4y ago
> With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on.
Also that means you can just iterate with different contexts until you can deal with the problem. How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?
- Workaccount2 4y ago>How many problems need 50 pages of context on top of what gpt4 knows for solving the next step? Perhaps the unknown unkowns? I envision a scenario where a super intelligent AI seemingly runs off the rails and becomes obsessed with overtly complex problems that are totally intractable to humans. Where we'd be like ants trying to make sense of cell phones.
- jules 4y agoMaybe it can do this with sleep, where it gets further training on the new input and output it produced that day.