3 ms·

You are right, but it is unclear to me where this limitation will fall in terms of actual abilities. GPT-4 can divide large tasks up into logical steps and ins

by _yb2s 4y ago

You are right, but it is unclear to me where this limitation will fall in terms of actual abilities. GPT-4 can divide large tasks up into logical steps and instructions that can be worked on independently by other instances, and it can create "compressed" condensed explanations of it's own state that can be stored externally and repeated back to it, or passed back and forth between instances. With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on.

IanCal 4y ago
> With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on. Also that means you can just iterate with different contexts until you can deal with the problem. How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?
- Workaccount2 4y ago
  >How many problems need 50 pages of context on top of what gpt4 knows for solving the next step? Perhaps the unknown unkowns? I envision a scenario where a super intelligent AI seemingly runs off the rails and becomes obsessed with overtly complex problems that are totally intractable to humans. Where we'd be like ants trying to make sense of cell phones.
  - jules 4y ago
    Maybe it can do this with sleep, where it gets further training on the new input and output it produced that day.
Sinidir 4y ago
That just gave me the idea that there could be a main instance that keeps a summary of the problem (List of paragraphs for each subproblem) and other instances that hold the larger text that these summaries are based on. Then on demand the main instance could get needed extra information for a subpoint back. Kinda like having forgein keys in a databas with which you can join in the relevant row!

by _yb2s 4y ago

IanCal 4y ago

> With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on. Also that means you can just iterate with different contexts until you can deal with the problem. How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?

Workaccount2 4y ago
>How many problems need 50 pages of context on top of what gpt4 knows for solving the next step? Perhaps the unknown unkowns? I envision a scenario where a super intelligent AI seemingly runs off the rails and becomes obsessed with overtly complex problems that are totally intractable to humans. Where we'd be like ants trying to make sense of cell phones.
- jules 4y ago
  Maybe it can do this with sleep, where it gets further training on the new input and output it produced that day.

Sinidir 4y ago

That just gave me the idea that there could be a main instance that keeps a summary of the problem (List of paragraphs for each subproblem) and other instances that hold the larger text that these summaries are based on. Then on demand the main instance could get needed extra information for a subpoint back. Kinda like having forgein keys in a databas with which you can join in the relevant row!