4 ms·

If you define AGI as "can do the work of a human sitting at a computer, end to end", then I'd say comparing yourself to it on a specific skill is the wrong test

by ggsp 1mo ago

If you define AGI as "can do the work of a human sitting at a computer, end to end", then I'd say comparing yourself to it on a specific skill is the wrong test. Can you hand it a role and walk away for a day/week/month? I can’t yet. I think that I'd want at least two things it doesn't have: the ability to retain what it learned yesterday (without me carrying it in the context window and thus micromanaging it), and the ability to prioritize correctly, i.e. tell which of the n things it could do next is the one that actually matters. Can't say for sure that those are enough, but not having them seems to be most of why I still have to "babysit" these incredible tools.

a2ff6eeb0 1mo ago
I agree, we're not at that kind of long horizon capability yet. You still need a human in the loop to do manual testing. For whatever reason, we managed to automate the skill before we managed to automate the focus. So, for now, humans need to stay in the loop and do low skilled labor to keep the skilled work the models do from going off the rails.
impjohn 1mo ago
To be fair, I think you can't hand a role over to someone you just hired and walk away for a week. No matter how much of a SME they are. Let's not forget human onboarding takes months. With the advantage of their knowledge not going into the void multiple times a day. That's likely the last missing piece, a solid system of memories that produces the same effect as short/long term memories in a person.
mostertoaster 1mo ago
I think fundamentally it is that. The ability to retain information. Like given a specific task it can do a thing amazingly well, but can it recall a thing. Its memory seems like a giant filing cabinet and it has to go scan like 20 million tokens worth of memory to recover things previously talked about. Human memory is more graph like, we don’t recall things exactly, but one thing links to another, we create a pattern of a thing, we mark what is important, and overtime what was important degrades or becomes less so. I feel like what makes it lack intelligence is it never seems to learn. Like it kind of does, but then doesn’t persist once too many other things are learned. I’m sure they’re probably working on this, but I feel like that is what I want far more than even better models, is a better memory system to recall and forget things that the models work on.

by ggsp 1mo ago

a2ff6eeb0 1mo ago

I agree, we're not at that kind of long horizon capability yet. You still need a human in the loop to do manual testing. For whatever reason, we managed to automate the skill before we managed to automate the focus. So, for now, humans need to stay in the loop and do low skilled labor to keep the skilled work the models do from going off the rails.

impjohn 1mo ago

To be fair, I think you can't hand a role over to someone you just hired and walk away for a week. No matter how much of a SME they are. Let's not forget human onboarding takes months. With the advantage of their knowledge not going into the void multiple times a day. That's likely the last missing piece, a solid system of memories that produces the same effect as short/long term memories in a person.

mostertoaster 1mo ago

I think fundamentally it is that. The ability to retain information. Like given a specific task it can do a thing amazingly well, but can it recall a thing. Its memory seems like a giant filing cabinet and it has to go scan like 20 million tokens worth of memory to recover things previously talked about. Human memory is more graph like, we don’t recall things exactly, but one thing links to another, we create a pattern of a thing, we mark what is important, and overtime what was important degrades or becomes less so. I feel like what makes it lack intelligence is it never seems to learn. Like it kind of does, but then doesn’t persist once too many other things are learned. I’m sure they’re probably working on this, but I feel like that is what I want far more than even better models, is a better memory system to recall and forget things that the models work on.