3 ms·
Until I can trust that when I send an AI agent off to do something that I will be successful without me babysitting and watching over it constantly AI won't tru
by yeck 3y ago
Until I can trust that when I send an AI agent off to do something that I will be successful without me babysitting and watching over it constantly AI won't truly be transformative (since the human bottleneck will remain).
This is one of the core promises of alignment. Without it how can there be trust? While there are probably short term slow downs with an alignment focus, ultimately it is necessary to avoid throwing darts in the dark.
- reissbaker 3y agoI wouldn't mind a focus on reliably following tasks with greater intelligence; what I think is negative utility is focusing more compute and research resources on hypothetical superintelligence alignment — the entire focus of Ilya's "Superalignment" project — when GPT-4 is still way, way sub-human-intelligence. For example, I don't think the GPT Store was in any way a dangerous idea, which seems to have been Ilya's claimed safety red line.
- yeck 3y agoI wouldn't call GPT-4 sub-human intelligence. While it's intelligence is less robust aggregate human intelligence, I don't think there is any one person alive who can compete with the breadth of GPT-4 knowledge. I also think that the potential of what currently is possible with existing models has not been fully realized. Good prompting strategies and reflection may already be able to produce a system that is effectively AGI. Might already exist in several labs.
- reissbaker 3y agoWikipedia has broader knowledge, and yet no one calls it intelligent. I'm talking about reasoning capability, and GPT-4 is well below human on complex tasks, especially multi-step tasks, which is why "autonomous agents" like AutoGPT, BabyAGI, etc are not yet very useful. For example: https://arxiv.org/abs/2311.09247 https://arxiv.org/abs/2311.09247