3 ms·

It really stands to reason that at some point these systems will just be "too smart" and we won't find any value in something that much more intelligent. It's l

It really stands to reason that at some point these systems will just be "too smart" and we won't find any value in something that much more intelligent. It's like a 4 year old trying to understand a seasoned professor explaining some concept, we just won't get it. Imagine a hypothetical scenario where some ultra-intelligent beings delivered a book with all the answers humans ever wanted, but we needed an IQ of 500 t understand it. This IMO is where we're heading. There might be some pretty sweet low hanging fruit to attain along the way, but at some stage, we'll hit a ceiling and the only way forward would be to augment our own intelligence.

throwaway22032 3y ago
That's why alignment is such a huge issue - the immediately obvious and tempting way to "solve" that problem is to give the machines control. The child-parent analogy stretches to describe it. A young child has an extremely fuzzy picture about what is going on - how did this house get here, why is there food in the fridge, why is the food wrapped in this clear thing, why does Dad disappear for a few hours every day, etc. At a young enough age they're not even thinking about it. Yet it all happens anyway and the child is provided for. Even in the best case of perfectly aligned AI, that's our future if we allow it to happen (I hope we don't...)
jiggawatts 3y ago
From what I've seen, the increases in LLM intelligence is not that type of smart. On the contrary, the better LLMs write in a more neutral and clear tone, a bit like a Wikipedia article. Actually, Wikipedia articles are already sort-of a "distillation of the Internet" into a type of average. By having many people edit them, the individual personal quirks are smoothed out, leaving a type of average-of-humans result. This reminds me of the "average faces" project about a decade ago where they blended photos of random people together. The results were often very attractive, because it's the small flaws that make normal people unattractive. The blending of many samples smooths out the flaws, resulting in a "perfect" but ordinary face. Not some post-human face, but a perfectly human face. That's what I see LLMs doing: as smart as us, but no smarter. Great breadth certainly, but not greater depth.

throwaway22032 3y ago

That's why alignment is such a huge issue - the immediately obvious and tempting way to "solve" that problem is to give the machines control. The child-parent analogy stretches to describe it. A young child has an extremely fuzzy picture about what is going on - how did this house get here, why is there food in the fridge, why is the food wrapped in this clear thing, why does Dad disappear for a few hours every day, etc. At a young enough age they're not even thinking about it. Yet it all happens anyway and the child is provided for. Even in the best case of perfectly aligned AI, that's our future if we allow it to happen (I hope we don't...)

jiggawatts 3y ago

From what I've seen, the increases in LLM intelligence is not that type of smart. On the contrary, the better LLMs write in a more neutral and clear tone, a bit like a Wikipedia article. Actually, Wikipedia articles are already sort-of a "distillation of the Internet" into a type of average. By having many people edit them, the individual personal quirks are smoothed out, leaving a type of average-of-humans result. This reminds me of the "average faces" project about a decade ago where they blended photos of random people together. The results were often very attractive, because it's the small flaws that make normal people unattractive. The blending of many samples smooths out the flaws, resulting in a "perfect" but ordinary face. Not some post-human face, but a perfectly human face. That's what I see LLMs doing: as smart as us, but no smarter. Great breadth certainly, but not greater depth.