3 ms·

"Isn't it just temporary, and AI will get better?" There are 2 instances of the word "understand" in the first paragraph, 3 if you count the beginning of the s

"Isn't it just temporary, and AI will get better?" There are 2 instances of the word "understand" in the first paragraph, 3 if you count the beginning of the second. In my book, "understanding" is a synonym for "intelligence" - the roots of the word are "read between the lines", where something else that just knowledge is, the ability to use and manipulate knowledge [1]. But the thing is, despite this tech being classified as "artificial intelligence", it does not understand a thing - or so little. So, if we extrapolate Betteridge's law of headlines, no it is not temporary for this type of technology. But I think connecting it with formal computations - inference engines for formal logic, calculators [2], etc. could be amazing. [1] https://en.wiktionary.org/wiki/intelligence https://en.wiktionary.org/wiki/intelligence - well, yes, that's a bit cherry-picked. [2] https://arxiv.org/abs/2406.03445 https://arxiv.org/abs/2406.03445 - amazing result, and maybe will find out that it is exactly what we do "under the skull", but doing arithmetic with Fourier transforms is not the best use of a microprocessor.

ramoz 1y ago
The underlying model architectures have issues that perpetuate as models get better. We're still using transformers and RL. * Optimized for task completion, with limited attention resources for global alignment (RL/RLAIF reward loops/hacking) * These systems run outside of chat now. file systems, CLIs, DBs, browsers → real-world side‑effects that you cannot train for. Hallucination becomes a problem of contradiction in the real world, and alignment is something an agent will struggle with as it's optimized to complete tasks. Which is why you see things like databases being dropped today. * These are baked-in problems, not even considering the the adversarial nuances of things like prompt injection. As AI advances, so do these issues. Maybe it's cliche from an AI safety perspective. But I can never get over https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence as we see micro instances of it in our day-to-day with today's agents. Again, an issue that has existed from the dawn of these types of models. https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s

ramoz 1y ago

The underlying model architectures have issues that perpetuate as models get better. We're still using transformers and RL. * Optimized for task completion, with limited attention resources for global alignment (RL/RLAIF reward loops/hacking) * These systems run outside of chat now. file systems, CLIs, DBs, browsers → real-world side‑effects that you cannot train for. Hallucination becomes a problem of contradiction in the real world, and alignment is something an agent will struggle with as it's optimized to complete tasks. Which is why you see things like databases being dropped today. * These are baked-in problems, not even considering the the adversarial nuances of things like prompt injection. As AI advances, so do these issues. Maybe it's cliche from an AI safety perspective. But I can never get over https://en.wikipedia.org/wiki/Instrumental_convergence https://en.wikipedia.org/wiki/Instrumental_convergence as we see micro instances of it in our day-to-day with today's agents. Again, an issue that has existed from the dawn of these types of models. https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s