3 ms·
> In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it. It's a token prediction machine. We've already generated m
by troupo 2y ago
> In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it.
It's a token prediction machine. We've already generated most of the ideas for it, and hardly any of them work because see below
> Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.
No. Gettin AI to work you need to make an AI, and not a token prediction machine which, however wonderful:
- does not understand what it is it's generating, and approaches generating code the same way it approaches generating haikus
- hallucinates and generates invalid data
> Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"
Indeed. Instead of asking why, you're wildly fantasizing about running out of ideas and pretending you can make this work through other means of communication.
- SrslyJosh 2y ago> hallucinates and generates invalid data The model is doing the exact same thing when it generates "correct" output as it does when it generates "incorrect" output. "Hallucination" is a misleading term, cooked up by people who either don't understand what's going on or who want to make it sound like the fundamental problems (models aren't intelligent, can't reason, and attach zero meaning to their input or output) can be solved with enough duct tape.
- Terr_ 2y agoI like the analogy of an Ouija board: "This magical conduit to supernatural forces sometimes gives out nonsense, but soon we'll fix it so that it always channels the correct ghosts and spirits."
- RogerL 2y agoIt was "cooked up" by people like Joshua Maynez, Shashi Narayan, et al, who I'm going to guess understand what is going on, and adopted by the rest of the field. https://aclanthology.org/2020.acl-main.173/ https://aclanthology.org/2020.acl-main.173/
- troupo 2y agoI like the term because it's concise and describes the output :) But true, it's just the output.