4 ms·

20-some years ago, I had this "20 questions" handheld electronic game that was eerily good at winning. I imagine it was a bunch of well-programmed tables of dat

by tjr 3y ago

20-some years ago, I had this "20 questions" handheld electronic game that was eerily good at winning. I imagine it was a bunch of well-programmed tables of data, but in any case, it's certainly possible for a machine to do well at this game. I think the more we see ChatGPT do things like "oh, I know this game -- I'm going to run a 20-year-old 20 Questions subroutine that is not part of my neural network language model to generate responses", it will become even more impressive.

smolder 3y ago
Yeah, ChatGPT could integrate Akinator[0] and trivially be great at the game. Without the help, though, It's a good, revealing benchmark for the LLMs ability. [0] https://en.akinator.com https://en.akinator.com
helen___keller 3y ago
> I think the more we see ChatGPT do things like "oh, I know this game -- I'm going to run a 20-year-old 20 Questions subroutine that is not part of my neural network language model to generate responses", it will become even more impressive. Agreed. Incidentally I’ve built a little toy version of a runtime for exactly this purpose - there’s a translation layer that’s given a bunch of available “APIs” (fed through the LLM context), and breaks down a high level goal into a structured series of API calls. the runtime parses these API calls, and natively executes some (e.g. run a program, write to the file system) and others result in LLM invocations. I’m sure OpenAI and crew are way ahead of me here, of course. I’m excited to see what the future holds in this field.
nr2x 3y ago
LLM for the foreseeable future function most reliably as a user interface layer for other system. I use GPT to “translate” natural language down into the API calls that get real data and it works great. I’d never trust it beyond that.
- marcosdumay 3y ago
  You trained it with "this phrase means this command" examples? How do you make it use your custom API? (Or you are not using your custom API?)
  - nr2x 3y ago
    Basically yeah, just a pretty detailed set of prompts and then “turn the next message into an api call” and it basically works perfectly. When I first heard the term “prompt engineer” I rolled my eyes, but now that I’ve gotten into it I see it’s really an art form.
JohnFen 3y ago
The first AI-style program I ever wrote (about 25 years ago. Yes, I'm old) played 20 questions, but it would "learn" from prior games, so the more you played, the better it performed. It got extremely good after a few hundred games.

by tjr 3y ago

smolder 3y ago

Yeah, ChatGPT could integrate Akinator[0] and trivially be great at the game. Without the help, though, It's a good, revealing benchmark for the LLMs ability. [0] https://en.akinator.com https://en.akinator.com

helen___keller 3y ago

> I think the more we see ChatGPT do things like "oh, I know this game -- I'm going to run a 20-year-old 20 Questions subroutine that is not part of my neural network language model to generate responses", it will become even more impressive. Agreed. Incidentally I’ve built a little toy version of a runtime for exactly this purpose - there’s a translation layer that’s given a bunch of available “APIs” (fed through the LLM context), and breaks down a high level goal into a structured series of API calls. the runtime parses these API calls, and natively executes some (e.g. run a program, write to the file system) and others result in LLM invocations. I’m sure OpenAI and crew are way ahead of me here, of course. I’m excited to see what the future holds in this field.

nr2x 3y ago

LLM for the foreseeable future function most reliably as a user interface layer for other system. I use GPT to “translate” natural language down into the API calls that get real data and it works great. I’d never trust it beyond that.

marcosdumay 3y ago
You trained it with "this phrase means this command" examples? How do you make it use your custom API? (Or you are not using your custom API?)
- nr2x 3y ago
  Basically yeah, just a pretty detailed set of prompts and then “turn the next message into an api call” and it basically works perfectly. When I first heard the term “prompt engineer” I rolled my eyes, but now that I’ve gotten into it I see it’s really an art form.

JohnFen 3y ago

The first AI-style program I ever wrote (about 25 years ago. Yes, I'm old) played 20 questions, but it would "learn" from prior games, so the more you played, the better it performed. It got extremely good after a few hundred games.