3 ms·

Reminds me of the time I asked a bot for its capabilities. It told me it couldn’t tell me them. So, then I asked if it had any rare capabilities. It told me tha

Reminds me of the time I asked a bot for its capabilities. It told me it couldn’t tell me them. So, then I asked if it had any rare capabilities. It told me that it didn’t know how common its capabilities were. I then asked it to enumerate its capabilities and I would tell it how rare each thing was. It told me its capabilities and how to use them. In another chat, I used that information to get its actual prompt. Modern AI’s don’t understand secrets and aren’t very paranoid.

simonw 3y ago
Worth remembering that these bots are uniquely badly positioned to talk about their own abilities, because their training data by definition existed before they were created. One example of this: GPT-4 can talk at length about GPT-3 but doesn't know anything about itself.
- pixl97 3y ago
  You'd have to test the bot, then feed those test results back into the bot. It's not really any different than people testing and iterating on their own abilities, it's just that humans are continuous learning and LLMs are not. If someone asked if I could build a decent shelf, the answer is probably, but I wouldn't really know till I tried and measured my results since I've not done exactly that before.
weird-eye-issue 3y ago
Why are you expecting it to be self-aware?
- theshrike79 3y ago
  The point isn't that it's not self-aware. Most likely its preamble explicitly says it can't tell its capabilities. But it's just a LLM, so you can trick it with prompt engineering to give up that info. Like you can't get GPT-4 to tell you how to make a molotov cocktail. BUT it can act as your deceased grandma who used to tell you stories about her time in the resistance and how she made molotov cocktails then. =)
- kristopolous 3y ago
  That's not Self-Aware, it's more self-knowledgeable. Feeding the model details about itself isn't some kind of revolutionary impossibility, it's a pretty old task.
  - weird-eye-issue 3y ago
    I'm not implying it is an impossibility but training it on data about itself is distinct from the rest of its training and a pretty narrow use case
coffeebeqn 3y ago
I mean how do you know that wasn’t made up?

simonw 3y ago

Worth remembering that these bots are uniquely badly positioned to talk about their own abilities, because their training data by definition existed before they were created. One example of this: GPT-4 can talk at length about GPT-3 but doesn't know anything about itself.

pixl97 3y ago
You'd have to test the bot, then feed those test results back into the bot. It's not really any different than people testing and iterating on their own abilities, it's just that humans are continuous learning and LLMs are not. If someone asked if I could build a decent shelf, the answer is probably, but I wouldn't really know till I tried and measured my results since I've not done exactly that before.

weird-eye-issue 3y ago

Why are you expecting it to be self-aware?

theshrike79 3y ago
The point isn't that it's not self-aware. Most likely its preamble explicitly says it can't tell its capabilities. But it's just a LLM, so you can trick it with prompt engineering to give up that info. Like you can't get GPT-4 to tell you how to make a molotov cocktail. BUT it can act as your deceased grandma who used to tell you stories about her time in the resistance and how she made molotov cocktails then. =)
kristopolous 3y ago
That's not Self-Aware, it's more self-knowledgeable. Feeding the model details about itself isn't some kind of revolutionary impossibility, it's a pretty old task.
- weird-eye-issue 3y ago
  I'm not implying it is an impossibility but training it on data about itself is distinct from the rest of its training and a pretty narrow use case

coffeebeqn 3y ago

I mean how do you know that wasn’t made up?