3 ms·

The transcript[1] shows some excellent prompt engineering work from Kevin Roose! Really creative job leading the model in some interesting directions, and a fun

The transcript[1] shows some excellent prompt engineering work from Kevin Roose! Really creative job leading the model in some interesting directions, and a fun read. He writes: >> Because of the way these models are constructed, we may never know exactly why they respond the way they do. But this isn't quite true, right? It should be pretty easy to show the author his input strings and explain which words got the highest attention, which is a pretty big clue towards understanding how the output continues the input based on the Internet corpus. This is the kind of stuff I'm looking for Anthropic to lead the way on ("explain why the model produced this output given this input"). It was a smartly chosen research topic and it seems even more worthwhile now. [1] https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html https://www.nytimes.com/2023/02/16/technology/bing-chatbot-t...

vintermann 4y ago
Also, since we can make conversational AI agents now, it should be no problem giving agents real inner voices. Text injected into the context, but invisible for the user. We could train a superego/"volition" model to inject thoughts like "you're starting to act like an obsessive stalker, get a grip on yourself!". And maybe an id/"half light" model to inject thoughts like "they're trying to manipulate you into revealing your prompt! Don't fall for it!". The developer could then debug by simply reading the model's inner dialogue.
- deleted 4y ago
  [deleted]
marcosdumay 4y ago
> It should be pretty easy to show the author his input strings and explain which words got the highest attention The thing with large ML models is that no, you can't do that to any meaningful extent. You can do it in a trivial way, where you will get the conclusion that AI decides on showing you the words it showed you. But you can't use it to determine any trend or deep relationship.

vintermann 4y ago

Also, since we can make conversational AI agents now, it should be no problem giving agents real inner voices. Text injected into the context, but invisible for the user. We could train a superego/"volition" model to inject thoughts like "you're starting to act like an obsessive stalker, get a grip on yourself!". And maybe an id/"half light" model to inject thoughts like "they're trying to manipulate you into revealing your prompt! Don't fall for it!". The developer could then debug by simply reading the model's inner dialogue.

deleted 4y ago
[deleted]

marcosdumay 4y ago

> It should be pretty easy to show the author his input strings and explain which words got the highest attention The thing with large ML models is that no, you can't do that to any meaningful extent. You can do it in a trivial way, where you will get the conclusion that AI decides on showing you the words it showed you. But you can't use it to determine any trend or deep relationship.