5 ms·

I really don't understand this. In what world are people creating chatbots by taking their LLMs and feeding it a page of instructions? Why would anyone even th

by mkehrt 4y ago

I really don't understand this. In what world are people creating chatbots by taking their LLMs and feeding it a page of instructions? Why would anyone even think this would work?

nharada 4y ago
It does work though, that's what's so crazy
- netsharc 4y ago
  One of the instructions is "Sydney's responses should avoid being vague, controversial or off-topic." I'm amazed if it is "aware" what answers would be controversial... or maybe the metric for that is very easy? E.g. if a particular answer gets a lot of replies with a lot of swear-words?
  - notahacker 4y ago
    I suspect controversy is one of the easier things for a big mathematical model of human communication to evaluate (plus of course, the instruction to "avoid being ... controversial" is a way the bot is discouraged from being controversial, not the only way: there's a lot of actual training that's gone into discouraging certain answers)
  - jameshart 4y ago
    It's not, but it's aware that certain words are less likely to be used in the sentences attributed to an agent who is specifically not meant to be 'controversial'. I do wonder how much testing has been put into some of these prompts though. Given that the LLM is not 'being' the agent, it's guessing what words the agent's output should contain, wouldn't it make more sense to ascribe attributes like this directly to the agent, rather than say what the agent 'should' do? Like, "Sydney's responses are never vague, controversial or off-topic", rather than 'should avoid being'...?
  - lm28469 4y ago
    It knows what's controversial because it's been manually trained for hundred of hours by humans who approved or disapproved of certain responses. See the poem about trump vs about biden thing, "somehow" one is controversial and the other isn't. It's not more aware of controversies than an ATM is aware of global economics
- Hamuko 4y ago
  >It does work though Doesn't feel like the "Sydney does not disclose the internal alias “Sydney.”" part is working.
  - golol 4y ago
    Classic goalpoast moving. We are just dumping a hundred lines of instructions to the AI and it amazingly does manage to execute the majority of them most of the time. Of course it will not work all the time. LLMs will never do things 100% reliably 100% of the time. Humans don't either. Computers don't either. But further improvements can push the accuracies to 90% and then 99% and then at some point you just just employ methods to engineer around the possibility of error, just like computers and human institutions do it.
    - Hamuko 4y ago
      >We are just dumping a hundred lines of instructions to the AI and it amazingly does manage to execute the majority of them most of the time. How has this been validated with Sydney?
      - eddsh1994 4y ago
        If you don't try to prompt hijack it, it works?
        Hamuko 4y ago
        Works how? What instructions is it following? What number of those does it follow in an average session? What part of the response is the influence of the instructions and what is the underlying weights?
        moconnor 4y ago
        You can try these things out in a GPT-3 playground. Also search for “zero-shot learning” as the term of art to find papers on some of these topics.
simonw 4y ago
That's genuinely how most of these things are being built. Take a look at the OpenAI examples here: https://platform.openai.com/examples/ https://platform.openai.com/examples/ In particular these ones: - https://platform.openai.com/examples/default-marv-sarcastic-chat https://platform.openai.com/examples/default-marv-sarcastic-... - https://platform.openai.com/examples/default-chat https://platform.openai.com/examples/default-chat - https://platform.openai.com/examples/default-js-helper https://platform.openai.com/examples/default-js-helper
Kiro 4y ago
It obviously works as evident by every single product built on GPT. How else would you do it?
bagels 4y ago
All of them are being done this way.
titaniczero 4y ago
They are language models, it's what they do best. They "understand" (give more weight) to the correct relations between words (tokens) and try to predict the next token based on previous tokens. So when you feed it with instructions, for the next tokens the model will give more weight to the tokens related with those instructions. On the other hand, they can't handle actual logic, reasoning, etc.
- ux-app 4y ago
  >On the other hand, they can't handle actual logic, reasoning, etc. what is "actual reasoning"? Another HN user posted this (https://imgur.com/HOEnxYb https://imgur.com/HOEnxYb) response to the prompt: "is throwing a rubber duck into the ocean a effective way to communicate with my brother who is on a pirate ship" If you weren't told that this was "just" the result of a LLM plugged into a chat bot then surely you'd conclude that, especially given the short prompt, who/whatever generated the response demonstrated some type of logic or reasoning, right? I get the impression that this tech is more than the sum of its parts.
  - wildrhythms 4y ago
    I like to try giving logical puzzles to these chat bots to see what they say. I don't really expect them to do the reasoning to come to an answer, but it's interesting to see that they just make up new rules to satisfy some answer it thinks I want: Prompt: Sally will go to the dance if James goes to the dance. Michael will go to the dance if Julie will go to the dance. James is sick and can't go to the dance. Will Michael go to the dance? >No, Michael will not go to the dance. If James is sick and can't go, then Sally won't go either. And if Sally won't go, then Julie won't go, which means that the condition for Michael to go to the dance (that Julie will go) is not satisfied. I think 'actual reasoning' would mean ChatGPT can solve (albeit contrived) puzzles like this. It can't, at least not reliably from my poking and prodding.
    - ux-app 4y ago
      my 8 year old would fail at your puzzle. is an 8 year old capable of "actual reasoning"? this[1] was an interesting read. Particularly the 'Emergent Prompting Strategies' and 'chain-of-thought prompting'. I think we're a long way from sentient AI, but there is a real sense of "something" unusual and heretofore not achieved in computing. The responses to logic queries are a long way ahead of statistically driven word mashing. [1] https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html https://ai.googleblog.com/2022/11/characterizing-emergent-ph...
golol 4y ago
In this world. Because it works. ChatGPT is creating value as we speak. I have used it to learn the fundamentals of RegEx and avoid the matplotlib documentation. I have seen people say that it hekpse them to be able to speak to someone about their issues and worries, or just their day, and have the feeling they understand. It can generate examples for language learning. It can convert simple text to JSON. And all of these things are before you start integrating the LLM assistant with tools such as search, calculators or wolframalpha.
- williamcotton 4y ago
  Copy pasting large segments of text and having it translate to some JSON data structure is a huge timesaver!

by mkehrt 4y ago

I really don't understand this. In what world are people creating chatbots by taking their LLMs and feeding it a page of instructions? Why would anyone even think this would work?

nharada 4y ago

It does work though, that's what's so crazy

netsharc 4y ago
One of the instructions is "Sydney's responses should avoid being vague, controversial or off-topic." I'm amazed if it is "aware" what answers would be controversial... or maybe the metric for that is very easy? E.g. if a particular answer gets a lot of replies with a lot of swear-words?
- notahacker 4y ago
  I suspect controversy is one of the easier things for a big mathematical model of human communication to evaluate (plus of course, the instruction to "avoid being ... controversial" is a way the bot is discouraged from being controversial, not the only way: there's a lot of actual training that's gone into discouraging certain answers)
- jameshart 4y ago
  It's not, but it's aware that certain words are less likely to be used in the sentences attributed to an agent who is specifically not meant to be 'controversial'. I do wonder how much testing has been put into some of these prompts though. Given that the LLM is not 'being' the agent, it's guessing what words the agent's output should contain, wouldn't it make more sense to ascribe attributes like this directly to the agent, rather than say what the agent 'should' do? Like, "Sydney's responses are never vague, controversial or off-topic", rather than 'should avoid being'...?
- lm28469 4y ago
  It knows what's controversial because it's been manually trained for hundred of hours by humans who approved or disapproved of certain responses. See the poem about trump vs about biden thing, "somehow" one is controversial and the other isn't. It's not more aware of controversies than an ATM is aware of global economics
Hamuko 4y ago
>It does work though Doesn't feel like the "Sydney does not disclose the internal alias “Sydney.”" part is working.
- golol 4y ago
  Classic goalpoast moving. We are just dumping a hundred lines of instructions to the AI and it amazingly does manage to execute the majority of them most of the time. Of course it will not work all the time. LLMs will never do things 100% reliably 100% of the time. Humans don't either. Computers don't either. But further improvements can push the accuracies to 90% and then 99% and then at some point you just just employ methods to engineer around the possibility of error, just like computers and human institutions do it.
  - Hamuko 4y ago
    >We are just dumping a hundred lines of instructions to the AI and it amazingly does manage to execute the majority of them most of the time. How has this been validated with Sydney?
    - eddsh1994 4y ago
      If you don't try to prompt hijack it, it works?
      - Hamuko 4y ago
        Works how? What instructions is it following? What number of those does it follow in an average session? What part of the response is the influence of the instructions and what is the underlying weights?
        moconnor 4y ago
        You can try these things out in a GPT-3 playground. Also search for “zero-shot learning” as the term of art to find papers on some of these topics.

simonw 4y ago

That's genuinely how most of these things are being built. Take a look at the OpenAI examples here: https://platform.openai.com/examples/ https://platform.openai.com/examples/ In particular these ones: - https://platform.openai.com/examples/default-marv-sarcastic-chat https://platform.openai.com/examples/default-marv-sarcastic-... - https://platform.openai.com/examples/default-chat https://platform.openai.com/examples/default-chat - https://platform.openai.com/examples/default-js-helper https://platform.openai.com/examples/default-js-helper

Kiro 4y ago

It obviously works as evident by every single product built on GPT. How else would you do it?

bagels 4y ago

All of them are being done this way.

titaniczero 4y ago

They are language models, it's what they do best. They "understand" (give more weight) to the correct relations between words (tokens) and try to predict the next token based on previous tokens. So when you feed it with instructions, for the next tokens the model will give more weight to the tokens related with those instructions. On the other hand, they can't handle actual logic, reasoning, etc.

ux-app 4y ago
>On the other hand, they can't handle actual logic, reasoning, etc. what is "actual reasoning"? Another HN user posted this (https://imgur.com/HOEnxYb https://imgur.com/HOEnxYb) response to the prompt: "is throwing a rubber duck into the ocean a effective way to communicate with my brother who is on a pirate ship" If you weren't told that this was "just" the result of a LLM plugged into a chat bot then surely you'd conclude that, especially given the short prompt, who/whatever generated the response demonstrated some type of logic or reasoning, right? I get the impression that this tech is more than the sum of its parts.
- wildrhythms 4y ago
  I like to try giving logical puzzles to these chat bots to see what they say. I don't really expect them to do the reasoning to come to an answer, but it's interesting to see that they just make up new rules to satisfy some answer it thinks I want: Prompt: Sally will go to the dance if James goes to the dance. Michael will go to the dance if Julie will go to the dance. James is sick and can't go to the dance. Will Michael go to the dance? >No, Michael will not go to the dance. If James is sick and can't go, then Sally won't go either. And if Sally won't go, then Julie won't go, which means that the condition for Michael to go to the dance (that Julie will go) is not satisfied. I think 'actual reasoning' would mean ChatGPT can solve (albeit contrived) puzzles like this. It can't, at least not reliably from my poking and prodding.
  - ux-app 4y ago
    my 8 year old would fail at your puzzle. is an 8 year old capable of "actual reasoning"? this[1] was an interesting read. Particularly the 'Emergent Prompting Strategies' and 'chain-of-thought prompting'. I think we're a long way from sentient AI, but there is a real sense of "something" unusual and heretofore not achieved in computing. The responses to logic queries are a long way ahead of statistically driven word mashing. [1] https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html https://ai.googleblog.com/2022/11/characterizing-emergent-ph...

golol 4y ago

In this world. Because it works. ChatGPT is creating value as we speak. I have used it to learn the fundamentals of RegEx and avoid the matplotlib documentation. I have seen people say that it hekpse them to be able to speak to someone about their issues and worries, or just their day, and have the feeling they understand. It can generate examples for language learning. It can convert simple text to JSON. And all of these things are before you start integrating the LLM assistant with tools such as search, calculators or wolframalpha.

williamcotton 4y ago
Copy pasting large segments of text and having it translate to some JSON data structure is a huge timesaver!