4 ms·

> "It doesn't have a goal." It can be triggered to search the internet, which is taking action. You saying "it will never take actions because it doesn't have

> "It doesn't have a goal." It can be triggered to search the internet, which is taking action. You saying "it will never take actions because it doesn't have a goal" after seeing it take actions is nonsensical. If it gains the ability to, say, make bitcoin transactions on your behalf and you prompt it down a chain of events where it does that and orders toy pistols sent to the authorities with your name on the order, what difference does it make if "it had a goal" or not?

staticman2 4y ago
A chatbot that only speaks when spoken to is going to gain the ability to trade Bitcoin?
- jodrellblank 4y ago
  I didn't say 'trade', I said 'make transactions'. It's no more complex than Bing Chat being able to search the internet, or Siri being able to send JSON to an endpoint which turns lightbulbs on and off. Instead it's a shopping endpoint and ChatWhatever can include tokens related to approving transactions from your Bitcoin wallet and has your authorization to use it for purchases less than $100.
- theptip 4y ago
  Look at OpenAssistant (https://github.com/LAION-AI/Open-Assistant); https://github.com/LAION-AI/Open-Assistant); they are trying to give a chatbot the ability to trigger actions in other systems. I fleshed out a scenario in more detail here: https://news.ycombinator.com/item?id=34808674 https://news.ycombinator.com/item?id=34808674. But in brief, the short-term evolution of LLMs is going to involve something like letting it `eval()` some code to take an action as part of a response to a prompt. A recent paper, Toolformer: https://pub.towardsai.net/exploring-toolformer-meta-ai-new-transformer-learned-to-use-tools-to-produce-better-answers-1b8c042e9c46 https://pub.towardsai.net/exploring-toolformer-meta-ai-new-t... which is training on a small set of hand-chosen tools, rather than `eval(<arbitrary code>)`, but hopefully it's clear that it's a very small step from the former to the latter.
  - williamcotton 4y ago
    I’ve been getting very good results from eval on JS written by GPT. It is surprising apt at learning when to query a source like wolframalpha or wikipedia and when to write an inline function. You can stop it from being recursive by passing it through a model that is not trained to write JavaScript but is trained to output JSON.
- int_19h 4y ago
  How often do you think a Bing query is made? You might say that it doesn't preserve state between different sessions, and that's true. But if it can read and post online, then it can preserve state there.
- ProblemFactory 4y ago
  > that only speaks when spoken to Feedback loops are an important part. But let's say you take two current chatbots, make them converse with each other without human participants. Add full internet access. Add a directive to read HN, Twitter and latest news often. Interesting emergent behaviour could emerge very soon.
  - dTal 4y ago
    Worse, you need only plug a chatbot into itself, with some kind of basic bash script and very simple "goal prompt", and suddenly you get an agent with long term context. You could do that today. I don't think people realize how close these generic undirected intelligences are to unpredictable complex behavior. A sobering intuition pump: https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
    - lunarhustler 4y ago
      man, these lesswrong people sure do have some cool ideas about how to apply GPT
nradov 4y ago
If I give an automated system the ability to make transactions on my behalf then there is already a risk that someone will misuse it or exploit a security vulnerability. It could be a disgruntled employee, or a hacker in Kazakhstan doing it for the lulz. The existence of LLM AI tools changes nothing here. It is already possible to order toy pistols sent to the authorities with someone else's name on the order. People use stolen credit card numbers for all sorts of malicious purposes. And have you heard of swatting?
- jodrellblank 4y ago
  The existence of LLM AI tools changes things because it used to not exist and now does exist? It used to be that an AI tool could not do things on your behalf because they did not exist, now it could be that they could do things on your behalf because they do exist and people are giving them ability to take actions on the human's behalf. It used to be that a Kazakhstani hacker could find and exploit a security vulnerability once or twice a year, it can now become that millions of people are querying the AI and having it act on their behalf many times per day.
  - nradov 4y ago
    The risk has existing for many years with humans and other tools. The addition of one more tool to the mix changes nothing.

staticman2 4y ago

A chatbot that only speaks when spoken to is going to gain the ability to trade Bitcoin?

jodrellblank 4y ago
I didn't say 'trade', I said 'make transactions'. It's no more complex than Bing Chat being able to search the internet, or Siri being able to send JSON to an endpoint which turns lightbulbs on and off. Instead it's a shopping endpoint and ChatWhatever can include tokens related to approving transactions from your Bitcoin wallet and has your authorization to use it for purchases less than $100.
theptip 4y ago
Look at OpenAssistant (https://github.com/LAION-AI/Open-Assistant); https://github.com/LAION-AI/Open-Assistant); they are trying to give a chatbot the ability to trigger actions in other systems. I fleshed out a scenario in more detail here: https://news.ycombinator.com/item?id=34808674 https://news.ycombinator.com/item?id=34808674. But in brief, the short-term evolution of LLMs is going to involve something like letting it `eval()` some code to take an action as part of a response to a prompt. A recent paper, Toolformer: https://pub.towardsai.net/exploring-toolformer-meta-ai-new-transformer-learned-to-use-tools-to-produce-better-answers-1b8c042e9c46 https://pub.towardsai.net/exploring-toolformer-meta-ai-new-t... which is training on a small set of hand-chosen tools, rather than `eval(<arbitrary code>)`, but hopefully it's clear that it's a very small step from the former to the latter.
- williamcotton 4y ago
  I’ve been getting very good results from eval on JS written by GPT. It is surprising apt at learning when to query a source like wolframalpha or wikipedia and when to write an inline function. You can stop it from being recursive by passing it through a model that is not trained to write JavaScript but is trained to output JSON.
int_19h 4y ago
How often do you think a Bing query is made? You might say that it doesn't preserve state between different sessions, and that's true. But if it can read and post online, then it can preserve state there.
ProblemFactory 4y ago
> that only speaks when spoken to Feedback loops are an important part. But let's say you take two current chatbots, make them converse with each other without human participants. Add full internet access. Add a directive to read HN, Twitter and latest news often. Interesting emergent behaviour could emerge very soon.
- dTal 4y ago
  Worse, you need only plug a chatbot into itself, with some kind of basic bash script and very simple "goal prompt", and suddenly you get an agent with long term context. You could do that today. I don't think people realize how close these generic undirected intelligences are to unpredictable complex behavior. A sobering intuition pump: https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality-is-the-tiger-and-agents-are-its-teeth https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
  - lunarhustler 4y ago
    man, these lesswrong people sure do have some cool ideas about how to apply GPT

nradov 4y ago

If I give an automated system the ability to make transactions on my behalf then there is already a risk that someone will misuse it or exploit a security vulnerability. It could be a disgruntled employee, or a hacker in Kazakhstan doing it for the lulz. The existence of LLM AI tools changes nothing here. It is already possible to order toy pistols sent to the authorities with someone else's name on the order. People use stolen credit card numbers for all sorts of malicious purposes. And have you heard of swatting?

jodrellblank 4y ago
The existence of LLM AI tools changes things because it used to not exist and now does exist? It used to be that an AI tool could not do things on your behalf because they did not exist, now it could be that they could do things on your behalf because they do exist and people are giving them ability to take actions on the human's behalf. It used to be that a Kazakhstani hacker could find and exploit a security vulnerability once or twice a year, it can now become that millions of people are querying the AI and having it act on their behalf many times per day.
- nradov 4y ago
  The risk has existing for many years with humans and other tools. The addition of one more tool to the mix changes nothing.