3 ms·

you certainly make an interesting point about fact and falsehood and how they interpret language; however, I disagree that this is relevant and completely disag

you certainly make an interesting point about fact and falsehood and how they interpret language; however, I disagree that this is relevant and completely disagree that personification has any relevance to the most immediate dangers the danger here is that we make extremely powerful tools extremely available and people use those tools to hurt people, possibly a lot of people, possibly even by accident the scary part is that hacker news is full of the exact people that have the knowledge and capability to do this, and look at the reaction to this post: a resounding fuck off

thomastjeffery 4y ago
I just explained explicitly how and why an LLM is not, and never will be, an "extremely powerful tool". A program that is actually a person in a black box would require the level of concern you are showing. Thankfully, that is not even remotely what we have here in reality.
- permo-w 4y ago
  have you tried gpt-4? how can you possibly say it's not an extremely powerful tool? what happens when someone determined links it - or thousands of its - or thousands of next year's 3x as powerful its -- up to the internet with a code interpreter and a hard-coded poor intention?
  - tome 4y ago
    I don't know; what happens? I'd really like to know what people are imagining it could do, especially if they've got some evidence for it!
    - permo-w 4y ago
      it's a rhetorical question. think about it yourself. try it yourself
  - qlm 4y ago
    What could it do that isn't already being done? I ask this sincerely. We already have bots constantly attacking every server they find 24/7. We already have botnets Is the concern that GPT will be able to find and exploit e.g. a novel RCE exploit in SSH?
    - lanstin 4y ago
      I wish we could ask it to answer questions about GitHub repos.
    - staunton 4y ago
      For example, it could scan social media and use that information to send a very large number of people very personalized messages that manipulate them into doing X. Here, X may include meeting other people to tempt/coerce them into doing Y. The reason this isn't possible now is that nobody has the time to look at millions of social media profiles and then write millions of personalized messages. To do it, you would need a lot of people who you would have to convince, including paying them. And even so, things like doomsday cults and terrorist organizations already exist.
  - quonn 4y ago
    Well, how can it be much worse than a smart human „hooked up to the internet“ with bad intentions? Other then generating comments or spam faster?
- yifanl 4y ago
  It's (to some subset of the audience), an extremely persuasive tool, which is powerful in its own right. It's actual capabilities mean very little, if some people can be convinced into doing decision making without actually doing any decision making by it (or literally any tool), that's cause for concern.
- JoshuaDavid 4y ago
  Can you give an example of a concrete task that you believe that neither GPT-4, nor its successors, will be able to accomplish within the next 2 years? Keep in mind that ChatGPT Plugins[1] exist. If you can't think of any tasks that a tool won't be able to accomplish within a 2 year period, I question your definition of "not extremely powerful". [1] https://openai.com/blog/chatgpt-plugins https://openai.com/blog/chatgpt-plugins
  - thomastjeffery 4y ago
    Categorize. Decide. Compare. Refuse. Do. --- Sure, it can exhibit these behaviors, but each behavior itself never actually leaves the text that humans wrote it into. That means we are reliant on how well the behaviors we want can be modeled into text patterns. Unfortunately, natural language is ambiguous. There are many identical patterns that contain different behavior. GPT's authors introduce some of these patterns as both "feature" and "limitation": Features: > The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests. Limitations: > ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. > given one phrasing of a question, the model can claim to not know the answer, but given a slight rephrase, can answer correctly. These are not separate. They are two sides of the same coin. GPT will never be able to look objectively at its own model. It will never be able to subjectively categorize patterns or tokens. If it could, it would not be an LLM: it would be an entirely new technology.
    - JoshuaDavid 4y ago
      I am not sure what you mean by "each behavior itself never actually leaves the text that humans wrote it into". Do you mean that it cannot generalize outside of its training distribution, or are you gesturing at something else? I notice that I am particularly confused by the example of "categorize" -- I am pretty sure you're not saying that ChatGPT can't categorize things, but then I have no good hypothesis for what you do mean.
      - thomastjeffery 4y ago
tome 4y ago
> people use those tools to hurt people, possibly a lot of people, possibly even by accident If it happens then hopefully it's by accident!

thomastjeffery 4y ago

I just explained explicitly how and why an LLM is not, and never will be, an "extremely powerful tool". A program that is actually a person in a black box would require the level of concern you are showing. Thankfully, that is not even remotely what we have here in reality.

permo-w 4y ago
have you tried gpt-4? how can you possibly say it's not an extremely powerful tool? what happens when someone determined links it - or thousands of its - or thousands of next year's 3x as powerful its -- up to the internet with a code interpreter and a hard-coded poor intention?
- tome 4y ago
  I don't know; what happens? I'd really like to know what people are imagining it could do, especially if they've got some evidence for it!
  - permo-w 4y ago
    it's a rhetorical question. think about it yourself. try it yourself
- qlm 4y ago
  What could it do that isn't already being done? I ask this sincerely. We already have bots constantly attacking every server they find 24/7. We already have botnets Is the concern that GPT will be able to find and exploit e.g. a novel RCE exploit in SSH?
  - lanstin 4y ago
    I wish we could ask it to answer questions about GitHub repos.
  - staunton 4y ago
    For example, it could scan social media and use that information to send a very large number of people very personalized messages that manipulate them into doing X. Here, X may include meeting other people to tempt/coerce them into doing Y. The reason this isn't possible now is that nobody has the time to look at millions of social media profiles and then write millions of personalized messages. To do it, you would need a lot of people who you would have to convince, including paying them. And even so, things like doomsday cults and terrorist organizations already exist.
- quonn 4y ago
  Well, how can it be much worse than a smart human „hooked up to the internet“ with bad intentions? Other then generating comments or spam faster?
yifanl 4y ago
It's (to some subset of the audience), an extremely persuasive tool, which is powerful in its own right. It's actual capabilities mean very little, if some people can be convinced into doing decision making without actually doing any decision making by it (or literally any tool), that's cause for concern.
JoshuaDavid 4y ago
Can you give an example of a concrete task that you believe that neither GPT-4, nor its successors, will be able to accomplish within the next 2 years? Keep in mind that ChatGPT Plugins[1] exist. If you can't think of any tasks that a tool won't be able to accomplish within a 2 year period, I question your definition of "not extremely powerful". [1] https://openai.com/blog/chatgpt-plugins https://openai.com/blog/chatgpt-plugins
- thomastjeffery 4y ago
  Categorize. Decide. Compare. Refuse. Do. --- Sure, it can exhibit these behaviors, but each behavior itself never actually leaves the text that humans wrote it into. That means we are reliant on how well the behaviors we want can be modeled into text patterns. Unfortunately, natural language is ambiguous. There are many identical patterns that contain different behavior. GPT's authors introduce some of these patterns as both "feature" and "limitation": Features: > The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests. Limitations: > ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. > given one phrasing of a question, the model can claim to not know the answer, but given a slight rephrase, can answer correctly. These are not separate. They are two sides of the same coin. GPT will never be able to look objectively at its own model. It will never be able to subjectively categorize patterns or tokens. If it could, it would not be an LLM: it would be an entirely new technology.
  - JoshuaDavid 4y ago
    I am not sure what you mean by "each behavior itself never actually leaves the text that humans wrote it into". Do you mean that it cannot generalize outside of its training distribution, or are you gesturing at something else? I notice that I am particularly confused by the example of "categorize" -- I am pretty sure you're not saying that ChatGPT can't categorize things, but then I have no good hypothesis for what you do mean.
    - thomastjeffery 4y ago

tome 4y ago

> people use those tools to hurt people, possibly a lot of people, possibly even by accident If it happens then hopefully it's by accident!