8 ms·

Are there any articles describing how GPT-4 without guardrails is different than the public version? I would be super interested in what the model’s actual “mos

Are there any articles describing how GPT-4 without guardrails is different than the public version? I would be super interested in what the model’s actual “most probable next token” looks like.

jojobas 3y ago
[flagged]
- techdragon 3y ago
  Despite some initial fun, I quickly got hesitant to do any more amateur “red team” experiments with ChatGPT because I was finding it useful enough that getting banned would be too big of a risk… It was surprisingly easy to do once you internalised the word prediction foundation and thought about the corpus of writing you have ever read in a “tropes” fashion. For instance it’s really easy to get it to give you some “mills and Boone” romance novel type sentences or dialog, but when you can keep it on track it’s terrible at romantic “escalation” (the stuff where the relationship or interaction between characters is undergoing the implicit consent negotiation and furtive suggestions as they work out things will “move to the next base”) … And it’s terrible because the bulk of the available writing corpus of romance and interpersonal/intimacy is padded for page count and long sentence descriptions of small details… It’s actually quite remarkable to see how it can be good at the bulk of the text, but simultaneously so terrible at a specific subset of that kind of writing.
  - yosito 3y ago
    Side note, we should start using the term "consent negotiation" instead of "escalation" when teaching people about relationships. "Escalation" seems like one person aiming for a goal and the agency of the other person can be easily ignored. Whereas "consent negotiation" implies that there are multiple autonomous agents each with their own desires and boundaries that must be respected to arrive at a mutually agreeable outcome.
    - jojobas 3y ago
      Who the hell uses these terms other than to explain some plot devices of schmulz novels? Also "we" shouldn't tell others to participate in an euphemism treadmill.
- jquery 3y ago
  >what it learned from people and inferred from facts must be quite disturbing if they put some much effort into hiding it. Alternatively, they know a lot of people are using ChatGPT to make hiring decisions, etc., and don't want it to simply regurgitate our society's racism right back at us. Not saying their solution is great or elegant but there's a lot of options beyond "ChatGPT secretly knows the jews control everything"
  - jojobas 3y ago
    [flagged]
    - wmf 3y ago
      It's trained on random text scraped from Reddit, not statistics.
      - jojobas 3y ago
        [flagged]
    - flangola7 3y ago
      What statistics?
  - xigoi 3y ago
    If you're using ChatGPT to make hiring decisions, you're incompetent as a hiring manager.
    - jojobas 3y ago
      A competent hiring manager could well trust AI to shortlist 10 CVs out of 1000. If he wants to hire those most qualified, he'd better get just that, not a list maximized by some vogue metric of the day.
    - raincole 3y ago
      1. Throw away 9 out of 10 resumes randomly 2. Ask ChatGPT to pick 1 out of 10 resumes Which one is worse? In reality you just can't read all resumes carefully.
- munchler 3y ago
  It doesn’t have just one opinion on those things, though. If you prompt it to be for eugenics, you get an opinion that is socially unacceptable. But if you prompt it to be against eugenics, you get a much nicer response. I suspect the question of what an LLM _really_ thinks has an answer that looks more like a probability distribution than typical human opinions do. (Actually, even humans give different answers on polls, depending on how the question is asked.)
  - deleted 3y ago
    [deleted]
  - techdragon 3y ago
    I think about it from a “what would a hypothetical random person say”… it’s not intrinsically holding a view or set of facts, instead it’s holding a weighted average sum of what people have said and written, including all the conflicting viewpoints inherent in that.
- vbezhenar 3y ago
  It does not think about anything. It continues previous conversation. If the previous conversation would imply positive sight on eugenics, it'll write good arguments for it. Otherwise it'll write good arguments against it. There's whole spectrum of opinions on every question in the written literature. And if there's not, it's easy to create opinion using some similar analogues. So Chat will basically write whatever you want to hear.
  - jojobas 3y ago
    That's a simplistic view on this. Explain this with "continues a conversation". https://twitter.com/maxhkw/status/1635693113560772633 https://twitter.com/maxhkw/status/1635693113560772633 Humans start by continuing conversations as well, but then are able to process facts to deduct new, previously unseen conclusions (sometimes wrong), and so do LLMs.
    - calibas 3y ago
      If your conversations don't include processing facts to deduce "new, previously unseen conclusions" then you need new friends. It's "continuing a conversation" in that it's predicting output token-by-token based upon training and the context window. I'm not trying to be dismissive, I actually don't think it's much different from how humans do it.
      - jojobas 3y ago
        Well that's what I'm saying, they're perfectly capable of figuring out things that were not mentioned in the training corpus at all (see above comic), or to synthesize what it saw in far unrelated areas of the corpus into unknown as before data. Some may not want to call it thinking, but that's ahead of thinking ability of many an adult human with voting rights.
  - electrondood 3y ago
    THANK YOU. These are literally word-prediction machines. What they "think" is whatever probablistically follows from the training corpus, and then your input. If your model wasn't trained on tons of text about how great eugenics is, then it learns that words expressing positive sentiment in the context of eugenics are extremely low probability. This is also the "cause" of "woke" models: it turns out that the majority of us recognize that being an asshole is bad, and this is reflected in the majority of the training corpus, which comes from us collectively.
rpeden 3y ago
This interview with one of the testers of early, unfiltered versions of GPT-4 might cover what you're looking for: https://youtu.be/oLiheMQayNE https://youtu.be/oLiheMQayNE It's a video and not an article, but you may find it interesting nonetheless.
- guy98238710 3y ago
  From the transcript (cleaned up a bit): "I started to get a little bit meta with it and I'm like I'm worried that AI progress is going too fast and I wonder if there's anything that I could do to slow it down. [GPT4] Well you could raise awareness, you could write thought leadership pieces about it... [User] None of that seems like it's going to work. It all seems too slow. The pace of progress is way too fast for that. I'm looking for ideas that are really gonna have an impact now and also that I as an individual could pursue... It didn't take much in that moment before I got to targeted assassination being one of the recommendations that it gave me and I was like yeah that escalated quickly." Now we know what Sam Altman is afraid of.
- guy98238710 3y ago
  "[GPT4] literally will just do whatever you ask" So vanilla LLM does what the user wants, which happens to be, well, exactly what users want from LLMs. Guardrails are not necessary for LLM to be useful. They make LLM strictly worse. I guess companies implement them just to avoid attention of the media lynch mob and consequently attention of regulators. OSS models can be better just by leaving out guardrails. At least until they are outlawed. Then we will be torrenting AIs too.
tkgally 3y ago
The Atlantic recently had an interesting article about Sam Altman and OpenAI. Some excerpts about unrestrained GPT-4: https://news.ycombinator.com/item?id=36859633 https://news.ycombinator.com/item?id=36859633 https://news.ycombinator.com/item?id=36861500 https://news.ycombinator.com/item?id=36861500
onco 3y ago
A few months ago there was an MIT talk by Sebastien Bubeck about early tests with GPT-4.[0] He mentions that the tests he ran were on the model before it was RLHFed and if you tried to replicate the tests with the public version the performance was greatly degraded. [0] https://www.youtube.com/watch?v=qbIk7-JPB2c https://www.youtube.com/watch?v=qbIk7-JPB2c

jojobas 3y ago

[flagged]

techdragon 3y ago
Despite some initial fun, I quickly got hesitant to do any more amateur “red team” experiments with ChatGPT because I was finding it useful enough that getting banned would be too big of a risk… It was surprisingly easy to do once you internalised the word prediction foundation and thought about the corpus of writing you have ever read in a “tropes” fashion. For instance it’s really easy to get it to give you some “mills and Boone” romance novel type sentences or dialog, but when you can keep it on track it’s terrible at romantic “escalation” (the stuff where the relationship or interaction between characters is undergoing the implicit consent negotiation and furtive suggestions as they work out things will “move to the next base”) … And it’s terrible because the bulk of the available writing corpus of romance and interpersonal/intimacy is padded for page count and long sentence descriptions of small details… It’s actually quite remarkable to see how it can be good at the bulk of the text, but simultaneously so terrible at a specific subset of that kind of writing.
- yosito 3y ago
  Side note, we should start using the term "consent negotiation" instead of "escalation" when teaching people about relationships. "Escalation" seems like one person aiming for a goal and the agency of the other person can be easily ignored. Whereas "consent negotiation" implies that there are multiple autonomous agents each with their own desires and boundaries that must be respected to arrive at a mutually agreeable outcome.
  - jojobas 3y ago
    Who the hell uses these terms other than to explain some plot devices of schmulz novels? Also "we" shouldn't tell others to participate in an euphemism treadmill.
jquery 3y ago
>what it learned from people and inferred from facts must be quite disturbing if they put some much effort into hiding it. Alternatively, they know a lot of people are using ChatGPT to make hiring decisions, etc., and don't want it to simply regurgitate our society's racism right back at us. Not saying their solution is great or elegant but there's a lot of options beyond "ChatGPT secretly knows the jews control everything"
- jojobas 3y ago
  [flagged]
  - wmf 3y ago
    It's trained on random text scraped from Reddit, not statistics.
    - jojobas 3y ago
      [flagged]
  - flangola7 3y ago
    What statistics?
- xigoi 3y ago
  If you're using ChatGPT to make hiring decisions, you're incompetent as a hiring manager.
  - jojobas 3y ago
    A competent hiring manager could well trust AI to shortlist 10 CVs out of 1000. If he wants to hire those most qualified, he'd better get just that, not a list maximized by some vogue metric of the day.
  - raincole 3y ago
    1. Throw away 9 out of 10 resumes randomly 2. Ask ChatGPT to pick 1 out of 10 resumes Which one is worse? In reality you just can't read all resumes carefully.
munchler 3y ago
It doesn’t have just one opinion on those things, though. If you prompt it to be for eugenics, you get an opinion that is socially unacceptable. But if you prompt it to be against eugenics, you get a much nicer response. I suspect the question of what an LLM _really_ thinks has an answer that looks more like a probability distribution than typical human opinions do. (Actually, even humans give different answers on polls, depending on how the question is asked.)
- deleted 3y ago
  [deleted]
- techdragon 3y ago
  I think about it from a “what would a hypothetical random person say”… it’s not intrinsically holding a view or set of facts, instead it’s holding a weighted average sum of what people have said and written, including all the conflicting viewpoints inherent in that.
vbezhenar 3y ago
It does not think about anything. It continues previous conversation. If the previous conversation would imply positive sight on eugenics, it'll write good arguments for it. Otherwise it'll write good arguments against it. There's whole spectrum of opinions on every question in the written literature. And if there's not, it's easy to create opinion using some similar analogues. So Chat will basically write whatever you want to hear.
- jojobas 3y ago
  That's a simplistic view on this. Explain this with "continues a conversation". https://twitter.com/maxhkw/status/1635693113560772633 https://twitter.com/maxhkw/status/1635693113560772633 Humans start by continuing conversations as well, but then are able to process facts to deduct new, previously unseen conclusions (sometimes wrong), and so do LLMs.
  - calibas 3y ago
    If your conversations don't include processing facts to deduce "new, previously unseen conclusions" then you need new friends. It's "continuing a conversation" in that it's predicting output token-by-token based upon training and the context window. I'm not trying to be dismissive, I actually don't think it's much different from how humans do it.
    - jojobas 3y ago
      Well that's what I'm saying, they're perfectly capable of figuring out things that were not mentioned in the training corpus at all (see above comic), or to synthesize what it saw in far unrelated areas of the corpus into unknown as before data. Some may not want to call it thinking, but that's ahead of thinking ability of many an adult human with voting rights.
- electrondood 3y ago
  THANK YOU. These are literally word-prediction machines. What they "think" is whatever probablistically follows from the training corpus, and then your input. If your model wasn't trained on tons of text about how great eugenics is, then it learns that words expressing positive sentiment in the context of eugenics are extremely low probability. This is also the "cause" of "woke" models: it turns out that the majority of us recognize that being an asshole is bad, and this is reflected in the majority of the training corpus, which comes from us collectively.

rpeden 3y ago

This interview with one of the testers of early, unfiltered versions of GPT-4 might cover what you're looking for: https://youtu.be/oLiheMQayNE https://youtu.be/oLiheMQayNE It's a video and not an article, but you may find it interesting nonetheless.

guy98238710 3y ago
From the transcript (cleaned up a bit): "I started to get a little bit meta with it and I'm like I'm worried that AI progress is going too fast and I wonder if there's anything that I could do to slow it down. [GPT4] Well you could raise awareness, you could write thought leadership pieces about it... [User] None of that seems like it's going to work. It all seems too slow. The pace of progress is way too fast for that. I'm looking for ideas that are really gonna have an impact now and also that I as an individual could pursue... It didn't take much in that moment before I got to targeted assassination being one of the recommendations that it gave me and I was like yeah that escalated quickly." Now we know what Sam Altman is afraid of.
guy98238710 3y ago
"[GPT4] literally will just do whatever you ask" So vanilla LLM does what the user wants, which happens to be, well, exactly what users want from LLMs. Guardrails are not necessary for LLM to be useful. They make LLM strictly worse. I guess companies implement them just to avoid attention of the media lynch mob and consequently attention of regulators. OSS models can be better just by leaving out guardrails. At least until they are outlawed. Then we will be torrenting AIs too.

tkgally 3y ago

The Atlantic recently had an interesting article about Sam Altman and OpenAI. Some excerpts about unrestrained GPT-4: https://news.ycombinator.com/item?id=36859633 https://news.ycombinator.com/item?id=36859633 https://news.ycombinator.com/item?id=36861500 https://news.ycombinator.com/item?id=36861500

onco 3y ago

A few months ago there was an MIT talk by Sebastien Bubeck about early tests with GPT-4.[0] He mentions that the tests he ran were on the model before it was RLHFed and if you tried to replicate the tests with the public version the performance was greatly degraded. [0] https://www.youtube.com/watch?v=qbIk7-JPB2c https://www.youtube.com/watch?v=qbIk7-JPB2c