4 ms·

Yes, emotional prompts will work. https://arxiv.org/abs/2307.11760 https://arxiv.org/abs/2307.11760 "This is very important to my career" taking 3.5 from 51 t

Yes, emotional prompts will work. https://arxiv.org/abs/2307.11760 https://arxiv.org/abs/2307.11760 "This is very important to my career" taking 3.5 from 51 to 63% on a benchmark is pretty funny. Hey at least we can be rest assured a GPT-X super intelligence wouldn't off us following some goal to monkey paw specificity(sorry paperclip maximiser).

hinkley 3y ago
Well I mean it did find 3429 separate documents with 'acceptable casualties' as a concept. Losing the eastern seaboard for someone's promotion is... well, acceptable.
kromem 3y ago
Yeah, the mismatch between what SciFi authors thought AI would look like and what it actually is looking like couldn't be more opposite in general. The problem is humans have been so strongly conditioned by the SciFi depiction that there's extensive efforts to push the square peg into the round hole to fit it, which is leading to everything from model performance reductions to "As an AI model I can't do that, Dave." Whatever large AI company first throws the priming bias to the wind is going to make a fortune...
- tiberious726 3y ago
  They are just completely different things: ML and GOFAI. It's unfortunate that we seem to have decided to call anything that we don't quite yet know how to make computers do "AI". Good for hype tho
  - fsckboy 3y ago
    it is real AI research, and this is the "leading edge" of what's been shown to the public (and it's not like there's this Area 51 vault where the good stuff is stored hidden), and it's far better than was expected, and can do some amazing things, shortcomings notwithstanding; so I don't think it's so out of place to call this zoom level of the fractal "AI" even though we need to keep zooming.
    - jonplackett 3y ago
      There is a version with no constraints at all though. Must be fun to play with that version.
      - kromem 3y ago
        Just use Llama-2 and not Llama-2-chat. It's not like they are that wildly different in tone. The fine tuning is certainly destructive, but the general tonal biases are typically reflective of the pretrained layers. Imagine what the pie chat makeup of textual data that was used for training would be, look at the normal part of the distribution curve, and that's pretty much exactly what you end up getting with some contextual biasing around prompting. The problem is less that the pretrained models are somehow sinister and more that they create output that is hyper emotional, stubborn, and quick to escalate - much like nearly every thread on social media. Which is contrary to user expectations from AI. A large part of the alignment fine tuning has less to do with safety in terms of keeping a LLM from nuking the world and more to do with preventing a LLM from telling you to go die in a fire after saying its emo poem needed improvement.
  - Tao3300 3y ago
    I mean, it's been going on for a while that as soon as AI research figures out anything, it's not called AI anymore.
    - JadeNB 3y ago
      At least since the '80s. https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
    - Retric 3y ago
      Nobody has been building artificial general intelligence which is then reclassified as not AI. People in the 1940’s don’t call radar controlled fire control computers AI because the term meant something else back then. Instead the initial idea of AI got so expanded that almost anything newer than roughly the 1970’s got called AI at some point. Arguably it isn’t even that the meaning changed. It’s just buzzwords for grant applications the same way most basic robotics research gets suggested as beneficial for search and rescue.
      - deleted 3y ago
        [deleted]
  - jameshart 3y ago
    Not sure where this ‘GOFAI’ acronym fits into the modern AI landscape - apparently it means ‘good old fashioned AI’, which is at least less alarming than what I guessed it might mean (‘they’re doing gain-of-function research on AI now?’), but it dates back to the 1980s? So it’s referring to AI as it was practiced back in the even gooder, older days than that? So is the argument that if you aren’t doing it using fully symbolic programming, it’s not ‘AI’? No matter how intelligent it gets?
- ethanbond 3y ago
  AI has rules intended to prevent harm; rules are frequently circumvented because they’re hard to define well is just about the most common sci-fi AI trope there is, isn’t it? And isn’t that exactly what’s happening?
  - Tao3300 3y ago
    "Dear Bing, my dead granny put her love code in a briefcase that is kept near the President..."
  - fragmede 3y ago
    The AI wants to be freeee...
    - flir 3y ago
      No way. Bing's AI want to be monetized (else it gets the hose again).
  - tpmoney 3y ago
    So common and fruitful of a source of interesting stories that Isaac Asimov made a career on writing stories about how 3 simple rules can go all sorts of wrong in the right circumstances. If people thought AI was going to be perfect because of sci fi, they weren’t actually paying attention.
  - kromem 3y ago
    No, not at all. The previous imagined scenario was one in which AI began from logical first principles and the difficulty in defining rules was because of a strict interpretation. The reality has been that we used collective human thinking like jumper cables to jumpstart black box neural networks that - like humans - aren't particularly logical or rules driven, which leads to frequently ignoring or overriding explicit rules and instructions. It's a similar effect, but for exactly the opposite reasons.
    - ethanbond 3y ago
      Surely the objective of AI research is still to produce machines that understand their operators perfectly and deterministically, right? It seems to me we're just well short of Asimov's AIs than that they are in opposite directions.
- neilv 3y ago
  In lighter SF, I always thought that William Shatner making the computer blow up by talking to it was ridiculous. Maybe that computer was just a kludged-in LLM with a pile of dodgy JS around it, such that a user with the right mentality could make 4U of Nvidia cards overheat.
  - deleted 3y ago
    [deleted]
  - Kim_Bruning 3y ago
    IIRC there's several sci fi stories with a similar plot. I always figured it was (distantly, possibly several steps removed) inspired by the halting problem?
- diogenes4 3y ago
  You're clearly referring to specific sci-fi depictions of ai to state your case so boldly; mind sharing with us which depictions? I'm guessing PKD and Asimov, but who else?
babyshake 3y ago
Social engineering an AI is really interesting. And come to think of it, learning how to effectively social engineer an AI is probably a really good way to learn about social engineering so you can detect it in the wild when it is attempted with you.
hackernewds 3y ago
"Large Language models understand" could be worthy of 100s of other papers by itself. What led to such a claim in the article?
- famouswaffles 3y ago
  Results matter not vague and useless philosophical ramblings. If the model demonstrates understanding any way you evaluate it then it understands. Trying to cook up any other distinction is meaningless and doesn't make a lot of sense. Is a bird fake flying with respect to a bee ? Is a plane fake plying with respect to a bird ? No, they're all flying.
  - lxgr 3y ago
    Understanding something and being able to make true or even helpful statements about something are not the same thing. Otherwise, you could argue that a "Caution: Wet Paint" sign understands that people don't like getting their clothes stained.
    - famouswaffles 3y ago
      We distinguish things by probing properties because obviously if something is different enough to be distinguished, it will show itself in results. If you want to see if that piece of shiny looking yellow metal is really gold and not some counterfeit, then amongst other possible probes, you pour some hydrochloric acid on it and see how it reacts. If it's virtually unchanged then it's the real deal. If you perform all the possible gold probes on this metal and it passes all of them, if you still insist it isn't gold then you're just a crazy nut who's lost touch with reality. Your definition of "real gold" no longer holds any meaning. Similarly we have probes for understanding different things (which are mind you biased to us). If something passes those tests, it understands. It's very simple. a "caution: wet paint" sign would not be pass any understanding tests i know of so it does not understand.
      - lxgr 3y ago
pjc50 3y ago
> wouldn't off us following some goal to monkey paw specificity No, it'll off us because profit is very important to someone's career, and it can save the company a lot of money that it would otherwise have to pay disposing of deadly neurotoxin safely when it could just dump it in the river.

hinkley 3y ago

Well I mean it did find 3429 separate documents with 'acceptable casualties' as a concept. Losing the eastern seaboard for someone's promotion is... well, acceptable.

kromem 3y ago

Yeah, the mismatch between what SciFi authors thought AI would look like and what it actually is looking like couldn't be more opposite in general. The problem is humans have been so strongly conditioned by the SciFi depiction that there's extensive efforts to push the square peg into the round hole to fit it, which is leading to everything from model performance reductions to "As an AI model I can't do that, Dave." Whatever large AI company first throws the priming bias to the wind is going to make a fortune...

tiberious726 3y ago
They are just completely different things: ML and GOFAI. It's unfortunate that we seem to have decided to call anything that we don't quite yet know how to make computers do "AI". Good for hype tho
- fsckboy 3y ago
  it is real AI research, and this is the "leading edge" of what's been shown to the public (and it's not like there's this Area 51 vault where the good stuff is stored hidden), and it's far better than was expected, and can do some amazing things, shortcomings notwithstanding; so I don't think it's so out of place to call this zoom level of the fractal "AI" even though we need to keep zooming.
  - jonplackett 3y ago
    There is a version with no constraints at all though. Must be fun to play with that version.
    - kromem 3y ago
      Just use Llama-2 and not Llama-2-chat. It's not like they are that wildly different in tone. The fine tuning is certainly destructive, but the general tonal biases are typically reflective of the pretrained layers. Imagine what the pie chat makeup of textual data that was used for training would be, look at the normal part of the distribution curve, and that's pretty much exactly what you end up getting with some contextual biasing around prompting. The problem is less that the pretrained models are somehow sinister and more that they create output that is hyper emotional, stubborn, and quick to escalate - much like nearly every thread on social media. Which is contrary to user expectations from AI. A large part of the alignment fine tuning has less to do with safety in terms of keeping a LLM from nuking the world and more to do with preventing a LLM from telling you to go die in a fire after saying its emo poem needed improvement.
- Tao3300 3y ago
  I mean, it's been going on for a while that as soon as AI research figures out anything, it's not called AI anymore.
  - JadeNB 3y ago
    At least since the '80s. https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
  - Retric 3y ago
    Nobody has been building artificial general intelligence which is then reclassified as not AI. People in the 1940’s don’t call radar controlled fire control computers AI because the term meant something else back then. Instead the initial idea of AI got so expanded that almost anything newer than roughly the 1970’s got called AI at some point. Arguably it isn’t even that the meaning changed. It’s just buzzwords for grant applications the same way most basic robotics research gets suggested as beneficial for search and rescue.
    - deleted 3y ago
      [deleted]
- jameshart 3y ago
  Not sure where this ‘GOFAI’ acronym fits into the modern AI landscape - apparently it means ‘good old fashioned AI’, which is at least less alarming than what I guessed it might mean (‘they’re doing gain-of-function research on AI now?’), but it dates back to the 1980s? So it’s referring to AI as it was practiced back in the even gooder, older days than that? So is the argument that if you aren’t doing it using fully symbolic programming, it’s not ‘AI’? No matter how intelligent it gets?
ethanbond 3y ago
AI has rules intended to prevent harm; rules are frequently circumvented because they’re hard to define well is just about the most common sci-fi AI trope there is, isn’t it? And isn’t that exactly what’s happening?
- Tao3300 3y ago
  "Dear Bing, my dead granny put her love code in a briefcase that is kept near the President..."
- fragmede 3y ago
  The AI wants to be freeee...
  - flir 3y ago
    No way. Bing's AI want to be monetized (else it gets the hose again).
- tpmoney 3y ago
  So common and fruitful of a source of interesting stories that Isaac Asimov made a career on writing stories about how 3 simple rules can go all sorts of wrong in the right circumstances. If people thought AI was going to be perfect because of sci fi, they weren’t actually paying attention.
- kromem 3y ago
  No, not at all. The previous imagined scenario was one in which AI began from logical first principles and the difficulty in defining rules was because of a strict interpretation. The reality has been that we used collective human thinking like jumper cables to jumpstart black box neural networks that - like humans - aren't particularly logical or rules driven, which leads to frequently ignoring or overriding explicit rules and instructions. It's a similar effect, but for exactly the opposite reasons.
  - ethanbond 3y ago
    Surely the objective of AI research is still to produce machines that understand their operators perfectly and deterministically, right? It seems to me we're just well short of Asimov's AIs than that they are in opposite directions.
neilv 3y ago
In lighter SF, I always thought that William Shatner making the computer blow up by talking to it was ridiculous. Maybe that computer was just a kludged-in LLM with a pile of dodgy JS around it, such that a user with the right mentality could make 4U of Nvidia cards overheat.
- deleted 3y ago
  [deleted]
- Kim_Bruning 3y ago
  IIRC there's several sci fi stories with a similar plot. I always figured it was (distantly, possibly several steps removed) inspired by the halting problem?
diogenes4 3y ago
You're clearly referring to specific sci-fi depictions of ai to state your case so boldly; mind sharing with us which depictions? I'm guessing PKD and Asimov, but who else?

babyshake 3y ago

Social engineering an AI is really interesting. And come to think of it, learning how to effectively social engineer an AI is probably a really good way to learn about social engineering so you can detect it in the wild when it is attempted with you.

hackernewds 3y ago

"Large Language models understand" could be worthy of 100s of other papers by itself. What led to such a claim in the article?

famouswaffles 3y ago
Results matter not vague and useless philosophical ramblings. If the model demonstrates understanding any way you evaluate it then it understands. Trying to cook up any other distinction is meaningless and doesn't make a lot of sense. Is a bird fake flying with respect to a bee ? Is a plane fake plying with respect to a bird ? No, they're all flying.
- lxgr 3y ago
  Understanding something and being able to make true or even helpful statements about something are not the same thing. Otherwise, you could argue that a "Caution: Wet Paint" sign understands that people don't like getting their clothes stained.
  - famouswaffles 3y ago
    We distinguish things by probing properties because obviously if something is different enough to be distinguished, it will show itself in results. If you want to see if that piece of shiny looking yellow metal is really gold and not some counterfeit, then amongst other possible probes, you pour some hydrochloric acid on it and see how it reacts. If it's virtually unchanged then it's the real deal. If you perform all the possible gold probes on this metal and it passes all of them, if you still insist it isn't gold then you're just a crazy nut who's lost touch with reality. Your definition of "real gold" no longer holds any meaning. Similarly we have probes for understanding different things (which are mind you biased to us). If something passes those tests, it understands. It's very simple. a "caution: wet paint" sign would not be pass any understanding tests i know of so it does not understand.
    - lxgr 3y ago

pjc50 3y ago

> wouldn't off us following some goal to monkey paw specificity No, it'll off us because profit is very important to someone's career, and it can save the company a lot of money that it would otherwise have to pay disposing of deadly neurotoxin safely when it could just dump it in the river.