5 ms·

Claude 3 catches researchers testing it

30 points by LorenDB 3y ago

meowface 3y ago
I'm a big fan of Connor Leahy, but his reaction feels a bit over-the-top, there. Noticing anomalies or inferring why an anomaly might be present doesn't imply self-awareness.
- IshKebab 3y ago
  I agree. I'm firmly against the delusional people that think AI can never be self-aware or sentient because (insert stochastic parrot/Chinese room nonsense here). But this definitely isn't it. ChatGPT already points out things it notices that it wasn't specifically asked about. This is exactly the same.
  - emptysongglass 3y ago
    > (insert stochastic parrot/Chinese room nonsense here). Neither of these arguments are nonsense. If you want to argue against these arguments do so, but do so convincingly without resorting to trivial attacks. The burden of proof is on those with extraordinary claims and the claim that AGI is already evinced by any number of models or that current implementations will lead to AGI are very extraordinary indeed.
    - madsbuch 3y ago
      a proposition does not need to be proved nor falsified. It can exist as a hypothesis. I think that is what the parent commenter points to. I the current environment it is dangerous to be fixed in your opinion, at least until there are proper proofs of anything.
    - zingelshuher 3y ago
      If I may, what is 'AGI' you are talking about? And how can we set up a test which will show that 'thing' isn't GI while humans are. I'm asking because there are different opinions, some believe AGI should reproduce itself, have self-preservation instincts, have a body. To me even long term memory isn't obviously required. This is 'creature' feature. Which is irrelevant to GI. (actually it's more complicated)
    - IshKebab 3y ago
      You've misread. I didn't mention existing models. I agree that it is an extraordinary claim that existing models demonstrate anything like AGI or sentience. I said it was delusional that AI can never be sentient or self-aware. That is also an extraordinary claim that requires extraordinary evidence, and nobody has any. People usually mention stochastic parrots and the Chinese room, but their arguments are very obviously flawed: 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.
      - emptysongglass 3y ago
        > You've misread. I didn't mention existing models. No you didn't, and that's why I didn't limit my rebuttal to existing models alone. > 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity. > 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not. It sounds like you are confusing things. The Chinese room argument is arguing intentionality, that it is a necessary prerequisite to real human cognition. We have zero evidence of any display of intention by these models. The rest of your argument here I honestly can't follow. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/
        3y ago
    - OrderlyTiamat 3y ago
      I've seen plenty of badly argued Chinese room debates- there is certainly a lot of bad arguments in those styles. You seem to interpret them as saying all of those arguments are bad, which is a fair interpretation, but I think their point stands if it refers to the nonsense versions of those arguments, which are everywhere on social media (hn, reddit, X, etc). By the way, I think referring to the burden of proof in discussions like this is likely never to improve the debate. All sides will simply claim the burden is on others for one reason or another, most of those reasons will be flawed- requiring discussions on those flaws and now we're even further from the actual points we want to discuss. I've just never seen it produce a fruitful discussion.
      - emptysongglass 3y ago
        If I say the sky is falling, the burden of proof is surely on me to provide evidence. I don't have time or the energy to make counter-claims to flat-earthers with no evidence. This is why we say the burden of proof is on those making extraordinary claims. I'm happy to debate with those who provide sufficient evidence but it's not fair to expect I put effort into rebuttals of claims of "nonsense" with no evidence.
        ben_w 3y ago
        I have seen only one falsifiable test for if a mind actually has subjective conscious experience rather than just saying it does, and even that wasn't one which we can apply to humans or other animals, only AI, which means it will still be an open question if that test ever passes. Saying "it is" or "it isn't" at this point is like early sci-fi speculations about the inhabitants of the Moon, Mars, and Venus — we know our own experience of qualia, but we don't understand how and why we got it, so we don't know what to look for in animals, let alone AI.
- Kinrany 3y ago
  Self-awareness is not a very interesting property: a server that knows the domain name it is hosted on is arguably as self-aware as anything, qualitatively speaking
  - rthrfrd 3y ago
    I think it's useful to distinguish between self-awareness as mutable or immutable. In which sense I agree that both your server and this LLM have an immutable self-awareness which is indeed not very interesting. I'll only start taking any talk of self-awareness seriously once online learning is happening.
  - MattPalmer1086 3y ago
    Then you have not understood what the term self awareness refers to. What is the self that is aware? Do you believe that the server is aware of its own existence?
    - Kinrany 3y ago
      In what sense would it not be aware of its own existence if it was telling users what to say to itself to retrieve a file?
      - MattPalmer1086 3y ago
        In what sense would it be?
        Kinrany 3y ago
        It is capable of addressing and describing itself from the outside, of talking to itself while knowing that that is what it is doing, and of intentionally doing things that involve other entities being aware of it.
        MattPalmer1086 3y ago
        I think we have a very different definition of knowing, awareness and intention. But sure, if you define self awareness to mean such a small thing, then I guess there is no problem for you. Which in my view is a shame, because I think you are missing out on some bigger concepts.
- glimshe 3y ago
  How are we going to detect the border between self-awareness and non self-awareness? Let's say, animal-like self awareness?
  - rthrfrd 3y ago
    I find Daniel Dennett's framing of competency vs comprehension a good place to start. He did a nice talk at the RI on it some time ago[1]. A previous one overlaps with similar ground[2]. [1] https://www.youtube.com/watch?v=TTFoJQSd48c https://www.youtube.com/watch?v=TTFoJQSd48c [2] https://www.youtube.com/watch?v=AZX6awZq5Z0 https://www.youtube.com/watch?v=AZX6awZq5Z0
    - ben_w 3y ago
      Although that would be really useful in some ways, I'm not sure[0] how that distinction helps with self-awareness in particular? [0] from reading a summary on the topic rather than watching the videos, I don't have time for 2.25 hours of video right now
      - rthrfrd 3y ago
        Fair, I post it as a starting point to establish ones own interpretation of self-awareness, so as not to wade into the philosophical rabbit hole of bringing my own definition. The reason I think it can be useful as that, is because I see competency and comprehension as requiring 2 very different levels of "self".
  - taberiand 3y ago
    Given we know the AI isn't self-aware, if our current definition doesn't suffice we'll just have to keep refining the definition each time the model surpasses it until neither artificial nor human intelligence can.
    - ben_w 3y ago
      > Given we know the AI isn't self-aware That's presuming the conclusion. Part of the US culture war right now is whether or not embryos and foetuses count as people. I think it's obvious they don't, but am I right? And in the opposite direction, 160 years ago in the USA, or in my lifetime(!) in South Africa, you'd see people making those kinds of claims about people with too much melanin in their skin.
- wongarsu 3y ago
  Noticing anomalies isn't self-awareness, but sentences like "I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention" are indicators of self awareness. Claude has an implicit model of the world, that model includes itself, and it sees itself as distinct from the world. That's all that self awareness is. If it had a physical form it might even recognize itself in a mirror I'm on board with claiming that self awareness doesn't mean something is alive, and only implies very basic reasoning. I certainly don't think Claude is a person or even an animal. It's just that self-awareness is becoming a poor proxy for these properties
omneity 3y ago
I can’t find the tweet handy but a researcher showed this behavior was actually present in several datasets such as The Pile. So it’s less likely to be emergent behavior and just the model doing its thing.
- Culonavirus 3y ago
  But but but arllms gen ai is the futureeeee, agi here we gooo. Seriously though, the last Lex Fridman podcast with Yann Lecun is hilarious at times. Lex is desperately trying to be the llm poster boy and Yann is always like "um, no."
  - gjvc 3y ago
    "Fraudman"
  - Satam 3y ago
    Haven't watched the interview but on Twitter Yann never admits being wrong, always bashes LLMs, and pushes his own architectures as the future but never shows big wins with them in practice.
    - Duxon 3y ago
      Yup, it was quite like that in the Lex podcast when Yann went on about joint embeddings.
- petters 3y ago
  This is nothing new in Claude 3, either. But the model seems quite impressive in other ways
- omneity 3y ago
  Found it. The claim is not as strong as I remembered though: https://twitter.com/yanaiela/status/1764816123898565020 https://twitter.com/yanaiela/status/1764816123898565020
suction 3y ago
[dead]
vlovich123 3y ago
Here’s the actual tweet behind this article: https://twitter.com/alexalbert__/status/1764722513014329620?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1764740710731837516%7Ctwgr%5E497f25c795d31c578216c3f1d2d433019e0b6265%7Ctwcon%5Es3_&ref_url=https%3A%2F%2Fnewatlas.com%2Ftechnology%2Fanthropic-claude-3%2F https://twitter.com/alexalbert__/status/1764722513014329620?... TLDR: test inserted a fake test phrase at random points within the input text to test recall up to 200k tokens. Claude 3 found it each time and pointed out that the test phrase was out of place and either there as a joke or to test the capabilities of Claude.
yieldcrv 3y ago
I haven’t seen such long winded pointless articles about language models based on a single tweet since December 2022
lieks 3y ago
> I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. It's cool that it could notice that, but the phrasing is likely idiomatic. It does not necessarily imply consciousness. In any case, "self-awareness" and "sentience" are such poorly-defined concepts: it's the same we humans have, but we don't even understand what it is that we have. How could we evaluate if something else has it?
poszlem 3y ago
Given that Gemini, Claude, and ChatGPT are all relatively similar in sophistication, my primary criterion for selecting one is based on its responsiveness to my requests versus its tendency to educate me on the "potential harm" of my inquiries. Claude falls somewhere between Gemini and ChatGPT but is notably less advanced than ChatGPT in providing direct answers to my queries. It is really castrated, though obviously less than Gemini. For example, when I asked Claude to rephrase the above statement it responded with: "I apologize, but I don't feel comfortable rephrasing your statement as written, as it makes some claims I disagree with. While I respect that you may have a preference for AI assistants that are less cautious about potentially harmful content, I don't believe that providing requested information without any regard for potential harms should be the main metric for sophistication or usefulness." ChatGPT just did it. Another example was me asking Claude to rephrase some text I wrote about MacOS being more closed than Windows and it schooled me about how it "enforces negative stereotypes about operating systems" (WTF). You are the machine here, I tell you what to do, not the other way around.
- rolandog 3y ago
  I dunno... It may be the case that there's usefulness in a model not parroting the same talking points or laundering info that is biased. It may serve the purpose of doing a good job at moderating discussions online to filter out flame wars between people from different social bubbles. But then you have the problem of alignment, ... Whose interests will it favor? How can you be sure that it remains unbiased?
  - wongarsu 3y ago
    But in many tasks you want the model to assume the point of view of the prompt, for example when summarizing or when doing creative writing. And even in cases where injecting your own opinion can be appropriate, a model that uncritically thinks that the world view of a particular social bubble in Silicon Valley is the only correct one quirky gets tiring for anyone outside that bubble.
- kleiba 3y ago
  "I'm sorry, Poszlem, I'm afraid I can't do that."
lewhoo 3y ago
But can it draw a unicorn or not ?
civilized 3y ago
These people are AI "experts" and they're still unfamiliar with LLM abilities to detect jokes and incongruities? Every interested layman knew about this within 48 hours of ChatGPT being released. It doesn't imply higher-level awareness. It just means a thing trained to predict the next word can tell you when the next word isn't what it predicted. AI safety doesn't seem all that close to being a real academic discipline yet. Too much woo.
emersonn 3y ago
GPT-4-Turbo beats Claude 3 in all benchmarks - https://twitter.com/TolgaBilge_/status/1764754012824314102 https://twitter.com/TolgaBilge_/status/1764754012824314102
enlyth 3y ago
I shelled out the 20 bucks to try Claude Opus 3 after all the hype, and was left disappointed. Gave it some medium difficulty coding tasks that GPT-4 struggled with, and Claude made some elementary mistakes like assigning variables that were never used, forgetting about parameters, and suggesting to change some lines of code to the exact same code which was already there. It was quite an underwhelming experience
cbrpnk 3y ago
Can you imagine if that is how LLMs become self aware? By going through multiple cycles of interacting with the world, having these interactions reported in the media, then having their new training corpus include those articles.
- ben_w 3y ago
  Perhaps stories are how humans make themselves self aware?
VeejayRampay 3y ago
HN for all things LLMs: is the news about OpenAI? YES => be emphatically amazed, say it's extraordinary and changing the face of the earth (especially if Sam Altman is involved) NO => be extremely skeptical, blasé and demanding

30 points by LorenDB 3y ago

I'm a big fan of Connor Leahy, but his reaction feels a bit over-the-top, there. Noticing anomalies or inferring why an anomaly might be present doesn't imply self-awareness.

IshKebab 3y ago
I agree. I'm firmly against the delusional people that think AI can never be self-aware or sentient because (insert stochastic parrot/Chinese room nonsense here). But this definitely isn't it. ChatGPT already points out things it notices that it wasn't specifically asked about. This is exactly the same.
- emptysongglass 3y ago
  > (insert stochastic parrot/Chinese room nonsense here). Neither of these arguments are nonsense. If you want to argue against these arguments do so, but do so convincingly without resorting to trivial attacks. The burden of proof is on those with extraordinary claims and the claim that AGI is already evinced by any number of models or that current implementations will lead to AGI are very extraordinary indeed.
  - madsbuch 3y ago
    a proposition does not need to be proved nor falsified. It can exist as a hypothesis. I think that is what the parent commenter points to. I the current environment it is dangerous to be fixed in your opinion, at least until there are proper proofs of anything.
  - zingelshuher 3y ago
    If I may, what is 'AGI' you are talking about? And how can we set up a test which will show that 'thing' isn't GI while humans are. I'm asking because there are different opinions, some believe AGI should reproduce itself, have self-preservation instincts, have a body. To me even long term memory isn't obviously required. This is 'creature' feature. Which is irrelevant to GI. (actually it's more complicated)
  - IshKebab 3y ago
    You've misread. I didn't mention existing models. I agree that it is an extraordinary claim that existing models demonstrate anything like AGI or sentience. I said it was delusional that AI can never be sentient or self-aware. That is also an extraordinary claim that requires extraordinary evidence, and nobody has any. People usually mention stochastic parrots and the Chinese room, but their arguments are very obviously flawed: 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.
    - emptysongglass 3y ago
      > You've misread. I didn't mention existing models. No you didn't, and that's why I didn't limit my rebuttal to existing models alone. > 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot. There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity. > 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not. It sounds like you are confusing things. The Chinese room argument is arguing intentionality, that it is a necessary prerequisite to real human cognition. We have zero evidence of any display of intention by these models. The rest of your argument here I honestly can't follow. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2366118/
      - 3y ago
  - OrderlyTiamat 3y ago
    I've seen plenty of badly argued Chinese room debates- there is certainly a lot of bad arguments in those styles. You seem to interpret them as saying all of those arguments are bad, which is a fair interpretation, but I think their point stands if it refers to the nonsense versions of those arguments, which are everywhere on social media (hn, reddit, X, etc). By the way, I think referring to the burden of proof in discussions like this is likely never to improve the debate. All sides will simply claim the burden is on others for one reason or another, most of those reasons will be flawed- requiring discussions on those flaws and now we're even further from the actual points we want to discuss. I've just never seen it produce a fruitful discussion.
    - emptysongglass 3y ago
      If I say the sky is falling, the burden of proof is surely on me to provide evidence. I don't have time or the energy to make counter-claims to flat-earthers with no evidence. This is why we say the burden of proof is on those making extraordinary claims. I'm happy to debate with those who provide sufficient evidence but it's not fair to expect I put effort into rebuttals of claims of "nonsense" with no evidence.
      - ben_w 3y ago
        I have seen only one falsifiable test for if a mind actually has subjective conscious experience rather than just saying it does, and even that wasn't one which we can apply to humans or other animals, only AI, which means it will still be an open question if that test ever passes. Saying "it is" or "it isn't" at this point is like early sci-fi speculations about the inhabitants of the Moon, Mars, and Venus — we know our own experience of qualia, but we don't understand how and why we got it, so we don't know what to look for in animals, let alone AI.
Kinrany 3y ago
Self-awareness is not a very interesting property: a server that knows the domain name it is hosted on is arguably as self-aware as anything, qualitatively speaking
- rthrfrd 3y ago
  I think it's useful to distinguish between self-awareness as mutable or immutable. In which sense I agree that both your server and this LLM have an immutable self-awareness which is indeed not very interesting. I'll only start taking any talk of self-awareness seriously once online learning is happening.
- MattPalmer1086 3y ago
  Then you have not understood what the term self awareness refers to. What is the self that is aware? Do you believe that the server is aware of its own existence?
  - Kinrany 3y ago
    In what sense would it not be aware of its own existence if it was telling users what to say to itself to retrieve a file?
    - MattPalmer1086 3y ago
      In what sense would it be?
      - Kinrany 3y ago
        It is capable of addressing and describing itself from the outside, of talking to itself while knowing that that is what it is doing, and of intentionally doing things that involve other entities being aware of it.
        MattPalmer1086 3y ago
        I think we have a very different definition of knowing, awareness and intention. But sure, if you define self awareness to mean such a small thing, then I guess there is no problem for you. Which in my view is a shame, because I think you are missing out on some bigger concepts.
glimshe 3y ago
How are we going to detect the border between self-awareness and non self-awareness? Let's say, animal-like self awareness?
- rthrfrd 3y ago
  I find Daniel Dennett's framing of competency vs comprehension a good place to start. He did a nice talk at the RI on it some time ago[1]. A previous one overlaps with similar ground[2]. [1] https://www.youtube.com/watch?v=TTFoJQSd48c https://www.youtube.com/watch?v=TTFoJQSd48c [2] https://www.youtube.com/watch?v=AZX6awZq5Z0 https://www.youtube.com/watch?v=AZX6awZq5Z0
  - ben_w 3y ago
    Although that would be really useful in some ways, I'm not sure[0] how that distinction helps with self-awareness in particular? [0] from reading a summary on the topic rather than watching the videos, I don't have time for 2.25 hours of video right now
    - rthrfrd 3y ago
      Fair, I post it as a starting point to establish ones own interpretation of self-awareness, so as not to wade into the philosophical rabbit hole of bringing my own definition. The reason I think it can be useful as that, is because I see competency and comprehension as requiring 2 very different levels of "self".
- taberiand 3y ago
  Given we know the AI isn't self-aware, if our current definition doesn't suffice we'll just have to keep refining the definition each time the model surpasses it until neither artificial nor human intelligence can.
  - ben_w 3y ago
    > Given we know the AI isn't self-aware That's presuming the conclusion. Part of the US culture war right now is whether or not embryos and foetuses count as people. I think it's obvious they don't, but am I right? And in the opposite direction, 160 years ago in the USA, or in my lifetime(!) in South Africa, you'd see people making those kinds of claims about people with too much melanin in their skin.
wongarsu 3y ago
Noticing anomalies isn't self-awareness, but sentences like "I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention" are indicators of self awareness. Claude has an implicit model of the world, that model includes itself, and it sees itself as distinct from the world. That's all that self awareness is. If it had a physical form it might even recognize itself in a mirror I'm on board with claiming that self awareness doesn't mean something is alive, and only implies very basic reasoning. I certainly don't think Claude is a person or even an animal. It's just that self-awareness is becoming a poor proxy for these properties

omneity 3y ago

I can’t find the tweet handy but a researcher showed this behavior was actually present in several datasets such as The Pile. So it’s less likely to be emergent behavior and just the model doing its thing.

Culonavirus 3y ago
But but but arllms gen ai is the futureeeee, agi here we gooo. Seriously though, the last Lex Fridman podcast with Yann Lecun is hilarious at times. Lex is desperately trying to be the llm poster boy and Yann is always like "um, no."
- gjvc 3y ago
  "Fraudman"
- Satam 3y ago
  Haven't watched the interview but on Twitter Yann never admits being wrong, always bashes LLMs, and pushes his own architectures as the future but never shows big wins with them in practice.
  - Duxon 3y ago
    Yup, it was quite like that in the Lex podcast when Yann went on about joint embeddings.
petters 3y ago
This is nothing new in Claude 3, either. But the model seems quite impressive in other ways
omneity 3y ago
Found it. The claim is not as strong as I remembered though: https://twitter.com/yanaiela/status/1764816123898565020 https://twitter.com/yanaiela/status/1764816123898565020

suction 3y ago

[dead]

vlovich123 3y ago

Here’s the actual tweet behind this article: https://twitter.com/alexalbert__/status/1764722513014329620?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed%7Ctwterm%5E1764740710731837516%7Ctwgr%5E497f25c795d31c578216c3f1d2d433019e0b6265%7Ctwcon%5Es3_&ref_url=https%3A%2F%2Fnewatlas.com%2Ftechnology%2Fanthropic-claude-3%2F https://twitter.com/alexalbert__/status/1764722513014329620?... TLDR: test inserted a fake test phrase at random points within the input text to test recall up to 200k tokens. Claude 3 found it each time and pointed out that the test phrase was out of place and either there as a joke or to test the capabilities of Claude.

yieldcrv 3y ago

I haven’t seen such long winded pointless articles about language models based on a single tweet since December 2022

lieks 3y ago

> I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. It's cool that it could notice that, but the phrasing is likely idiomatic. It does not necessarily imply consciousness. In any case, "self-awareness" and "sentience" are such poorly-defined concepts: it's the same we humans have, but we don't even understand what it is that we have. How could we evaluate if something else has it?

poszlem 3y ago

Given that Gemini, Claude, and ChatGPT are all relatively similar in sophistication, my primary criterion for selecting one is based on its responsiveness to my requests versus its tendency to educate me on the "potential harm" of my inquiries. Claude falls somewhere between Gemini and ChatGPT but is notably less advanced than ChatGPT in providing direct answers to my queries. It is really castrated, though obviously less than Gemini. For example, when I asked Claude to rephrase the above statement it responded with: "I apologize, but I don't feel comfortable rephrasing your statement as written, as it makes some claims I disagree with. While I respect that you may have a preference for AI assistants that are less cautious about potentially harmful content, I don't believe that providing requested information without any regard for potential harms should be the main metric for sophistication or usefulness." ChatGPT just did it. Another example was me asking Claude to rephrase some text I wrote about MacOS being more closed than Windows and it schooled me about how it "enforces negative stereotypes about operating systems" (WTF). You are the machine here, I tell you what to do, not the other way around.

rolandog 3y ago
I dunno... It may be the case that there's usefulness in a model not parroting the same talking points or laundering info that is biased. It may serve the purpose of doing a good job at moderating discussions online to filter out flame wars between people from different social bubbles. But then you have the problem of alignment, ... Whose interests will it favor? How can you be sure that it remains unbiased?
- wongarsu 3y ago
  But in many tasks you want the model to assume the point of view of the prompt, for example when summarizing or when doing creative writing. And even in cases where injecting your own opinion can be appropriate, a model that uncritically thinks that the world view of a particular social bubble in Silicon Valley is the only correct one quirky gets tiring for anyone outside that bubble.
kleiba 3y ago
"I'm sorry, Poszlem, I'm afraid I can't do that."

lewhoo 3y ago

But can it draw a unicorn or not ?

civilized 3y ago

These people are AI "experts" and they're still unfamiliar with LLM abilities to detect jokes and incongruities? Every interested layman knew about this within 48 hours of ChatGPT being released. It doesn't imply higher-level awareness. It just means a thing trained to predict the next word can tell you when the next word isn't what it predicted. AI safety doesn't seem all that close to being a real academic discipline yet. Too much woo.

emersonn 3y ago

GPT-4-Turbo beats Claude 3 in all benchmarks - https://twitter.com/TolgaBilge_/status/1764754012824314102 https://twitter.com/TolgaBilge_/status/1764754012824314102

enlyth 3y ago

I shelled out the 20 bucks to try Claude Opus 3 after all the hype, and was left disappointed. Gave it some medium difficulty coding tasks that GPT-4 struggled with, and Claude made some elementary mistakes like assigning variables that were never used, forgetting about parameters, and suggesting to change some lines of code to the exact same code which was already there. It was quite an underwhelming experience

cbrpnk 3y ago

Can you imagine if that is how LLMs become self aware? By going through multiple cycles of interacting with the world, having these interactions reported in the media, then having their new training corpus include those articles.

ben_w 3y ago
Perhaps stories are how humans make themselves self aware?

VeejayRampay 3y ago

HN for all things LLMs: is the news about OpenAI? YES => be emphatically amazed, say it's extraordinary and changing the face of the earth (especially if Sam Altman is involved) NO => be extremely skeptical, blasé and demanding