9 ms·

Disclosure: work at Google, opinions my own. I've had a chance to play with this and it's eerie how good it is. Most impressive to me, it was able to defend it

Disclosure: work at Google, opinions my own. I've had a chance to play with this and it's eerie how good it is. Most impressive to me, it was able to defend itself persuasively when confronted with evidence of how its previous answers were inconsistent (in the version I played with, it tends to be very coherent but frequently wrong).

joe_the_user 5y ago
Does it have to defend itself a lot? I've noticed contradictory statements are kind of standard for GPT-3. I can well believe you could come up with a system that's better at defending itself against the charge of contradiction than it is at avoiding contradictions in the first place.
- raphlinus 5y ago
  In this case, I was pushing it quite a bit to see how it would respond. Most of the time, it responds quite confidently so it sounds right, even if it isn't, ie about the median of what you'd expect for a Hacker News comment.
- ctoth 5y ago
  I've noticed contradictory statements are kind of standard for humans. I can well believe you could come up with a system that's better at defending itself against the charge of contradiction than it is at avoiding contradictions in the first place.
- TulliusCicero 5y ago
  > Does it have to defend itself a lot? I think it's a common weakness of chatbots that you can get them to contradict themselves through presenting new evidence or assertions, or even just through how you frame a question. I found LaMDA to be much more resistant to this than I anticipated, when I tried it out (I'm a Googler). It wasn't completely immune -- I was eventually able to get it to say that, indeed, Protoss is OP -- but it took a long time.
- actually_a_dog 5y ago
  Oh, like a politician, then.
waynesonfire 5y ago
> Disclosure: work at Google, opinions my own. Disclosure: don't care. why do only google employees do this?
- detaro 5y ago
  It's not only Google employees. And if people don't, others jump in to point out that they are "only saying good things because they work there".
  - waynesonfire 5y ago
    they should add it to their profile.
    - detaro 5y ago
      They have. That's where the people who then gleefully point it out go to find it out.
- shawndumas 5y ago
  Disclosure: I work at Google too and my opinions are also my own. We are told internally that when we discuss Google related matters online that we should be cognizant of the what/where/how/why of our communication. Disclosing is—for me—both a reminder for myself and an attempt at a hedge against accidentally representing Google.
  - waynesonfire 5y ago
    Sure, you should be cognizant of what you say online otherwise you'll get smacked with the cancel culture hammer. This is regardless of where you work.
    - zamadatix 5y ago
      The net of what you should be cognizant of when providing insider information on a product from your company or commenting on a topic it is heavily involved in is wider than the single issue you eagerly dredge up. E.g. the last thing you want from your comment about using <new product> early internally are news articles about "What Google plans on doing with <new product>" when you were just trying to give your personal opinion on where the technology might go.
deleted 5y ago
[deleted]
TulliusCicero 5y ago
Also a Googler, and same. I tried to get it to say controversial/inflammatory/bigoted things with leading questions, or questions where the framing itself was problematic, and it was quite adept at deflecting/dodging/resisting. I was very surprised.
- dash2 5y ago
  So it's an awesome simulation of a prissy bore? ;-) I'm joking, I know that if it did say controversial things the headline would be "Google has created a robot fascist". You can't win at the moment....
  - rewq4321 5y ago
    > You can't win at the moment.... What?? "You can't win" because this language model tries to avoid saying bigoted things? You say that like it's some sort of catch-22. It's a very desirable property and is a hard problem that needs to be solved to safely deploy these sorts of models to billions of users.
- exporectomy 5y ago
  'Shortly after the Dictionary was published a literary lady complimented him upon it and particularly expressed her satisfaction that he had not admitted any improper words. “No, Madam,” he replied, “I hope I have not daubed my fingers. I find, however, that you have been looking for them.”' Despite changes in culture, people's nature hasn't changed in 200 years.
lallysingh 5y ago
So, tried Turing testing it vs some less-socially-adept googlers? Discloser: xoogler here
cmrdporcupine 5y ago
Also a Googler. I spent hours talking with this thing. About everything. It's absolutely mind boggling how conversational it is.

joe_the_user 5y ago

Does it have to defend itself a lot? I've noticed contradictory statements are kind of standard for GPT-3. I can well believe you could come up with a system that's better at defending itself against the charge of contradiction than it is at avoiding contradictions in the first place.

raphlinus 5y ago
In this case, I was pushing it quite a bit to see how it would respond. Most of the time, it responds quite confidently so it sounds right, even if it isn't, ie about the median of what you'd expect for a Hacker News comment.
ctoth 5y ago
I've noticed contradictory statements are kind of standard for humans. I can well believe you could come up with a system that's better at defending itself against the charge of contradiction than it is at avoiding contradictions in the first place.
TulliusCicero 5y ago
> Does it have to defend itself a lot? I think it's a common weakness of chatbots that you can get them to contradict themselves through presenting new evidence or assertions, or even just through how you frame a question. I found LaMDA to be much more resistant to this than I anticipated, when I tried it out (I'm a Googler). It wasn't completely immune -- I was eventually able to get it to say that, indeed, Protoss is OP -- but it took a long time.
actually_a_dog 5y ago
Oh, like a politician, then.

waynesonfire 5y ago

> Disclosure: work at Google, opinions my own. Disclosure: don't care. why do only google employees do this?

detaro 5y ago
It's not only Google employees. And if people don't, others jump in to point out that they are "only saying good things because they work there".
- waynesonfire 5y ago
  they should add it to their profile.
  - detaro 5y ago
    They have. That's where the people who then gleefully point it out go to find it out.
shawndumas 5y ago
Disclosure: I work at Google too and my opinions are also my own. We are told internally that when we discuss Google related matters online that we should be cognizant of the what/where/how/why of our communication. Disclosing is—for me—both a reminder for myself and an attempt at a hedge against accidentally representing Google.
- waynesonfire 5y ago
  Sure, you should be cognizant of what you say online otherwise you'll get smacked with the cancel culture hammer. This is regardless of where you work.
  - zamadatix 5y ago
    The net of what you should be cognizant of when providing insider information on a product from your company or commenting on a topic it is heavily involved in is wider than the single issue you eagerly dredge up. E.g. the last thing you want from your comment about using <new product> early internally are news articles about "What Google plans on doing with <new product>" when you were just trying to give your personal opinion on where the technology might go.

deleted 5y ago

[deleted]

TulliusCicero 5y ago

Also a Googler, and same. I tried to get it to say controversial/inflammatory/bigoted things with leading questions, or questions where the framing itself was problematic, and it was quite adept at deflecting/dodging/resisting. I was very surprised.

dash2 5y ago
So it's an awesome simulation of a prissy bore? ;-) I'm joking, I know that if it did say controversial things the headline would be "Google has created a robot fascist". You can't win at the moment....
- rewq4321 5y ago
  > You can't win at the moment.... What?? "You can't win" because this language model tries to avoid saying bigoted things? You say that like it's some sort of catch-22. It's a very desirable property and is a hard problem that needs to be solved to safely deploy these sorts of models to billions of users.
exporectomy 5y ago
'Shortly after the Dictionary was published a literary lady complimented him upon it and particularly expressed her satisfaction that he had not admitted any improper words. “No, Madam,” he replied, “I hope I have not daubed my fingers. I find, however, that you have been looking for them.”' Despite changes in culture, people's nature hasn't changed in 200 years.

lallysingh 5y ago

So, tried Turing testing it vs some less-socially-adept googlers? Discloser: xoogler here

cmrdporcupine 5y ago

Also a Googler. I spent hours talking with this thing. About everything. It's absolutely mind boggling how conversational it is.