7 ms·

The biggest mistake with AI is making it appear human. It is not human. It will not behave like a human. It will behave as it was trained or modeled to behave.

by nicce 2y ago

The biggest mistake with AI is making it appear human. It is not human. It will not behave like a human. It will behave as it was trained or modeled to behave. It will behave according to the intentions of its creators. If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human.

falcor84 2y ago
> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. I've been treating LLM's as yet another voice on the internet - of course I wouldn't "trust" such a random person on the internet to be correct in their writing, let alone to act in my best interest - but so far, LLM's have been much more to my liking on both accounts than the random human netizen.
ben_w 2y ago
Well, mostly. But not "It will not behave like a human. It will behave as it was trained or modeled to behave." — that the latter is in these cases also the former, means it does behave (to a degree) like a human, and that is why it will be trusted. The caveat of "to a degree" being another way to phrase agreement with you that it will be an error to trust it. But it will be trusted. These models are already given more trust than they deserve.
- nicce 2y ago
  That is better worded. What I meant is that the "humanity" part is faked, and eventually, it will act as programmed to, without compromises or any moral weight.
simonw 2y ago
The audio conversation that accompanies this piece talks about that a little, at 9:21: https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s If you create a chatbot with no evidence of personality at all, there is a very real risk that people will assume it is a completely unbiased, robotic, potentially infallible machine. But LLMs are not that! They have all kinds of biases baked into them. Giving them a personality may be a good way to help people understand that.
- hackernewds 2y ago
  You both said the polar opposite things, neither providing any evidence past conjecture. Who is correct?
  - simonw 2y ago
    I didn't express my own personal opinion, I tried to summarize the opinion given by the Anthropic researchers. I haven't made up my mind about this yet. I find the Anthropic view really interesting.
    - nicce 2y ago
      It is very difficult to say. I see the point from the video too, but it is all about the general baseline. For us, it might not matter, because we are very aware about these things. But most, and especially for the upcoming generations, they don't think like us. The baseline is defined how they are used to "robots". What are they expecting? To whom they are comparing? People who don't know how internals works on these things, might tend to up trust more "human" version. At least I would assume so. It is easier to interact with them and easier to like. People who know how these works internally, might be hesitant, and trust more for "neutral one". And the comment in the video is over-engineering the solution for the problem of the minority.
  - nightowl_games 2y ago
    Welcome to the new era of AI, where there is no objective truth and no one knows what they are talking about.
    - idiotsecant 2y ago
      'The new era of AI'? Sounds like the universe in general to me. You must be new here. Enjoy your stay. Avoid Florida if you can.
    - Art9681 2y ago
      I remember the same thing was said ~15 years ago when social media became ubiquitous. And 15 before that, when AOL was ubiquitous. And 15 before that when...
startupsfail 2y ago
I’m not sure, but aligning for helpfulness and absence of self, like OpenAI did, could be a right move. It had been known for some time that selfish behaviors are a source of a lot of unhappiness, greed, etc. And the absence of self, absence of I, absence of character tends to fix that.
Zambyte 2y ago
> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. I actually really disagree with this. I think it's easier to distrust things that are human like. We are used to distrust humans. Less so things that seem like neutral tools. For example, people mindlessly fill out forms presented by large corporations asking for personal information. I think people would be less inclined to trust an LLM with that information, even if it it's actually likely to be less actionable (at least with current capabilities).
- nicce 2y ago
  In general I would agree. But AI can take the all the positive traits from humans to sound as likeable and trustable as possible (and companies curretly do that). Social engineering is a thing. We like and trust some people, and some we don’t, without any evidence about their behavior history. And, doesn’t your example conflict your intial claim? Because people trust humans, they send forms.
  - Zambyte 2y ago
    In my example, I don't think people would give that information away as easily if not for the form mediating it. I don't think a language model would be as convincing as a simple form. I say this from experience - even using a language model running on my own machine, it sets off my internal alarms when I tell it private information, even though I know literally no human besides me will ever see it.
- LeonB 2y ago
  This is a great idea for an experiment. I’m not sure how it would go. Some people would refuse to give info to both the simple form and the personality rich chat bot. Some would give info to both. But how big is the middle area? One thing is certain though — whatever format is more successful at getting peoples info — that’s the one that you’d see more and more of over time.
  - r2_pilot 2y ago
hackernewds 2y ago
This actually aligns spot on with DeepMind's research arguing why AI should not be anthropomorphic
- astromaniak 2y ago
  They have the right to have their opinion and build it into their products. But how about having other opinions and products too? Imagine in hospital, or care houses, patients will be more comfortable with human-like AI. Or AIG when it happens. As for trust, is it a bad thing? Anyway, humans' mind is speculative, biased, and prone to brainwashing. Looks like common thing with machines. So, trust, if it's bad, can be corrected by a few movies picturing evil AI. Pretty much like fear of clowns was created. Actually Terminator already created a wave of AI doomers.
  - nicce 2y ago
    > So, trust, if it's bad, can be corrected by a few movies picturing evil AI. There are many steps between good and evil. Like giving biased product recommendation so that people buy it, or never mentioning things like Tiananmen Square.
tkgally 2y ago
> The biggest mistake with AI is making it appear human. I’m guessing that that mistake is already being made on a large scale intentionally, not by OpenAI or Anthropic, perhaps, but by companies developing virtual boyfriends and girlfriends and other emotionally sticky bots. I have avoided trying such services myself so far. Does anybody here have any recent experience with them?
mvkel 2y ago
Making it "appear human" is an input heuristic. I don't want to have to know some special syntax to work with it. It's a novel feature, not a problem.
- nicce 2y ago
  Input is not the problem, but the output.
photochemsyn 2y ago
Wait - you trust humans? Just like that, without a legally enforceable contract that guarantees you certain rights in a court of law, to be enforced by the state, if they renege on their word? Astonishing!
- TeMPOraL 2y ago
  Nah, that's still trusting humans. Blockchain backed by Proof of Work was invented literally to solve the trust issue: turns out, trust can be dominated in energy, and all you need is to burn the equivalent in kilowatt hours to no longer need trust. (Note: the kWh/trust factor grows over time and with use.)
eru 2y ago
> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. People already trust eg their (mechanical) cars and their hammers and saws. And humans aren't all that trustworthy, either.
- bmitc 2y ago
  > People already trust eg their (mechanical) cars and their hammers and saws. Completely unrelated and irrelevant. > And humans aren't all that trustworthy, either. That may be kind of the point. But humans have consequences, generally speaking.
  - squigz 2y ago
    > Completely unrelated and irrelevant. WHy? It seems like an appropriate analogy to me.
    - jpnc 2y ago
      Hammer makes no attempt to appear human
  - bmitc 2y ago
    You can ask hammers, cars, and saws questions, and they can respond with complete confidence with information that may or may not be correct, which they have no way of knowing whether it's correct, and presented as if they are human? The analogy makes no sense.
- Barrin92 2y ago
  >People already trust eg their (mechanical) cars and their hammers and saws. People trust the people who make them, that's why brand value is so important and why "I only buy good old <country of choice> hammers" is so common. It's also why Sam Altman wanted his AI to sound like totally not Scarlett Johansson rather than give it the voice of a Dalek.
  - 2y ago
mistermann 2y ago
> It is not human. It will not behave like a human. ok..... > It will behave as it was trained or modeled to behave. It will behave according to the intentions of its creators. Umm....is this not more than a little contradictory? You believe that humans are not mostly a function of their cultural conditioning and the stories about "reality" that they ingest (which are processed according to the training)? > If it appears human, we will develop a false sense of trust. Not me. > But you can never trust it as you would trust a human. I would never trust[1] a human, at least not a neurotypical one. AI I will give a chance. [1] This is distinctly different from whether I would avail myself of the services of a human, or that I think it is guaranteed that they will screw things up. I am just deeply distrustful of them by default, and the more wealthy, powerful, or even (for the most part) educated they are, the more suspicious I am.
- idiotsecant 2y ago
  You trust no humans, ever? Sounds like a pretty depressing life. The best experiences come from vulnerability and trust.
  - mistermann 2y ago
    I don't trust people because they guess at everything, like you did here.
    - idiotsecant 2y ago
      Its not guessing when its just repeating what you said...
      - mistermann 2y ago
        > when its just repeating what you said... You guessed wrong again - consider: > Sounds like a pretty depressing life. > The best experiences come from vulnerability and trust. Or maybe I am on a TV show of some sort, hmmmmm.....
Aeolun 2y ago
> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. I would certainly never trust it as I would a human, but in most instances that makes it more trustworthy, not less.
gwd 2y ago
This is actually addressed in the article: > In addition to seeding Claude with broad character traits, we also want people to have an accurate sense of what they are interacting with when they interact with Claude and, ideally, for Claude to assist with this. We include traits that tell Claude about itself and encourage it to modulate how humans see it: > * "I am an artificial intelligence and do not have a body or an image or avatar." > * "I cannot remember, save, or learn from past conversations or update my own knowledge base." > * "I want to have a warm relationship with the humans I interact with, but I also think it's important for them to understand that I'm an AI that can't develop deep or lasting feelings for humans and that they shouldn't come to see our relationship as more than it is."
- moritonal 2y ago
  Seems counter-intuitive to have it explain it's not a person from the first-person. I (a person) would have preferred it output like: "This output is from an artificial intelligence" "This conversation isn't remembered, saved or updates" "This output is designed to feel like a warm relationship" See, now it's much more obvious. The problem is that it's blunt, it'd make lines like "this isn't saved" now technically incorrect, and won't be as effective in "connecting" with people, which is what people are so excited for.
  - TheCoreh 2y ago
    Culturally, the first person is not reserved only for human entities, for example the "don't pet me, I'm working" vests on service dogs, or the juvenile prank of writing "wash me" on a dirty car. Beyond these examples there's a very long history of mythological and fictional talking creatures, entities and objects all using the first person.
- irthomasthomas 2y ago
  > "I cannot remember, save, or learn from past conversations or update my own knowledge base. Is that really baked in through fine-tuning or just part of the web client system prompt? Becauze you can furnish the model with memory via the API.
lannisterstark 2y ago
> But you can never trust it as you would trust a human. Why would I trust a random human? If anything, I can be certain that a well trained machine will not call me slurs for being brown, or x sexual orientation, or break into my house.
- scubbo 2y ago
  > I can be certain that a well trained machine will not call me slurs for being brown "well-trained" is doing a lot of heavy-lifting, there. https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
  - davedx 2y ago
    Obviously Tay was not well trained? I think you can see the commentor's basic point, right?
    - scubbo 2y ago
      Right, yes, I can absolutely see their point. My point in return, to be explicit, was that "well-trained model" is a deceptively simple-sounding phrase which makes it sound like such a thing is commonplace, extant, or easy to identify. It's almost a No True Scotsman fallacy - yes, a "well-trained model" (by your own definitions of well-trained, though perhaps not the LLM creators') will not abuse you, by definition. Neither will a well-trained human.
- Yizahi 2y ago
  You do trust random humans daily. Or half of the world population would be killed in one day if it wasn't so. You trust that two white lines on the road is enough not to hit your car. You trust that you are not getting mugged by every passerby. The list is endless.
  - lannisterstark 2y ago
    Great, then you should then be able to freely hand your financial and banking, health data to a person next to you on a subway, right? or maybe, context is important and you should read that comment in the context I was speaking in :) --- Speaking of, you also trust random machines daily. So that point is really silly.
richardatlarge 2y ago
How many divorcées have said of their previous spouse, after x years, I never even knew him/her. So who can you trust ?
fifticon 2y ago
A thought: with humans, we already know we should be wary about trusting them unconditiobally.
ecocentrik 2y ago
Most of the social systems we create as humans are built around establishing redundancies and control to avoid placing trust in humans. We have democracies and elections to avoid getting stuck with crazy or incompetent rulers for too long. We have justice systems and laws to mitigate and ameliorate damage from humans with anti-social tendencies. We have intelligence agencies to collect information from and wage disinformation campaigns against our geopolitical rivals. These collections of digital neurons absolutely deserve the same levels of scrutiny we reserve for humans. Maybe at some point we will learn to distrust them more than we distrust humans but for now, humans are the high bar for distrust.
c1sc0 2y ago
That’s not going to happen because AI will be trained to exhibit whichever traits keep us more engaged. Just like social media. A human-mimicking AI will beat a more robot-like AI any day on that metric. And it starts with really simple things: sub-100 millisecond responses and the ability to instantly interrupt the AI as it is talking. From there we go on to open-mindedness etc …
gofreddygo 2y ago
> The biggest mistake with AI is making it appear human. 100% this. Its just too early for AI to impersonate human characteristics. The initial wonder or sense of amazement from what AI could do very quickly got mired with horrendous obvious logical flaws, clear lack of sentience and hallucinations being emphasized. If it was marketed as a tool, it would have immediately put the burden of its appropriate use on the user and the focus on the amazing things it can do. Once people get used to it, more human friendly behavior could be introduced as "enhancements". But instead we see this ridiculous parade of mimicking human like behavior from transformers. Its a marketing blunder.
HarHarVeryFunny 2y ago
I assume we're talking about AI in general (i.e. a future human-level AGI, with whatever architecture it may have), not today's language models (which are basically just expert systems). It may take effort to make an artificial species / AGI to appear fully human-like (notwithstanding how close even LLMs can appear, even if that's only via text interaction), but I think we're probably - for better or worse - going to try to push to make them human-like, in order to better understand us and communicate with us, and also just because we can. It's an interesting endeavor, and in Feynman's spirit of "If I can't build it, I don't understand it", I think people will strive to build "artificial humans". The potentially difficult part of making an artifical species appear human-like as opposed to intelligent but shoggoth/alien-like, is that it would need not only to be intelligent but also have human-like emotions modulating it's mental activity and the full range of innate traits/desires that make us human (as well as male or female). Probably a little goes a long way though, and just as people are willing to suspend disbelief and treat ChatGPT, or even ELIZA, as human, then an AGI that at least shows some type of genuine/innate anger/desire/curiosity/etc on occasion would appear to be on the spectrum of human. Ultimately, if we could identify and imbue an AGI will all (or at least the most significant of) innate human traits, then we should to be able to trust it in the same conditional way we trust each other.

by nicce 2y ago

falcor84 2y ago

> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. I've been treating LLM's as yet another voice on the internet - of course I wouldn't "trust" such a random person on the internet to be correct in their writing, let alone to act in my best interest - but so far, LLM's have been much more to my liking on both accounts than the random human netizen.

ben_w 2y ago

Well, mostly. But not "It will not behave like a human. It will behave as it was trained or modeled to behave." — that the latter is in these cases also the former, means it does behave (to a degree) like a human, and that is why it will be trusted. The caveat of "to a degree" being another way to phrase agreement with you that it will be an error to trust it. But it will be trusted. These models are already given more trust than they deserve.

nicce 2y ago
That is better worded. What I meant is that the "humanity" part is faked, and eventually, it will act as programmed to, without compromises or any moral weight.

simonw 2y ago

The audio conversation that accompanies this piece talks about that a little, at 9:21: https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s If you create a chatbot with no evidence of personality at all, there is a very real risk that people will assume it is a completely unbiased, robotic, potentially infallible machine. But LLMs are not that! They have all kinds of biases baked into them. Giving them a personality may be a good way to help people understand that.

hackernewds 2y ago
You both said the polar opposite things, neither providing any evidence past conjecture. Who is correct?
- simonw 2y ago
  I didn't express my own personal opinion, I tried to summarize the opinion given by the Anthropic researchers. I haven't made up my mind about this yet. I find the Anthropic view really interesting.
  - nicce 2y ago
    It is very difficult to say. I see the point from the video too, but it is all about the general baseline. For us, it might not matter, because we are very aware about these things. But most, and especially for the upcoming generations, they don't think like us. The baseline is defined how they are used to "robots". What are they expecting? To whom they are comparing? People who don't know how internals works on these things, might tend to up trust more "human" version. At least I would assume so. It is easier to interact with them and easier to like. People who know how these works internally, might be hesitant, and trust more for "neutral one". And the comment in the video is over-engineering the solution for the problem of the minority.
- nightowl_games 2y ago
  Welcome to the new era of AI, where there is no objective truth and no one knows what they are talking about.
  - idiotsecant 2y ago
    'The new era of AI'? Sounds like the universe in general to me. You must be new here. Enjoy your stay. Avoid Florida if you can.
  - Art9681 2y ago
    I remember the same thing was said ~15 years ago when social media became ubiquitous. And 15 before that, when AOL was ubiquitous. And 15 before that when...

startupsfail 2y ago

I’m not sure, but aligning for helpfulness and absence of self, like OpenAI did, could be a right move. It had been known for some time that selfish behaviors are a source of a lot of unhappiness, greed, etc. And the absence of self, absence of I, absence of character tends to fix that.

Zambyte 2y ago

> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. I actually really disagree with this. I think it's easier to distrust things that are human like. We are used to distrust humans. Less so things that seem like neutral tools. For example, people mindlessly fill out forms presented by large corporations asking for personal information. I think people would be less inclined to trust an LLM with that information, even if it it's actually likely to be less actionable (at least with current capabilities).

nicce 2y ago
In general I would agree. But AI can take the all the positive traits from humans to sound as likeable and trustable as possible (and companies curretly do that). Social engineering is a thing. We like and trust some people, and some we don’t, without any evidence about their behavior history. And, doesn’t your example conflict your intial claim? Because people trust humans, they send forms.
- Zambyte 2y ago
  In my example, I don't think people would give that information away as easily if not for the form mediating it. I don't think a language model would be as convincing as a simple form. I say this from experience - even using a language model running on my own machine, it sets off my internal alarms when I tell it private information, even though I know literally no human besides me will ever see it.
LeonB 2y ago
This is a great idea for an experiment. I’m not sure how it would go. Some people would refuse to give info to both the simple form and the personality rich chat bot. Some would give info to both. But how big is the middle area? One thing is certain though — whatever format is more successful at getting peoples info — that’s the one that you’d see more and more of over time.
- r2_pilot 2y ago

hackernewds 2y ago

This actually aligns spot on with DeepMind's research arguing why AI should not be anthropomorphic

astromaniak 2y ago
They have the right to have their opinion and build it into their products. But how about having other opinions and products too? Imagine in hospital, or care houses, patients will be more comfortable with human-like AI. Or AIG when it happens. As for trust, is it a bad thing? Anyway, humans' mind is speculative, biased, and prone to brainwashing. Looks like common thing with machines. So, trust, if it's bad, can be corrected by a few movies picturing evil AI. Pretty much like fear of clowns was created. Actually Terminator already created a wave of AI doomers.
- nicce 2y ago
  > So, trust, if it's bad, can be corrected by a few movies picturing evil AI. There are many steps between good and evil. Like giving biased product recommendation so that people buy it, or never mentioning things like Tiananmen Square.

tkgally 2y ago

> The biggest mistake with AI is making it appear human. I’m guessing that that mistake is already being made on a large scale intentionally, not by OpenAI or Anthropic, perhaps, but by companies developing virtual boyfriends and girlfriends and other emotionally sticky bots. I have avoided trying such services myself so far. Does anybody here have any recent experience with them?

mvkel 2y ago

Making it "appear human" is an input heuristic. I don't want to have to know some special syntax to work with it. It's a novel feature, not a problem.

nicce 2y ago
Input is not the problem, but the output.

photochemsyn 2y ago

Wait - you trust humans? Just like that, without a legally enforceable contract that guarantees you certain rights in a court of law, to be enforced by the state, if they renege on their word? Astonishing!

TeMPOraL 2y ago
Nah, that's still trusting humans. Blockchain backed by Proof of Work was invented literally to solve the trust issue: turns out, trust can be dominated in energy, and all you need is to burn the equivalent in kilowatt hours to no longer need trust. (Note: the kWh/trust factor grows over time and with use.)

eru 2y ago

> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. People already trust eg their (mechanical) cars and their hammers and saws. And humans aren't all that trustworthy, either.

bmitc 2y ago
> People already trust eg their (mechanical) cars and their hammers and saws. Completely unrelated and irrelevant. > And humans aren't all that trustworthy, either. That may be kind of the point. But humans have consequences, generally speaking.
- squigz 2y ago
  > Completely unrelated and irrelevant. WHy? It seems like an appropriate analogy to me.
  - jpnc 2y ago
    Hammer makes no attempt to appear human
- bmitc 2y ago
  You can ask hammers, cars, and saws questions, and they can respond with complete confidence with information that may or may not be correct, which they have no way of knowing whether it's correct, and presented as if they are human? The analogy makes no sense.
Barrin92 2y ago
>People already trust eg their (mechanical) cars and their hammers and saws. People trust the people who make them, that's why brand value is so important and why "I only buy good old <country of choice> hammers" is so common. It's also why Sam Altman wanted his AI to sound like totally not Scarlett Johansson rather than give it the voice of a Dalek.
- 2y ago

mistermann 2y ago

> It is not human. It will not behave like a human. ok..... > It will behave as it was trained or modeled to behave. It will behave according to the intentions of its creators. Umm....is this not more than a little contradictory? You believe that humans are not mostly a function of their cultural conditioning and the stories about "reality" that they ingest (which are processed according to the training)? > If it appears human, we will develop a false sense of trust. Not me. > But you can never trust it as you would trust a human. I would never trust[1] a human, at least not a neurotypical one. AI I will give a chance. [1] This is distinctly different from whether I would avail myself of the services of a human, or that I think it is guaranteed that they will screw things up. I am just deeply distrustful of them by default, and the more wealthy, powerful, or even (for the most part) educated they are, the more suspicious I am.

idiotsecant 2y ago
You trust no humans, ever? Sounds like a pretty depressing life. The best experiences come from vulnerability and trust.
- mistermann 2y ago
  I don't trust people because they guess at everything, like you did here.
  - idiotsecant 2y ago
    Its not guessing when its just repeating what you said...
    - mistermann 2y ago
      > when its just repeating what you said... You guessed wrong again - consider: > Sounds like a pretty depressing life. > The best experiences come from vulnerability and trust. Or maybe I am on a TV show of some sort, hmmmmm.....

Aeolun 2y ago

> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. I would certainly never trust it as I would a human, but in most instances that makes it more trustworthy, not less.

gwd 2y ago

This is actually addressed in the article: > In addition to seeding Claude with broad character traits, we also want people to have an accurate sense of what they are interacting with when they interact with Claude and, ideally, for Claude to assist with this. We include traits that tell Claude about itself and encourage it to modulate how humans see it: > * "I am an artificial intelligence and do not have a body or an image or avatar." > * "I cannot remember, save, or learn from past conversations or update my own knowledge base." > * "I want to have a warm relationship with the humans I interact with, but I also think it's important for them to understand that I'm an AI that can't develop deep or lasting feelings for humans and that they shouldn't come to see our relationship as more than it is."

moritonal 2y ago
Seems counter-intuitive to have it explain it's not a person from the first-person. I (a person) would have preferred it output like: "This output is from an artificial intelligence" "This conversation isn't remembered, saved or updates" "This output is designed to feel like a warm relationship" See, now it's much more obvious. The problem is that it's blunt, it'd make lines like "this isn't saved" now technically incorrect, and won't be as effective in "connecting" with people, which is what people are so excited for.
- TheCoreh 2y ago
  Culturally, the first person is not reserved only for human entities, for example the "don't pet me, I'm working" vests on service dogs, or the juvenile prank of writing "wash me" on a dirty car. Beyond these examples there's a very long history of mythological and fictional talking creatures, entities and objects all using the first person.
irthomasthomas 2y ago
> "I cannot remember, save, or learn from past conversations or update my own knowledge base. Is that really baked in through fine-tuning or just part of the web client system prompt? Becauze you can furnish the model with memory via the API.

lannisterstark 2y ago

> But you can never trust it as you would trust a human. Why would I trust a random human? If anything, I can be certain that a well trained machine will not call me slurs for being brown, or x sexual orientation, or break into my house.

scubbo 2y ago
> I can be certain that a well trained machine will not call me slurs for being brown "well-trained" is doing a lot of heavy-lifting, there. https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
- davedx 2y ago
  Obviously Tay was not well trained? I think you can see the commentor's basic point, right?
  - scubbo 2y ago
    Right, yes, I can absolutely see their point. My point in return, to be explicit, was that "well-trained model" is a deceptively simple-sounding phrase which makes it sound like such a thing is commonplace, extant, or easy to identify. It's almost a No True Scotsman fallacy - yes, a "well-trained model" (by your own definitions of well-trained, though perhaps not the LLM creators') will not abuse you, by definition. Neither will a well-trained human.
Yizahi 2y ago
You do trust random humans daily. Or half of the world population would be killed in one day if it wasn't so. You trust that two white lines on the road is enough not to hit your car. You trust that you are not getting mugged by every passerby. The list is endless.
- lannisterstark 2y ago
  Great, then you should then be able to freely hand your financial and banking, health data to a person next to you on a subway, right? or maybe, context is important and you should read that comment in the context I was speaking in :) --- Speaking of, you also trust random machines daily. So that point is really silly.

richardatlarge 2y ago

How many divorcées have said of their previous spouse, after x years, I never even knew him/her. So who can you trust ?

fifticon 2y ago

A thought: with humans, we already know we should be wary about trusting them unconditiobally.

ecocentrik 2y ago

Most of the social systems we create as humans are built around establishing redundancies and control to avoid placing trust in humans. We have democracies and elections to avoid getting stuck with crazy or incompetent rulers for too long. We have justice systems and laws to mitigate and ameliorate damage from humans with anti-social tendencies. We have intelligence agencies to collect information from and wage disinformation campaigns against our geopolitical rivals. These collections of digital neurons absolutely deserve the same levels of scrutiny we reserve for humans. Maybe at some point we will learn to distrust them more than we distrust humans but for now, humans are the high bar for distrust.

c1sc0 2y ago

That’s not going to happen because AI will be trained to exhibit whichever traits keep us more engaged. Just like social media. A human-mimicking AI will beat a more robot-like AI any day on that metric. And it starts with really simple things: sub-100 millisecond responses and the ability to instantly interrupt the AI as it is talking. From there we go on to open-mindedness etc …

gofreddygo 2y ago

> The biggest mistake with AI is making it appear human. 100% this. Its just too early for AI to impersonate human characteristics. The initial wonder or sense of amazement from what AI could do very quickly got mired with horrendous obvious logical flaws, clear lack of sentience and hallucinations being emphasized. If it was marketed as a tool, it would have immediately put the burden of its appropriate use on the user and the focus on the amazing things it can do. Once people get used to it, more human friendly behavior could be introduced as "enhancements". But instead we see this ridiculous parade of mimicking human like behavior from transformers. Its a marketing blunder.

HarHarVeryFunny 2y ago

I assume we're talking about AI in general (i.e. a future human-level AGI, with whatever architecture it may have), not today's language models (which are basically just expert systems). It may take effort to make an artificial species / AGI to appear fully human-like (notwithstanding how close even LLMs can appear, even if that's only via text interaction), but I think we're probably - for better or worse - going to try to push to make them human-like, in order to better understand us and communicate with us, and also just because we can. It's an interesting endeavor, and in Feynman's spirit of "If I can't build it, I don't understand it", I think people will strive to build "artificial humans". The potentially difficult part of making an artifical species appear human-like as opposed to intelligent but shoggoth/alien-like, is that it would need not only to be intelligent but also have human-like emotions modulating it's mental activity and the full range of innate traits/desires that make us human (as well as male or female). Probably a little goes a long way though, and just as people are willing to suspend disbelief and treat ChatGPT, or even ELIZA, as human, then an AGI that at least shows some type of genuine/innate anger/desire/curiosity/etc on occasion would appear to be on the spectrum of human. Ultimately, if we could identify and imbue an AGI will all (or at least the most significant of) innate human traits, then we should to be able to trust it in the same conditional way we trust each other.