20 ms·

They stole my voice with AI

524 points by sounds 2y ago

Marvinkw 2y ago
[dead]
rayval 2y ago
[flagged]
- MrLeap 2y ago
  > His case would be much more compelling If he convinces you, what will you be compelled to do?
  - saghm 2y ago
    Believe him? Which ostensibly is the entire point of writing a blog post about it?
- ShrimpHawk 2y ago
  If you read the article you would have seen multiple links to his channel and videos where you could compare
- hsjdhdvsk 2y ago
  That is easy, got his YouTube channel and listen to any of his vids. Jeff had been around in the tech community for a while. I have used his ansible roles almost a decade ago.
- shortformblog 2y ago
  He provided a whole YouTube video on his popular channel, with more than 750,000 subscribers: https://www.youtube.com/watch?v=UMofZIT9FcQ https://www.youtube.com/watch?v=UMofZIT9FcQ
- ambicapter 2y ago
  RTFA
- sieabahlpark 2y ago
  [dead]
ummonk 2y ago
I don't see why using AI would get around Midler vs. Ford. If anything, there is even less of an argument to be made in your defense when you use AI to replicate a voice, instead of using another voice actor to replicate the voice.
- anothernewdude 2y ago
  Real solution is to never use the voice actors again, and cut them out from the very beginning.
- oxygen_crisis 2y ago
  The court explicitly limited their decision to the voices of professional singers in that case: > ...these observations hold true of singing, especially singing by a singer of renown. The singer manifests herself in the song. To impersonate her voice is to pirate her identity... > We need not and do not go so far as to hold that every imitation of a voice to advertise merchandise is actionable. We hold only that when a distinctive voice of a professional singer is widely known and is deliberately imitated in order to sell a product, the sellers have appropriated what is not theirs...
  - ConorSheehan1 2y ago
    Doesn't this have an obvious edge case for every singer from now on though? If your voice is cloned before you become a singer of renown you have no protection.
    - vkou 2y ago
      Which is precisely why film producers are trying to get the power to do this to their actors. They aren't going to use AI to have Tom Cruise in their film. He won't sign these rights away. But they sure as hell want to have the next Tom Cruise sign those rights away as a condition for being hired to be Random Bystander #4 in a straight-to-streaming C-film. Then, once he becomes successful and famous, they won't have to pay him a cent to keep using him, forever. I can't wait for the future where even more of the wealth people who do work generate will be siphoned off to the owners.
wwweston 2y ago
I appreciate his pointer to precedent, but the truth is that while precedent is a start, we're going to need to do work with principles beyond precedent. When tech introduces unprecedented capabilities, we will either figure out how to draw boundaries within which it (among other features of society) works for people, not against them, or we'll let it lead us closer to a world in which the strong do what they will and the weak (or those just trying to keep a camry running) suffer what they must.
- toomuchtodo 2y ago
  California recently signed some legislation into effect. It’s a start. Congress is working on “No Artificial Intelligence Fake Replicas And Unauthorized Duplications Act.” Still in dev in the House, but has bipartisan support. Call your congressperson, ask them to co-sponsor and/or vote for it. https://www.cbsnews.com/losangeles/news/california-bills-protecting-actors-performers-from-a-i-replicas-signed-into-law-by-gov-newsom/ https://www.cbsnews.com/losangeles/news/california-bills-pro... https://salazar.house.gov/media/press-releases/salazar-introduces-no-ai-fraud-act https://salazar.house.gov/media/press-releases/salazar-intro... https://files.constantcontact.com/1849eea4801/695cfd71-1d24-4146-a453-3dab7d49babd.pdf https://files.constantcontact.com/1849eea4801/695cfd71-1d24-...
  - Dalewyn 2y ago
    It's going to be an interesting First Amendment question.
    - giancarlostoro 2y ago
      Might as well make photoshopping and manual audio tweaking / impersonation illegal, since its the same ballpark, just less effort.
      - Buttons840 2y ago
        We need better liable and slander enforcement. Treat realistic media as a truth claim that is subject to liable laws.
meiraleal 2y ago
It is like becoming meme famous: is up to you how to monetize it, nobody owes you nothing.
- azinman2 2y ago
  But they should owe you for stealing your likeliness without your awareness to promote their products. This isn’t for satire purposes.
  - meiraleal 2y ago
    How much? This IP/copyright mentality is so 90s/2000s. Brings back memories of Napster. This Jeff Geerling which I have never heard of will cry until some Spotify of AI appears and pay him some pennies. Maybe he wants to be the James Hetfield/Metallica of this generation. This guy doesn't have 1/10000 of the relevance Metallica had at the time tho.
    - shortformblog 2y ago
      You think someone stealing the dude’s voice is the same as people downloading Metallica songs? Are you trying not to be taken seriously?
    - keyboardcaper 2y ago
      Funny, I first heard of Jeff Geerling over ten years ago in my dev circles.
    - lokar 2y ago
      Your inability to look even few decades into the future to see the impact of this is depressing. You only seem to care about yourself and your current grift.
      - salawat 2y ago
        Welcome to Hacker News/YCombinator. It's what they do here. Ever since joining the community, it's been a speedrun of things explicitly covered as unethical during my CS Ethics course. Mainly stay here to keep a finger on how far things have slid down the "man made horrors beyond comprehension".
GaggiX 2y ago
>I haven't decided what to do. Make a video, say what you think, get views, and probably put more pressure on Elecrow to respond.
- its-summertime 2y ago
  They did make a video, https://www.youtube.com/watch?v=UMofZIT9FcQ https://www.youtube.com/watch?v=UMofZIT9FcQ It was linked in the article.
- m463 2y ago
  What I want to know is... Does this controversy all become free publicity for elecrow?
donatj 2y ago
Maybe I am crazy but I don't really think it sounds that much like him. It's a little similar but different. It's slightly higher pitch, more nasal, and the intonation is a little different.
- re 2y ago
  As someone who hasn't heard of him before, from the first few seconds of this video, I'd say it sounds similar enough to be an imperfect AI clone. https://www.youtube.com/watch?v=UMofZIT9FcQ https://www.youtube.com/watch?v=UMofZIT9FcQ
- mattl 2y ago
  I’ve watched hundreds of his videos and it sounds very much like him.
- hysan 2y ago
  As some who has watched all his videos and livestreams, I think that it very much sounds like him.
- unraveller 2y ago
  With the tools I'm aware of you just add clips of as many types of voices you want blended in and it blends everything in them to an unknowable uncontrollable degree plus entropy of the system. I suspect their story is they have added in more pleasant sounding voices to the mix which provides enough differentiation. Question is: who is to say how much is needed before it escapes likeness theft? The king of generic nerd voices is going to claim excessive likeness and the accused lifter isn't going to reveal his whole process. Also tuning AI voices by ear is surely possible soon so category kings are not saved by demanding to be left out of training. A ministry of voice authority sounds bleak.
- sentientslug 2y ago
  It is clearly trained on his voice. The intonation and pitch differences you describe are just because it’s AI generated and not human speech.
- ahaucnx 2y ago
carmackfan 2y ago
This whole argument rests on the absurd assumption that you can "own" a voice as if it's property. Does this mean people can own the patterns of vibration in air? It's completely nonsensical.
- ThrowawayTestr 2y ago
  You can own your likeness. Does that mean you can own the photos that represent your face? Yes, yes you can. Why should you voice be different?
  - 3np 2y ago
    Easy: The air is free but the electromagnetic spectrum is regulated. /s
  - unraveller 2y ago
    Human voices store shared accent data too not just biometrics, your siblings/twins/sorority sisters likely present with extremely similar voices which can cause the same confusion as AI clones. is it most famous sibling first gets the copyright or are siblings the only ones with the power to supersede it or must they alter their voice? Simple voice never really was guaranteed to be a unique part of identity since it changes so easily over time and ages.
  - carmackfan 2y ago
    > You can own your likeness. How?
    - npteljes 2y ago
      By law: https://en.wikipedia.org/wiki/Personality_rights https://en.wikipedia.org/wiki/Personality_rights
- kube-system 2y ago
  Intellectual property is, in fact, recognized by many legal jurisdictions. And audio works are typically included in that. However, in this situation, the right of publicity is probably more applicable.
XorNot 2y ago
The idea that stolen voice tones are going to matter at all is one of the shortest sighted bits of AI investment - powered by Hollywood "never make anything new" thinking. In about 5 years AI voices will be bespoke and more pleasant to listen to then any real human: they're not limited by vocal cord stress, can be altered at will, and can easily be calibrated by surveying user engagement. Subtly tweaking voice output and monitoring engagement is going to be the way forward.
- Barrin92 2y ago
  Stolen voices matter because what's being stolen here is the authors likeness, his reputation that he's build in the YouTube tech space and used for commercial products he had already reviewed. They chose exactly his voice for that reason. While AI voices will aesthetically be indistinguishable or even preferable they aren't going to carry any reputation or authenticity, which by definition is scarce and therefore valuable. In fact they're likely going to matter more because in a sea of generic commodified slop demand for people who command unique brand value goes up, not down. That's why influencers make the big bucks in advertising these days.
  - geerlingguy 2y ago
    Exactly. If it was a brand of pen ink cartridges or dishwasher detergent, that's one thing (still would be wrong, but not as egregious, and I might never have known it happened). The fact is, Elecrow's a company I've worked with in the past (never signed any contracts, but reviewed a product of theirs 4 years ago that they provided). They're active in the exact same space my YouTube audience is (Pi, microcontrollers, hobby electronics, homelab). There are a number of potential Elecrow customers who also subscribe to my YouTube channel (one of them alerted me to the tutorial series, in fact), and I would rather not have people be confused thinking I've sold my likeness or voice to be used for corporate product tutorials. Especially any competitors to Elecrow, who I may have a relationship with, that could be soured if they think I'm suddenly selling my voice/online persona for Elecrow's use.
  - XorNot 2y ago
    Again: this literally only matters currently in people trying to steal a voice. There's already VTubers who's whole visual identity is synthetic. Why wouldn't the same happen in any other space where performance can affect the perception of content, but you can now simply engineer the performance? Like I said: give it 5 years and you'll have influencers who no one has ever heard the voice of, because they don't make content with their own.
mediumsmart 2y ago
It’s the Wild West and will be for some time but I agree, they should have the decency to use only voices of the dear departed. The library should be open source and hosted on GitHub. the talking dead seems like a good name for it. Obviously we will have to put it to a vote among the living.
- deleted 2y ago
  [deleted]
- giraffe_lady 2y ago
  That's even worse imo. There's a reason nearly every major world belief system includes a proscription against necromancy and this is exactly what they mean. The living should not speak with the mouths of the dead.
  - echoangle 2y ago
    I’m pretty sure that was a joke.
    - johnnyanmac 2y ago
      Poe's law is in full effect with AI. I've heard some absolutely sociopathic takes with what AI should be allowed to do, said with utmost sincerity.
  - voiceblue 2y ago
    It seems more likely that necrophilia was a major problem (compared to today), given how the Egyptians handled it and stories like Botan Dōrō. Very strange that you’re saying cloning voices with AI is “exactly what they mean”…?
    - giraffe_lady 2y ago
      I think the strictly anthropological view is that it's because of the political and social power you accrue if you can convince people the ancestors are on your side. I'm sure there are several reasons though and I accept that one as likely part of it also.
at_a_remove 2y ago
More and more I am starting to wish I had gone ahead with the novel I had sketched out in the 1990s. The backdrop was a kind of post-imitative-AI collapse of trust in society, because it had become effortless to fake up, say, your least favorite political candidate talking about the merits of eating babies, so the various echo chambers bore a kind of ghastly fruit, each stance finding its own "evidence" for its beliefs, right down to the flat earth types. Paranoia runs rampant, and so on. It looks like we're heading in that direction.
- 8n4vidtmkvmk 2y ago
  You can still write it. It'll sound like you're just jumping on the AI bandwagon, but considering it's all the rage, it might still help your sales.
thih9 2y ago
We have 100s of tools that are about voice cloning - of course we’ll get content with cloned voices. Same as it happens with unauthorized use of someone’s images. And platforms and their moderation teams have processes in place to report and remove that. Looks like we need something similar for voice.
LegitShady 2y ago
I don't know where you live, but where I live I believe this sort of thing meets all the required elements for fraud.
surfingdino 2y ago
It's all fun and games until someone produces a recording of somebody else saying something incriminating and it will be used in court. This is the part of AI I hate.
- 8n4vidtmkvmk 2y ago
  It'll be bad for a few years, but surely at some point it'll become inadmissible in court because it's too easy to fake, right? But then what do we do, if video and audio footage is inadmissible?
  - Ylpertnodi 2y ago
    Where i am, video and audio are not admissible in court...'too easily faked'. A car bump i was in was dash-cammed, but all my team could do was second by second analysis of the video, and present that to the court. I did win, but it was very costly to do so.
    - 8n4vidtmkvmk 2y ago
      That's crazy. For dash cam videos you can at least compare if the footage matches with the damage on the cars. And hopefully both parties have video footage, then you can really compare if something doesn't add up.
- echoangle 2y ago
  Unless you also hate Image Editors, I don’t really get this point. Preserving forms of evidence isn’t really a primary concern when evaluating new useful technology.
- left-struck 2y ago
  It’s worse than that. People will start claiming that real, incriminating voice recordings of them are fakes as well. I think this matters more in the court of public opinion than in real court in both cases though.
cr3cr3 2y ago
Initially I'd say well if you're a public figure and upload your own voice online, of course this will happen. So its something to expect, however, this shouldn't be a problem for Jeff to solve... instead it should be YouTube's problem as they profit from the video monetization. Eventually they'll have to have some kind of detection for all uploaded content.
- absentmoon 2y ago
  I strongly disagree. I don't know the rights around one's own voice, but the idea that you suddenly lose ownership of something because you shared it online is the exact thing that many people take issue with when it is written in the terms of service for social networks, creator tools (adobe), etc.
  - cr3cr3 2y ago
    I didn't mention ownership and I don't think you should lose it (nor does one lose it really even in this case, legally). But I do think that in cases like these, where there's money involved and YouTube, that they should have the means to prevent it.
    - johnnyanmac 2y ago
      Issue is that Youtube profits off it while signing away liability. There's no incentive to prevent it.
gyudin 2y ago
It's not even close to his voice lmao, just has similar cadence.
golol 2y ago
>I remember when OpenAI practically cloned Scarlett Johanssen's voice... There is absolutely zero evidence for this. I find it infuriating that this keeps being stated as a fact. So they go and hire a voice actor and clearly use her voice to train, but then they also scrape Scarlett Johansson from youtube and splice it into the training data to make the voice a bit more like hers? Really does that sound realistic?
- klabb3 2y ago
  Yeah it does. Motive: Altman had some weird boyish thing for her and they asked her first, she said no. Means: Lots of available data to use from her movies. They probably trained a model first without releasing it just because it’s ridiculously easy. Especially for OpenAI. Opportunity: AI is astonishingly good at laundering and remixing without exposing the training set, for previously-unseen levels of plausible deniability.
  - golol 2y ago
    > AI is astonishingly good at laundering and remixing without exposing the training set They just about manage to make a good multimodal transformer that can generate audio and you expect that right away they can also interpolate in latent space? How does that actually work? It's not so simple. What benefit do they have from training on Scarlett Johansson's data, because they sure as hell have a big risk. They clearly hired a voice actress and they clearly told her to sound like Scarlett Johansson in "Her", and the end result perfectly fits with that. The voice doesn't "uncannily sound like SJ", no it just vaguely resembles her voice and mostly just mimicks the mannerisms from the movie. For me this is a perfect example of Occam's razor. One explanation is simple and realistic. The other explanation requires significantly more advanced AI control than OpenAI has claim/demonstrated, and it requires Altman to be so obsessed with the SJ idea that he goes out of his way to secretly train on her voice, risking legal exposure, while still hiring a voice actress.
    - klabb3 2y ago
      To be clear I’m not saying it did happen for sure. Only that it’s realistic and not exactly far fetched given everything that’s publicly known. > they can also interpolate in latent space? How does that actually work? I don’t know. I assumed it is the case because of the vast and excellent interpolation AI appears to do in other domains.
adityaathalye 2y ago
If LLMs are the ultimate remix machine, then is anyone with a RAG a digital DJ? One can't help but wonder what theft even means any more, when it comes to digital information. With the (lack of) legal precedent, it feels like the wild wild west of intellectual property and copyright law. Like, if even a superstar like Scarlett Johansson can only write a pained letter about OpenAI's hustle to mimic her "Her" persona, what can the comparatively garden-variety niche nerd do? Like Geerling, feel equally sad / angry / frustrated, but merely say "Please for the love of all that is good, be nice and follow an honour code.".
- wruza 2y ago
  what theft even means any more They dragged the term through different phases, but that’s just projection of will. Theft is undefined for objects with .copy() interface. It’s still there when you look at it. People have to adjust expectations, not laws. Computers replaced computers, now voice acting replaces voice actors. Your popularity doesn’t mean anything really and wouldn’t it be unfair if only popular could spare their jobs.
  - the_gorilla 2y ago
    > Theft is undefined for objects with .copy() interface. > Computers replaced computers, now voice acting replaces voice actors. It's incredible what web development does to someone's ability to communicate ideas.
    - ninalanyon 2y ago
      The original meaning of the word computer was a person who calculates. Looks perfectly clear to me.
      - anal_reactor 2y ago
        I have a very funny example of how using outdated words might lead to miscommunication, but unfortunately that particular one could get me banned, so let's just say that I'm a huge fan of picking a version of langauge from one specific time period and sticking to it, instead of having the reader do all the guesswork.
scotty79 2y ago
> I remember when OpenAI practically cloned Scarlett Johanssen's voice Except that never happened and the voice belonged to a completely different voice actress and Scarlett Johanssen had exactly zero right to prevent this person from making money as a voice actress lending it to AI. These complaints remind me a little bit of the story that a man complained that his photo was used to illustrate the article about how all hipsters look the same and it eventually turned out it wasn't his photo.
- probably_wrong 2y ago
  OprnAI says it never happened. They also took the voice down despite having it front and center at release time. And let's not forget that, if they didn't want people to think of Scarlett Johansson, then there would be no reason for their CEO to tweet "Her".
  - scotty79 2y ago
    "Her" is a brilliant movie about AI. The tweet was on point. And apparently they didn't give much crap about the specific voice since they taken it down because of mildly bad publicity. It was just supposed to be nice bonus gesture.
- sourraspberry 2y ago
  They reached out to Scarlett Johanssen. She said no. So they found a soundalike and Tweeted out references to the movie Her (starring Scarlett Johanssen as an AI chat bot) in the days leading up to launch. Scummy as fuck from OpenAI regardless of the technical legal rights and issues involved.
  - scotty79 2y ago
    What's scummy about it? If you want a specific kind of voice and approach one person about it but she doesn't agree what's wrong with asking another one? If there were just two blonds in the world, one famous and the other not and you wanted a blonde actress for the role and she said no is it scummy to hire the other one? Is it scummy to hire "discount" Matt Damon instead of Matt Damon?
swag314 2y ago
https://www.youtube.com/watch?v=IeTybKL1pM4 https://www.youtube.com/watch?v=IeTybKL1pM4
cranium 2y ago
(Obviously not a lawyer) Overlooking the AI part, isn't it a gross misrepresentation of Jeff's opinion or an unauthorized use of his image? By using his voice, it creates an implicit (fabricated) endorsement for their product and that feels very wrong. I'm sure laws exists to deal with these cases, since way before AI existed.
- mft_ 2y ago
  I’ve been thinking something similar recently. We’ve had people who are skilled voice mimics for ever, and they mostly exercise their skills for comedy/satire, and not for misrepresenting people’s opinions. IANAL either but I guess this is based on solid legal grounds, and misrepresenting people would be relatively easy to deal with legally. I guess the difference is democratisation - we’ve moved from very few people having this skill, to virtually anyone with a computer being able to do something similar. And so policing it will be much tougher, and likely beyond the means of someone like Jeff Geerling if it would require legal action to remedy.
  - aversis_ 2y ago
    Just wait till someone starts auto-deepfaking their way out of college exams and job interviews. Computers made graphic design approachable, but early adopters oversaturated the market before it stabilized. We’ll eventually figure out social norms and regulations for AI voice mimicry too, but there will be chaos first. Also, tech always moves faster than law. By the time courts catch up, this will be old news.
bookofbiden 2y ago
[dead]
ryzvonusef 2y ago
Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And this was before AI was easy to access. You can say a lot of things about 'oh backward countries' but this will not stay there, this will spread. You can't just give a toddler a knife and then blame them for stabbing someone. Has nothing to do with fame, with security, with copyright. This will get people killed. And we have no tools to control this. https://x.com/search?q=blasphemy https://x.com/search?q=blasphemy I fear the future.
- pnut 2y ago
  I guess then, you should use AI to generate videos of all of the lynch mob leadership committing blasphemy and let them sort it out internally?
  - ryzvonusef 2y ago
    You joke but, given the religious/sectarian nature of the issue, all it does is empower one sect to act against the leaders of the other sect. Check the twitter link, you won't have to scroll much to find a mullah being blasted for blasphemy. No one is safe.
    - javcasas 2y ago
      Well, then target dead people. Find someone who died 6 months ago, have AI make them a blasphemer, have the sect look for dead people. After the third or fourth time, they'll stop. Not because they want to stop, but because they don't trust this being another prank by Mullah Achmed the Fool.
      - ryzvonusef 2y ago
        Just like how you are the traffic, you are also the mob Mobs are not formal organisations with leadership and membership dues. Mobs are general everyday people who are sick and tird of this miserable life and need an excuse to rabble and rouse. So in this case, fake dead person's blasphemy will only be channeled towards their family members, mobs just need an excuse to get crazy. AI is not what causes mobs, it just helps lower the incitement barrier by providing quality bait.
aaron695 2y ago
[dead]
4ndrewl 2y ago
Let's just assume you can't trust any ugc on the internet from now. It's all done, but fun whilst it lasted.
paganel 2y ago
For what it's worth this seems to be a thing, for example here's a video promoting some batteries-thingie with the help of AI-voice based on a (famous, in some circles) podcast girl. [1] https://old.reddit.com/r/redscarepod/comments/1fmiiwt/which_one_of_you_did_this/ https://old.reddit.com/r/redscarepod/comments/1fmiiwt/which_...
- deleted 2y ago
  [deleted]
sandreas 2y ago
This is exactly the reason why I'm not open sourcing a tool I developed where you can take an audio book together with an epub to build an ljspeech dataset and train a voice model. Although it was not too hard to create I believe making it easier is something i don't like to achieve... I hate to say this but ruining a narrators existence with AI seems to get easier every day.
- sureglymop 2y ago
  I do think that the floodgates are open now. AI is still absolutely terrible to mediocre at coding at best but for more trivial tasks like this it'll get indistinguishably good AND businesses are ready to sell before even beginning to think of the consequences.
  - sandreas 2y ago
    This. I'm still wondering why audio book giants like amazon/audible are not having license contracts with their narrators to do supportive narrating tasks by AI paying Them a fee for AI generating their voice. It probably would be win win... Regarding how easy it was to clone my favorite narrators voice with open source tools I'm a bit afraid of what amazon could do with a whole cloud and massive man power
moffkalast 2y ago
Yeah it's pretty much an inevitability that anyone who's ever posted more than 10 seconds of their voice will have it stolen eventually if anyone has a motive to do it. That's already all it takes to few shot a decent TTS _today_. Most likely all existing youtubers will have complete voice and video digital clones made out of them. Then you can also tune an LLM on their scripts and it'll respond in the same character as well. In theory you could also bring back ones who are dead, which would be very interesting in a historical sense. Like if we had hundreds of hours of Napoleon talking in front of a camera, it would be trivial to recreate a digital version of him for anthropologic study, maybe even having various figures debate things with each other. That's what historians a century later after we all die will be able to do with impunity.
rishikeshs 2y ago
Slightly off topic, but what’s that logo at the bottom of his website? Is that some sort of a coat of arms?
- geerlingguy 2y ago
  It's part of the coat of arms for the Geerling family, yes.
  - rishikeshs 2y ago
    Thanks. Would be great if you could write a bit about it.
    - geerlingguy 2y ago
      Honestly I'm not the heraldry guy or ancestry guy in my family, but I thought it looked cool and helped make an SVG of it for a shirt my uncle made for all the family one year, so I slapped it in the footer of the blog :)
      - rishikeshs 2y ago
        Haha that’s wonderful. I’m fascinated by coat of arms. I’m from Southern India, and We don’t have anything like this except for may be royal families. I’m thinking of making one myself, but not sure how this works and hence I asked! :)
        geerlingguy 2y ago
        Heh, the Geerling family is far from royal, it's just something a lot of families in Europe seemed to do (heraldry), and the idea is pretty neat—try to add symbols to your family crest that you want your family to try to strive for (like honor, strength, love, peace, etc.). I think we even did a thing in one of my art classes on heraldry once, and the idea was to try to figure out what symbols you would choose for your own personal crest, based on what kind of things we'd see in historical ones.
benterix 2y ago
Elecrow seems a Chinese company, right? In that case, I don't expect any reply.
pmarreck 2y ago
[flagged]
- maximus-decimus 2y ago
  The first one sounds more like Scarlet Johanson to me. It's not just about the voice itself, it's about the sensuality and gravitas of the voice. The first one has Scralet's warmth and feminity whereas the second one sounds a robot impersonator trying mimicking scarlet's voice without understanding emotions. So to me you really proved that the voice actress sounds more like Scarlet than a literal AI replication of Scarlet. If that's now damning, I don't know what is.
  - pmarreck 2y ago
    [flagged]
- ahiknsr 2y ago
  > Scarlett's baseless accusation https://www.theguardian.com/film/article/2024/jul/18/scarlett-johansson-chatgpt-voice-openai-sam-altman https://www.theguardian.com/film/article/2024/jul/18/scarlet... > The actor made headlines earlier this year when she issued a public statement saying OpenAI’s chief executive, Sam Altman, had contacted her in September 2023 to provide the voice for Sky, in an ode to her role in the Spike Jonze film Her, but she had declined for “personal reasons”. In May, as the company released Sky, Altman shared a tweet that simply read “her”.
  - pmarreck 2y ago
    This is not evidence, and it is still baseless. They literally hired a completely different voice actress. Nothing else needs to be said. What the fsck does it matter if it was "similar to" Scarlett Johannsen, even if it was (which it wasn't)? Should an actor be able to sue another actor who looks similar because of the work they got?
- 127 2y ago
  I'm in the camp that thinks celebrities take way too much mental space of the regular person on the internet. It distorts ones ability to differentiate as there are these huge gravity centers that everybody makes comparisons to, even subconsciously. If Scarlett Johansson wasn't famous and Sam Altman didn't offer her the job, everybody would have no trouble in recognizing the differences in the voice. Priming is an extraordinarily powerful cognitive bias.
eth0up 2y ago
My aunt is (supposedly) in her 90s, having left the US for Ecuador 10 or so years ago. We've remained in regular contact via phone over the years. Recently we went more than a few months without speaking. She has/had two numbers; magic jack and google. When I tried to call her, the magic jack was no longer in service and google said something about "unavailable". I reached out to my cousin (my aunt's daughter) to inquire. I was told her number (and perhaps other things) had been "hacked", whatever that means. She had recently broken her hip and was in a hospital recovering. With this on my mind, I received a call (from the google number), strangely, while processing files with GPT. My skepticism was primed and ready, possibly making me paranoid. However, I did my due diligence and asked dozens of questions, mostly boring things that she typically wouldn't have patience for. Sometimes she'd reply with a reasonable answer and sometimes not, which made it difficult to evaluate. Toward the end, I asked where she was. She said, with an awkward tempo "I'm at home, in Cuenca", which I found odd because she'd normally just say she was at home, period. I then pressed her to tell me where she was before she returned home. She said she didn't understand. I rephrased the question, stating that it was a simple inquiry, eg "where were you before going home?" She said "this is getting too strange and confusing " and killed the call. I notified my cousin, telling her I thought something was suspicious, still cognizant of all the characteristics one would expect from a 90 year old recovering from a serious injury. My cousin might, technology wise, be in AOL territory. About 5 days later, I received a call from my aunt, on the google line. This time,I was more passive and cautious, but again, asked dozens of boring questions to probe the situation. I was surprised by both her ability to answer certain questions and also her inability to answer some questions. I tried to ask questions on topics we'd never discussed, in case the line had been tapped for a long time and referencing was established by an imposter. I had begun to suspect I had been paranoid. But several aspects were burning me: 1) typing noises in the background 2) Shatneresque pauses for nearly every reply 3) refusal to answer some specific questions. At the end of our apparent conversation, I asked her to do a very serious favor for me: send me a selfie, with one hand making the thumbs up gesture. She replied "I'll send you a photo of my passport ". I replied "that's stupid, ridiculous and serves no purpose. Don't do that. Understand? Do NOT send me a passport photo. I'm asking you something very important. Do exactly what I asked. Will you do this?" Her reply: "yes. What is your email address?" This was odd. I told her she already knew and it's the same one she'd had for years. She asked that I tell her anyway. Ok, 90 years old, traumatic injury, possible prescription drugs... "It's my full name @ xyzmail com". We killed the call. I immediately called my cousin and told her of my suspicions, including some my aunt's babbling about all her finances and accounts being inaccessible. She said that was strange because she just deposited 8k into her account. Meanwhile, a notification appears in the phone, an email from my aunt. It's a photo of her passport. Having no authority in this situation, but plenty well annoyed, I immediately jumped on a real computer and ran the photo through exiftool. The photograph was taken in 2023 and it was August of 2024. I then grabbed the geo coordinates (cryptically presented in exiftool) and with some effort, geolocated the image to right on top of her former residence, in Cuenca. I still don't know WTF is going on and my cousin thinks I'm a dingbat. But what I know for sure, is this is an age where such things are plausible enough and will soon be inevitable. The way I think may be deranged, but I truly don't even know if my aunt still exists. But I can have a pretty compelling conversation, either with her, or something strongly resembling her, minus the Shatneresque pauses, typing noises and selective amnesia.
vonnik 2y ago
California banned several forms of deepfake and digital replicas without consent just days ago. https://techcrunch.com/2024/09/19/here-is-whats-illegal-under-californias-8-and-counting-new-ai-laws/ https://techcrunch.com/2024/09/19/here-is-whats-illegal-unde... Not sure if those laws apply to Jeff tho, as they concern porn, politics and employer contracts.
djoldman 2y ago
> I remember when OpenAI practically cloned Scarlett Johanssen's voice ... I don't have a dog in this fight but just to be clear, OpenAI has stated that they paid a voice actor to create the voice ("Sky") that sounds like Scarlett Johanssen. There was no "cloning" or "stealing" (that they say). https://openai.com/index/how-the-voices-for-chatgpt-were-chosen/ https://openai.com/index/how-the-voices-for-chatgpt-were-cho...
- exitb 2y ago
  This is subtly wrong. They tried to hire the celebrity, got refused, then hired a different talent to do her “natural voice”. The official story is that it just happens to sound alike.
- kbelder 2y ago
  It's interesting that this rumor has been pointed out at least three times in this discussion, and every time it's voted down. Doesn't fit with the passions of many posters, I guess.
  - johnnyanmac 2y ago
    because it's an uncharitable interpretation at best, and misleading at worst. They approached her, they refused, and then they hired a sound-a-like. That is textbook Ford v Milder. They may have been absolutely fine if Altman never approached Scarlet. But context matters.
ei23 2y ago
I’m a small tech YouTuber and I’ve also had contact with Elecrow. As far as I know, employees (not just at Elecrow) receive rewards, promotions, or commissions when they secure long term partnerships and video collaborations with YouTubers. Perhaps someone thought it would be clever clone Jeffs voice since his channel is quite popular in this field. This certainly isn't great PR for Elecrow right now. I would also wonder if they will confess to that this was intentional...
hcks 2y ago
[flagged]
znpy 2y ago
We’ve been in a post-truth world for T least ten years anyway. We already had fake news and organizations willingly spread fake news. We had clearly fake pictures and people believing that. Flat-earthers, no-vax and whatever. This is just another brick in the wall.
segmondy 2y ago
What if they had someone that sounded just like you and had the person do a voice over? What if they had someone that sounded like you, had the person give them sample and used AI to generate voice? What if the hired someone that could imitate your voice, then had the person give them sample and used AI to generate voice?
singleshot_ 2y ago
When you say that lawyers always cost a lot of money: I’d absolutely do this pro bono but more than likely you’re not in a state where I’m licensed. You can absolutely positively find a free lawyer if your issue is interesting enough. This is the most interesting issue of our day.
cityzen 2y ago
Ex-Google CEO says successful AI startups can steal IP and hire lawyers to ‘clean up the mess’ / “But if nobody uses your product, it doesn’t matter that you stole all the content,” Eric Schmidt said during a recent talk at Stanford that has been taken offline. Since that guy was CEO of Google it’s all good right??? https://www.theverge.com/2024/8/14/24220658/google-eric-schmidt-stanford-talk-ai-startups-openai https://www.theverge.com/2024/8/14/24220658/google-eric-schm...
- johnnyanmac 2y ago
  well that does nail down one huge societal issue that multiple domains need to address: fine and punishments are so low that they are simply an expense instead of a deathblow. We definitely need to overhaul a lot of these white collar fines. I just watched a video today and learned the federal maximum fine for being caught using child labor is capped at $15k per worker. No wonder child labor has skyrocketed over the decade.
veunes 2y ago
Investment in tools that can verify the authenticity of audio and video content is crucial
t0bia_s 2y ago
We should adopt and use to it. There will be more and more fake AI created content every day so we should be confrontend with it to learn how to react to it. Regulating prolong adoption and take resources.
oehpr 2y ago
I want to ask and answer two of my own questions here: 1. Why clone Jeff's voice? When I was messing with stable diffusion using Automatic1111's interface, I noticed it came with a big list of artists to add to the prompt to stylize the image in some way. There was a big row in the media about ai art reproducing artists work and many artists came forward feeling it was a personal attack. But... I mean the truth is more general than that. When I pressed a button to insert a random name into a prompt, my goal was not "yes give me this person's art for free", it was "style this somehow". I wasn't personally interested in any particular artist, I honestly would have preferred a bunch of sliders. Jeff here is clearly a good speaker. That's a practiced talent and voice actors exist because it's hard. Elecrow wanted a voice over and they wanted it to be as good as they could make it. Jeff is very good. So did they want Jeff? I think what they really wanted was a good and cogent narration with the tenor of a person. Not a machine making noises that sound like english. If they had an easy way to get that, we wouldn't be talking about it here. 2. What function does copyright serve? Well. I think a reasonable argument would be that if people were able to reproduce your work for free, you would quickly find yourself without a monetary incentive to make more of it. So. What happens if you combine answer 1 with answer 2? I think it leads to: "We should consider making it illegal to automatically reproduce the work of an artisan.", you know, the luddic argument. An argument that has been perceived to be, more or less, settled. So it seems to me: That for individuals, harms matter, and for society, it doesn't.
- 1shooner 2y ago
  >I think what they really wanted was a good and cogent narration with the tenor of a person. Not a machine making noises that sound like english. If they had an easy way to get that, we wouldn't be talking about it here. I think the general assumption is that they wanted to, at the very least, strongly imply his endorsement of the product or video. Which I would say they did effectively. If I had happened on a clip of one of these videos outside the context of this controversy, I could have easily gotten the impression he was working with the vendor.
- lesostep 2y ago
  >> But... I mean the truth is more general than that. When I pressed a button to insert a random name into a prompt, my goal was not "yes give me this person's art for free", it was "style this somehow". yeah and that's the problem. The style of an artist is a developed thing. To think that one could borrow your style not through learning and caring, but through mathematically analyzing the width, and colors, and patterns and applying it to a random noise — that's kinda insulting. If nobody cares about my real work, why do they care about using my style, then? Develop your own an teach your AI on that, if there really isn't any difference. People say that AI learns how a human would. But a human wouldn't (couldn't!) learn like an AI can. He can't look at the pixels, can't mechanically churn through patterns. If someone can learn from art like AI learns art, I would also be opposed to them learning anything from me :D
Corrado 2y ago
So, do we "solve" this by impersonating a voice from a very litigious company? Let's say we clone Micky Mouse's voice to say some blasphemous things. Does that bring the full weight of Disney down on this? Will that force a change? Would it be a good change?
- ycombinatrix 2y ago
  isn't Mickey in the public domain now
rldjbpin 2y ago
from the discourse here, the main pain point really is the accessibility to do something like this thanks to the new models. IANAL and not sure about regional precedence on these topics, but there are plenty of ads where lookalikes or voice actors are used to use someone's likeness. they are mostly in satire, but there is yet to be a case where there was a litigation over this or prior approval needed. we have ai-based voice abuse in the political sphere, and where there was only one legislation for banning the use in voice calls for one country (https://news.ycombinator.com/item?id=39304736 https://news.ycombinator.com/item?id=39304736), another country actively used the same underlying tech to aid their own rallies (https://news.ycombinator.com/item?id=40532157 https://news.ycombinator.com/item?id=40532157). the tools are here to stay, but what is fair use needs to be defined more than ever.
- johnnyanmac 2y ago
  >they are mostly in satire Satire is one of the few use cases of fair use that hasn't been torn down. So that tracks. > there is yet to be a case where there was a litigation over this or prior approval needed. There's quite a few over impersonation. Most broadcast media knows how to skirt the rules though.
milabogdanova 2y ago
[dead]
m3kw9 2y ago
how do people know if its a very similar sounding voice, or identical?
- johnnyanmac 2y ago
  Not to be crude, but they have ears. and you'll hear uncanny likeness of a voice you follow everyday when it comes up. It's kind of like saying "how do we know that face is similar/identical"? Humans are surprisingly good at knowing when something feels off in other humans. Even if we lack the vocabulary to fully explain the difference.
  - m3kw9 2y ago
    I think you unknowingly paired the speaking style, tone patterns, levels and spacing etc which isn’t actually part of their voice, but combining it with the voice print you get a clear picture. If someone cloned my voice and start speaking differently from me, different levels(soft loud) and have an accent, nobody would recognize it was me.
  - echoangle 2y ago
    The problem is that there are a lot of humans, and a lot of humans with online presence. I wouldn’t really be confident in identifying a „stolen face“ online because you’re bound to look similar to someone when creating a face. Does that mean that the look was stolen or is it just coincidence? What is the likelihood of a random AI voice sounding similar to some real online creator?
    - johnnyanmac 2y ago
      >I wouldn’t really be confident in identifying a „stolen face“ online because you’re bound to look similar to someone when creating a face. Does that mean that the look was stolen or is it just coincidence? Luckily, national ID can easily solve that problem offline. It'd be a more interesting problem if some company searched far and wide to identify someone that has a similar face to [famous person]. And that's part of the issue in that there's no way to verify "is my voiced sampled here?". Opening the black box can help with that, but that's something companies are fighting to the death to not allow. > What is the likelihood of a random AI voice sounding similar to some real online creator? Given how much data was scraped, less likely than the fact that their data is in fact in the training set.

524 points by sounds 2y ago

Marvinkw 2y ago

[dead]

rayval 2y ago

[flagged]

MrLeap 2y ago
> His case would be much more compelling If he convinces you, what will you be compelled to do?
- saghm 2y ago
  Believe him? Which ostensibly is the entire point of writing a blog post about it?
ShrimpHawk 2y ago
If you read the article you would have seen multiple links to his channel and videos where you could compare
hsjdhdvsk 2y ago
That is easy, got his YouTube channel and listen to any of his vids. Jeff had been around in the tech community for a while. I have used his ansible roles almost a decade ago.
shortformblog 2y ago
He provided a whole YouTube video on his popular channel, with more than 750,000 subscribers: https://www.youtube.com/watch?v=UMofZIT9FcQ https://www.youtube.com/watch?v=UMofZIT9FcQ
ambicapter 2y ago
RTFA
sieabahlpark 2y ago
[dead]

ummonk 2y ago

I don't see why using AI would get around Midler vs. Ford. If anything, there is even less of an argument to be made in your defense when you use AI to replicate a voice, instead of using another voice actor to replicate the voice.

anothernewdude 2y ago
Real solution is to never use the voice actors again, and cut them out from the very beginning.
oxygen_crisis 2y ago
The court explicitly limited their decision to the voices of professional singers in that case: > ...these observations hold true of singing, especially singing by a singer of renown. The singer manifests herself in the song. To impersonate her voice is to pirate her identity... > We need not and do not go so far as to hold that every imitation of a voice to advertise merchandise is actionable. We hold only that when a distinctive voice of a professional singer is widely known and is deliberately imitated in order to sell a product, the sellers have appropriated what is not theirs...
- ConorSheehan1 2y ago
  Doesn't this have an obvious edge case for every singer from now on though? If your voice is cloned before you become a singer of renown you have no protection.
  - vkou 2y ago
    Which is precisely why film producers are trying to get the power to do this to their actors. They aren't going to use AI to have Tom Cruise in their film. He won't sign these rights away. But they sure as hell want to have the next Tom Cruise sign those rights away as a condition for being hired to be Random Bystander #4 in a straight-to-streaming C-film. Then, once he becomes successful and famous, they won't have to pay him a cent to keep using him, forever. I can't wait for the future where even more of the wealth people who do work generate will be siphoned off to the owners.

wwweston 2y ago

I appreciate his pointer to precedent, but the truth is that while precedent is a start, we're going to need to do work with principles beyond precedent. When tech introduces unprecedented capabilities, we will either figure out how to draw boundaries within which it (among other features of society) works for people, not against them, or we'll let it lead us closer to a world in which the strong do what they will and the weak (or those just trying to keep a camry running) suffer what they must.

toomuchtodo 2y ago
California recently signed some legislation into effect. It’s a start. Congress is working on “No Artificial Intelligence Fake Replicas And Unauthorized Duplications Act.” Still in dev in the House, but has bipartisan support. Call your congressperson, ask them to co-sponsor and/or vote for it. https://www.cbsnews.com/losangeles/news/california-bills-protecting-actors-performers-from-a-i-replicas-signed-into-law-by-gov-newsom/ https://www.cbsnews.com/losangeles/news/california-bills-pro... https://salazar.house.gov/media/press-releases/salazar-introduces-no-ai-fraud-act https://salazar.house.gov/media/press-releases/salazar-intro... https://files.constantcontact.com/1849eea4801/695cfd71-1d24-4146-a453-3dab7d49babd.pdf https://files.constantcontact.com/1849eea4801/695cfd71-1d24-...
- Dalewyn 2y ago
  It's going to be an interesting First Amendment question.
  - giancarlostoro 2y ago
    Might as well make photoshopping and manual audio tweaking / impersonation illegal, since its the same ballpark, just less effort.
    - Buttons840 2y ago
      We need better liable and slander enforcement. Treat realistic media as a truth claim that is subject to liable laws.

meiraleal 2y ago

It is like becoming meme famous: is up to you how to monetize it, nobody owes you nothing.

azinman2 2y ago
But they should owe you for stealing your likeliness without your awareness to promote their products. This isn’t for satire purposes.
- meiraleal 2y ago
  How much? This IP/copyright mentality is so 90s/2000s. Brings back memories of Napster. This Jeff Geerling which I have never heard of will cry until some Spotify of AI appears and pay him some pennies. Maybe he wants to be the James Hetfield/Metallica of this generation. This guy doesn't have 1/10000 of the relevance Metallica had at the time tho.
  - shortformblog 2y ago
    You think someone stealing the dude’s voice is the same as people downloading Metallica songs? Are you trying not to be taken seriously?
  - keyboardcaper 2y ago
    Funny, I first heard of Jeff Geerling over ten years ago in my dev circles.
  - lokar 2y ago
    Your inability to look even few decades into the future to see the impact of this is depressing. You only seem to care about yourself and your current grift.
    - salawat 2y ago
      Welcome to Hacker News/YCombinator. It's what they do here. Ever since joining the community, it's been a speedrun of things explicitly covered as unethical during my CS Ethics course. Mainly stay here to keep a finger on how far things have slid down the "man made horrors beyond comprehension".

GaggiX 2y ago

>I haven't decided what to do. Make a video, say what you think, get views, and probably put more pressure on Elecrow to respond.

its-summertime 2y ago
They did make a video, https://www.youtube.com/watch?v=UMofZIT9FcQ https://www.youtube.com/watch?v=UMofZIT9FcQ It was linked in the article.
m463 2y ago
What I want to know is... Does this controversy all become free publicity for elecrow?

donatj 2y ago

Maybe I am crazy but I don't really think it sounds that much like him. It's a little similar but different. It's slightly higher pitch, more nasal, and the intonation is a little different.

re 2y ago
As someone who hasn't heard of him before, from the first few seconds of this video, I'd say it sounds similar enough to be an imperfect AI clone. https://www.youtube.com/watch?v=UMofZIT9FcQ https://www.youtube.com/watch?v=UMofZIT9FcQ
mattl 2y ago
I’ve watched hundreds of his videos and it sounds very much like him.
hysan 2y ago
As some who has watched all his videos and livestreams, I think that it very much sounds like him.
unraveller 2y ago
With the tools I'm aware of you just add clips of as many types of voices you want blended in and it blends everything in them to an unknowable uncontrollable degree plus entropy of the system. I suspect their story is they have added in more pleasant sounding voices to the mix which provides enough differentiation. Question is: who is to say how much is needed before it escapes likeness theft? The king of generic nerd voices is going to claim excessive likeness and the accused lifter isn't going to reveal his whole process. Also tuning AI voices by ear is surely possible soon so category kings are not saved by demanding to be left out of training. A ministry of voice authority sounds bleak.
sentientslug 2y ago
It is clearly trained on his voice. The intonation and pitch differences you describe are just because it’s AI generated and not human speech.
ahaucnx 2y ago

carmackfan 2y ago

This whole argument rests on the absurd assumption that you can "own" a voice as if it's property. Does this mean people can own the patterns of vibration in air? It's completely nonsensical.

ThrowawayTestr 2y ago
You can own your likeness. Does that mean you can own the photos that represent your face? Yes, yes you can. Why should you voice be different?
- 3np 2y ago
  Easy: The air is free but the electromagnetic spectrum is regulated. /s
- unraveller 2y ago
  Human voices store shared accent data too not just biometrics, your siblings/twins/sorority sisters likely present with extremely similar voices which can cause the same confusion as AI clones. is it most famous sibling first gets the copyright or are siblings the only ones with the power to supersede it or must they alter their voice? Simple voice never really was guaranteed to be a unique part of identity since it changes so easily over time and ages.
- carmackfan 2y ago
  > You can own your likeness. How?
  - npteljes 2y ago
    By law: https://en.wikipedia.org/wiki/Personality_rights https://en.wikipedia.org/wiki/Personality_rights
kube-system 2y ago
Intellectual property is, in fact, recognized by many legal jurisdictions. And audio works are typically included in that. However, in this situation, the right of publicity is probably more applicable.

XorNot 2y ago

The idea that stolen voice tones are going to matter at all is one of the shortest sighted bits of AI investment - powered by Hollywood "never make anything new" thinking. In about 5 years AI voices will be bespoke and more pleasant to listen to then any real human: they're not limited by vocal cord stress, can be altered at will, and can easily be calibrated by surveying user engagement. Subtly tweaking voice output and monitoring engagement is going to be the way forward.

Barrin92 2y ago
Stolen voices matter because what's being stolen here is the authors likeness, his reputation that he's build in the YouTube tech space and used for commercial products he had already reviewed. They chose exactly his voice for that reason. While AI voices will aesthetically be indistinguishable or even preferable they aren't going to carry any reputation or authenticity, which by definition is scarce and therefore valuable. In fact they're likely going to matter more because in a sea of generic commodified slop demand for people who command unique brand value goes up, not down. That's why influencers make the big bucks in advertising these days.
- geerlingguy 2y ago
  Exactly. If it was a brand of pen ink cartridges or dishwasher detergent, that's one thing (still would be wrong, but not as egregious, and I might never have known it happened). The fact is, Elecrow's a company I've worked with in the past (never signed any contracts, but reviewed a product of theirs 4 years ago that they provided). They're active in the exact same space my YouTube audience is (Pi, microcontrollers, hobby electronics, homelab). There are a number of potential Elecrow customers who also subscribe to my YouTube channel (one of them alerted me to the tutorial series, in fact), and I would rather not have people be confused thinking I've sold my likeness or voice to be used for corporate product tutorials. Especially any competitors to Elecrow, who I may have a relationship with, that could be soured if they think I'm suddenly selling my voice/online persona for Elecrow's use.
- XorNot 2y ago
  Again: this literally only matters currently in people trying to steal a voice. There's already VTubers who's whole visual identity is synthetic. Why wouldn't the same happen in any other space where performance can affect the perception of content, but you can now simply engineer the performance? Like I said: give it 5 years and you'll have influencers who no one has ever heard the voice of, because they don't make content with their own.

mediumsmart 2y ago

It’s the Wild West and will be for some time but I agree, they should have the decency to use only voices of the dear departed. The library should be open source and hosted on GitHub. the talking dead seems like a good name for it. Obviously we will have to put it to a vote among the living.

deleted 2y ago
[deleted]
giraffe_lady 2y ago
That's even worse imo. There's a reason nearly every major world belief system includes a proscription against necromancy and this is exactly what they mean. The living should not speak with the mouths of the dead.
- echoangle 2y ago
  I’m pretty sure that was a joke.
  - johnnyanmac 2y ago
    Poe's law is in full effect with AI. I've heard some absolutely sociopathic takes with what AI should be allowed to do, said with utmost sincerity.
- voiceblue 2y ago
  It seems more likely that necrophilia was a major problem (compared to today), given how the Egyptians handled it and stories like Botan Dōrō. Very strange that you’re saying cloning voices with AI is “exactly what they mean”…?
  - giraffe_lady 2y ago
    I think the strictly anthropological view is that it's because of the political and social power you accrue if you can convince people the ancestors are on your side. I'm sure there are several reasons though and I accept that one as likely part of it also.

at_a_remove 2y ago

More and more I am starting to wish I had gone ahead with the novel I had sketched out in the 1990s. The backdrop was a kind of post-imitative-AI collapse of trust in society, because it had become effortless to fake up, say, your least favorite political candidate talking about the merits of eating babies, so the various echo chambers bore a kind of ghastly fruit, each stance finding its own "evidence" for its beliefs, right down to the flat earth types. Paranoia runs rampant, and so on. It looks like we're heading in that direction.

8n4vidtmkvmk 2y ago
You can still write it. It'll sound like you're just jumping on the AI bandwagon, but considering it's all the rage, it might still help your sales.

thih9 2y ago

We have 100s of tools that are about voice cloning - of course we’ll get content with cloned voices. Same as it happens with unauthorized use of someone’s images. And platforms and their moderation teams have processes in place to report and remove that. Looks like we need something similar for voice.

LegitShady 2y ago

I don't know where you live, but where I live I believe this sort of thing meets all the required elements for fraud.

surfingdino 2y ago

It's all fun and games until someone produces a recording of somebody else saying something incriminating and it will be used in court. This is the part of AI I hate.

8n4vidtmkvmk 2y ago
It'll be bad for a few years, but surely at some point it'll become inadmissible in court because it's too easy to fake, right? But then what do we do, if video and audio footage is inadmissible?
- Ylpertnodi 2y ago
  Where i am, video and audio are not admissible in court...'too easily faked'. A car bump i was in was dash-cammed, but all my team could do was second by second analysis of the video, and present that to the court. I did win, but it was very costly to do so.
  - 8n4vidtmkvmk 2y ago
    That's crazy. For dash cam videos you can at least compare if the footage matches with the damage on the cars. And hopefully both parties have video footage, then you can really compare if something doesn't add up.
echoangle 2y ago
Unless you also hate Image Editors, I don’t really get this point. Preserving forms of evidence isn’t really a primary concern when evaluating new useful technology.
left-struck 2y ago
It’s worse than that. People will start claiming that real, incriminating voice recordings of them are fakes as well. I think this matters more in the court of public opinion than in real court in both cases though.

cr3cr3 2y ago

Initially I'd say well if you're a public figure and upload your own voice online, of course this will happen. So its something to expect, however, this shouldn't be a problem for Jeff to solve... instead it should be YouTube's problem as they profit from the video monetization. Eventually they'll have to have some kind of detection for all uploaded content.

absentmoon 2y ago
I strongly disagree. I don't know the rights around one's own voice, but the idea that you suddenly lose ownership of something because you shared it online is the exact thing that many people take issue with when it is written in the terms of service for social networks, creator tools (adobe), etc.
- cr3cr3 2y ago
  I didn't mention ownership and I don't think you should lose it (nor does one lose it really even in this case, legally). But I do think that in cases like these, where there's money involved and YouTube, that they should have the means to prevent it.
  - johnnyanmac 2y ago
    Issue is that Youtube profits off it while signing away liability. There's no incentive to prevent it.

gyudin 2y ago

It's not even close to his voice lmao, just has similar cadence.

golol 2y ago

>I remember when OpenAI practically cloned Scarlett Johanssen's voice... There is absolutely zero evidence for this. I find it infuriating that this keeps being stated as a fact. So they go and hire a voice actor and clearly use her voice to train, but then they also scrape Scarlett Johansson from youtube and splice it into the training data to make the voice a bit more like hers? Really does that sound realistic?

klabb3 2y ago
Yeah it does. Motive: Altman had some weird boyish thing for her and they asked her first, she said no. Means: Lots of available data to use from her movies. They probably trained a model first without releasing it just because it’s ridiculously easy. Especially for OpenAI. Opportunity: AI is astonishingly good at laundering and remixing without exposing the training set, for previously-unseen levels of plausible deniability.
- golol 2y ago
  > AI is astonishingly good at laundering and remixing without exposing the training set They just about manage to make a good multimodal transformer that can generate audio and you expect that right away they can also interpolate in latent space? How does that actually work? It's not so simple. What benefit do they have from training on Scarlett Johansson's data, because they sure as hell have a big risk. They clearly hired a voice actress and they clearly told her to sound like Scarlett Johansson in "Her", and the end result perfectly fits with that. The voice doesn't "uncannily sound like SJ", no it just vaguely resembles her voice and mostly just mimicks the mannerisms from the movie. For me this is a perfect example of Occam's razor. One explanation is simple and realistic. The other explanation requires significantly more advanced AI control than OpenAI has claim/demonstrated, and it requires Altman to be so obsessed with the SJ idea that he goes out of his way to secretly train on her voice, risking legal exposure, while still hiring a voice actress.
  - klabb3 2y ago
    To be clear I’m not saying it did happen for sure. Only that it’s realistic and not exactly far fetched given everything that’s publicly known. > they can also interpolate in latent space? How does that actually work? I don’t know. I assumed it is the case because of the vast and excellent interpolation AI appears to do in other domains.

adityaathalye 2y ago

If LLMs are the ultimate remix machine, then is anyone with a RAG a digital DJ? One can't help but wonder what theft even means any more, when it comes to digital information. With the (lack of) legal precedent, it feels like the wild wild west of intellectual property and copyright law. Like, if even a superstar like Scarlett Johansson can only write a pained letter about OpenAI's hustle to mimic her "Her" persona, what can the comparatively garden-variety niche nerd do? Like Geerling, feel equally sad / angry / frustrated, but merely say "Please for the love of all that is good, be nice and follow an honour code.".

wruza 2y ago
what theft even means any more They dragged the term through different phases, but that’s just projection of will. Theft is undefined for objects with .copy() interface. It’s still there when you look at it. People have to adjust expectations, not laws. Computers replaced computers, now voice acting replaces voice actors. Your popularity doesn’t mean anything really and wouldn’t it be unfair if only popular could spare their jobs.
- the_gorilla 2y ago
  > Theft is undefined for objects with .copy() interface. > Computers replaced computers, now voice acting replaces voice actors. It's incredible what web development does to someone's ability to communicate ideas.
  - ninalanyon 2y ago
    The original meaning of the word computer was a person who calculates. Looks perfectly clear to me.
    - anal_reactor 2y ago
      I have a very funny example of how using outdated words might lead to miscommunication, but unfortunately that particular one could get me banned, so let's just say that I'm a huge fan of picking a version of langauge from one specific time period and sticking to it, instead of having the reader do all the guesswork.

scotty79 2y ago

> I remember when OpenAI practically cloned Scarlett Johanssen's voice Except that never happened and the voice belonged to a completely different voice actress and Scarlett Johanssen had exactly zero right to prevent this person from making money as a voice actress lending it to AI. These complaints remind me a little bit of the story that a man complained that his photo was used to illustrate the article about how all hipsters look the same and it eventually turned out it wasn't his photo.

probably_wrong 2y ago
OprnAI says it never happened. They also took the voice down despite having it front and center at release time. And let's not forget that, if they didn't want people to think of Scarlett Johansson, then there would be no reason for their CEO to tweet "Her".
- scotty79 2y ago
  "Her" is a brilliant movie about AI. The tweet was on point. And apparently they didn't give much crap about the specific voice since they taken it down because of mildly bad publicity. It was just supposed to be nice bonus gesture.
sourraspberry 2y ago
They reached out to Scarlett Johanssen. She said no. So they found a soundalike and Tweeted out references to the movie Her (starring Scarlett Johanssen as an AI chat bot) in the days leading up to launch. Scummy as fuck from OpenAI regardless of the technical legal rights and issues involved.
- scotty79 2y ago
  What's scummy about it? If you want a specific kind of voice and approach one person about it but she doesn't agree what's wrong with asking another one? If there were just two blonds in the world, one famous and the other not and you wanted a blonde actress for the role and she said no is it scummy to hire the other one? Is it scummy to hire "discount" Matt Damon instead of Matt Damon?

swag314 2y ago

https://www.youtube.com/watch?v=IeTybKL1pM4 https://www.youtube.com/watch?v=IeTybKL1pM4

cranium 2y ago

(Obviously not a lawyer) Overlooking the AI part, isn't it a gross misrepresentation of Jeff's opinion or an unauthorized use of his image? By using his voice, it creates an implicit (fabricated) endorsement for their product and that feels very wrong. I'm sure laws exists to deal with these cases, since way before AI existed.

mft_ 2y ago
I’ve been thinking something similar recently. We’ve had people who are skilled voice mimics for ever, and they mostly exercise their skills for comedy/satire, and not for misrepresenting people’s opinions. IANAL either but I guess this is based on solid legal grounds, and misrepresenting people would be relatively easy to deal with legally. I guess the difference is democratisation - we’ve moved from very few people having this skill, to virtually anyone with a computer being able to do something similar. And so policing it will be much tougher, and likely beyond the means of someone like Jeff Geerling if it would require legal action to remedy.
- aversis_ 2y ago
  Just wait till someone starts auto-deepfaking their way out of college exams and job interviews. Computers made graphic design approachable, but early adopters oversaturated the market before it stabilized. We’ll eventually figure out social norms and regulations for AI voice mimicry too, but there will be chaos first. Also, tech always moves faster than law. By the time courts catch up, this will be old news.

bookofbiden 2y ago

[dead]

ryzvonusef 2y ago

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And this was before AI was easy to access. You can say a lot of things about 'oh backward countries' but this will not stay there, this will spread. You can't just give a toddler a knife and then blame them for stabbing someone. Has nothing to do with fame, with security, with copyright. This will get people killed. And we have no tools to control this. https://x.com/search?q=blasphemy https://x.com/search?q=blasphemy I fear the future.

pnut 2y ago
I guess then, you should use AI to generate videos of all of the lynch mob leadership committing blasphemy and let them sort it out internally?
- ryzvonusef 2y ago
  You joke but, given the religious/sectarian nature of the issue, all it does is empower one sect to act against the leaders of the other sect. Check the twitter link, you won't have to scroll much to find a mullah being blasted for blasphemy. No one is safe.
  - javcasas 2y ago
    Well, then target dead people. Find someone who died 6 months ago, have AI make them a blasphemer, have the sect look for dead people. After the third or fourth time, they'll stop. Not because they want to stop, but because they don't trust this being another prank by Mullah Achmed the Fool.
    - ryzvonusef 2y ago
      Just like how you are the traffic, you are also the mob Mobs are not formal organisations with leadership and membership dues. Mobs are general everyday people who are sick and tird of this miserable life and need an excuse to rabble and rouse. So in this case, fake dead person's blasphemy will only be channeled towards their family members, mobs just need an excuse to get crazy. AI is not what causes mobs, it just helps lower the incitement barrier by providing quality bait.

aaron695 2y ago

[dead]

4ndrewl 2y ago

Let's just assume you can't trust any ugc on the internet from now. It's all done, but fun whilst it lasted.

paganel 2y ago

For what it's worth this seems to be a thing, for example here's a video promoting some batteries-thingie with the help of AI-voice based on a (famous, in some circles) podcast girl. [1] https://old.reddit.com/r/redscarepod/comments/1fmiiwt/which_one_of_you_did_this/ https://old.reddit.com/r/redscarepod/comments/1fmiiwt/which_...

deleted 2y ago
[deleted]

sandreas 2y ago

This is exactly the reason why I'm not open sourcing a tool I developed where you can take an audio book together with an epub to build an ljspeech dataset and train a voice model. Although it was not too hard to create I believe making it easier is something i don't like to achieve... I hate to say this but ruining a narrators existence with AI seems to get easier every day.

sureglymop 2y ago
I do think that the floodgates are open now. AI is still absolutely terrible to mediocre at coding at best but for more trivial tasks like this it'll get indistinguishably good AND businesses are ready to sell before even beginning to think of the consequences.
- sandreas 2y ago
  This. I'm still wondering why audio book giants like amazon/audible are not having license contracts with their narrators to do supportive narrating tasks by AI paying Them a fee for AI generating their voice. It probably would be win win... Regarding how easy it was to clone my favorite narrators voice with open source tools I'm a bit afraid of what amazon could do with a whole cloud and massive man power

moffkalast 2y ago

Yeah it's pretty much an inevitability that anyone who's ever posted more than 10 seconds of their voice will have it stolen eventually if anyone has a motive to do it. That's already all it takes to few shot a decent TTS _today_. Most likely all existing youtubers will have complete voice and video digital clones made out of them. Then you can also tune an LLM on their scripts and it'll respond in the same character as well. In theory you could also bring back ones who are dead, which would be very interesting in a historical sense. Like if we had hundreds of hours of Napoleon talking in front of a camera, it would be trivial to recreate a digital version of him for anthropologic study, maybe even having various figures debate things with each other. That's what historians a century later after we all die will be able to do with impunity.

rishikeshs 2y ago

Slightly off topic, but what’s that logo at the bottom of his website? Is that some sort of a coat of arms?

geerlingguy 2y ago
It's part of the coat of arms for the Geerling family, yes.
- rishikeshs 2y ago
  Thanks. Would be great if you could write a bit about it.
  - geerlingguy 2y ago
    Honestly I'm not the heraldry guy or ancestry guy in my family, but I thought it looked cool and helped make an SVG of it for a shirt my uncle made for all the family one year, so I slapped it in the footer of the blog :)
    - rishikeshs 2y ago
      Haha that’s wonderful. I’m fascinated by coat of arms. I’m from Southern India, and We don’t have anything like this except for may be royal families. I’m thinking of making one myself, but not sure how this works and hence I asked! :)
      - geerlingguy 2y ago
        Heh, the Geerling family is far from royal, it's just something a lot of families in Europe seemed to do (heraldry), and the idea is pretty neat—try to add symbols to your family crest that you want your family to try to strive for (like honor, strength, love, peace, etc.). I think we even did a thing in one of my art classes on heraldry once, and the idea was to try to figure out what symbols you would choose for your own personal crest, based on what kind of things we'd see in historical ones.

benterix 2y ago

Elecrow seems a Chinese company, right? In that case, I don't expect any reply.

pmarreck 2y ago

[flagged]

maximus-decimus 2y ago
The first one sounds more like Scarlet Johanson to me. It's not just about the voice itself, it's about the sensuality and gravitas of the voice. The first one has Scralet's warmth and feminity whereas the second one sounds a robot impersonator trying mimicking scarlet's voice without understanding emotions. So to me you really proved that the voice actress sounds more like Scarlet than a literal AI replication of Scarlet. If that's now damning, I don't know what is.
- pmarreck 2y ago
  [flagged]
ahiknsr 2y ago
> Scarlett's baseless accusation https://www.theguardian.com/film/article/2024/jul/18/scarlett-johansson-chatgpt-voice-openai-sam-altman https://www.theguardian.com/film/article/2024/jul/18/scarlet... > The actor made headlines earlier this year when she issued a public statement saying OpenAI’s chief executive, Sam Altman, had contacted her in September 2023 to provide the voice for Sky, in an ode to her role in the Spike Jonze film Her, but she had declined for “personal reasons”. In May, as the company released Sky, Altman shared a tweet that simply read “her”.
- pmarreck 2y ago
  This is not evidence, and it is still baseless. They literally hired a completely different voice actress. Nothing else needs to be said. What the fsck does it matter if it was "similar to" Scarlett Johannsen, even if it was (which it wasn't)? Should an actor be able to sue another actor who looks similar because of the work they got?
127 2y ago
I'm in the camp that thinks celebrities take way too much mental space of the regular person on the internet. It distorts ones ability to differentiate as there are these huge gravity centers that everybody makes comparisons to, even subconsciously. If Scarlett Johansson wasn't famous and Sam Altman didn't offer her the job, everybody would have no trouble in recognizing the differences in the voice. Priming is an extraordinarily powerful cognitive bias.

eth0up 2y ago

My aunt is (supposedly) in her 90s, having left the US for Ecuador 10 or so years ago. We've remained in regular contact via phone over the years. Recently we went more than a few months without speaking. She has/had two numbers; magic jack and google. When I tried to call her, the magic jack was no longer in service and google said something about "unavailable". I reached out to my cousin (my aunt's daughter) to inquire. I was told her number (and perhaps other things) had been "hacked", whatever that means. She had recently broken her hip and was in a hospital recovering. With this on my mind, I received a call (from the google number), strangely, while processing files with GPT. My skepticism was primed and ready, possibly making me paranoid. However, I did my due diligence and asked dozens of questions, mostly boring things that she typically wouldn't have patience for. Sometimes she'd reply with a reasonable answer and sometimes not, which made it difficult to evaluate. Toward the end, I asked where she was. She said, with an awkward tempo "I'm at home, in Cuenca", which I found odd because she'd normally just say she was at home, period. I then pressed her to tell me where she was before she returned home. She said she didn't understand. I rephrased the question, stating that it was a simple inquiry, eg "where were you before going home?" She said "this is getting too strange and confusing " and killed the call. I notified my cousin, telling her I thought something was suspicious, still cognizant of all the characteristics one would expect from a 90 year old recovering from a serious injury. My cousin might, technology wise, be in AOL territory. About 5 days later, I received a call from my aunt, on the google line. This time,I was more passive and cautious, but again, asked dozens of boring questions to probe the situation. I was surprised by both her ability to answer certain questions and also her inability to answer some questions. I tried to ask questions on topics we'd never discussed, in case the line had been tapped for a long time and referencing was established by an imposter. I had begun to suspect I had been paranoid. But several aspects were burning me: 1) typing noises in the background 2) Shatneresque pauses for nearly every reply 3) refusal to answer some specific questions. At the end of our apparent conversation, I asked her to do a very serious favor for me: send me a selfie, with one hand making the thumbs up gesture. She replied "I'll send you a photo of my passport ". I replied "that's stupid, ridiculous and serves no purpose. Don't do that. Understand? Do NOT send me a passport photo. I'm asking you something very important. Do exactly what I asked. Will you do this?" Her reply: "yes. What is your email address?" This was odd. I told her she already knew and it's the same one she'd had for years. She asked that I tell her anyway. Ok, 90 years old, traumatic injury, possible prescription drugs... "It's my full name @ xyzmail com". We killed the call. I immediately called my cousin and told her of my suspicions, including some my aunt's babbling about all her finances and accounts being inaccessible. She said that was strange because she just deposited 8k into her account. Meanwhile, a notification appears in the phone, an email from my aunt. It's a photo of her passport. Having no authority in this situation, but plenty well annoyed, I immediately jumped on a real computer and ran the photo through exiftool. The photograph was taken in 2023 and it was August of 2024. I then grabbed the geo coordinates (cryptically presented in exiftool) and with some effort, geolocated the image to right on top of her former residence, in Cuenca. I still don't know WTF is going on and my cousin thinks I'm a dingbat. But what I know for sure, is this is an age where such things are plausible enough and will soon be inevitable. The way I think may be deranged, but I truly don't even know if my aunt still exists. But I can have a pretty compelling conversation, either with her, or something strongly resembling her, minus the Shatneresque pauses, typing noises and selective amnesia.

vonnik 2y ago

California banned several forms of deepfake and digital replicas without consent just days ago. https://techcrunch.com/2024/09/19/here-is-whats-illegal-under-californias-8-and-counting-new-ai-laws/ https://techcrunch.com/2024/09/19/here-is-whats-illegal-unde... Not sure if those laws apply to Jeff tho, as they concern porn, politics and employer contracts.

djoldman 2y ago

> I remember when OpenAI practically cloned Scarlett Johanssen's voice ... I don't have a dog in this fight but just to be clear, OpenAI has stated that they paid a voice actor to create the voice ("Sky") that sounds like Scarlett Johanssen. There was no "cloning" or "stealing" (that they say). https://openai.com/index/how-the-voices-for-chatgpt-were-chosen/ https://openai.com/index/how-the-voices-for-chatgpt-were-cho...

exitb 2y ago
This is subtly wrong. They tried to hire the celebrity, got refused, then hired a different talent to do her “natural voice”. The official story is that it just happens to sound alike.
kbelder 2y ago
It's interesting that this rumor has been pointed out at least three times in this discussion, and every time it's voted down. Doesn't fit with the passions of many posters, I guess.
- johnnyanmac 2y ago
  because it's an uncharitable interpretation at best, and misleading at worst. They approached her, they refused, and then they hired a sound-a-like. That is textbook Ford v Milder. They may have been absolutely fine if Altman never approached Scarlet. But context matters.

ei23 2y ago

I’m a small tech YouTuber and I’ve also had contact with Elecrow. As far as I know, employees (not just at Elecrow) receive rewards, promotions, or commissions when they secure long term partnerships and video collaborations with YouTubers. Perhaps someone thought it would be clever clone Jeffs voice since his channel is quite popular in this field. This certainly isn't great PR for Elecrow right now. I would also wonder if they will confess to that this was intentional...

hcks 2y ago

[flagged]

znpy 2y ago

We’ve been in a post-truth world for T least ten years anyway. We already had fake news and organizations willingly spread fake news. We had clearly fake pictures and people believing that. Flat-earthers, no-vax and whatever. This is just another brick in the wall.

segmondy 2y ago

What if they had someone that sounded just like you and had the person do a voice over? What if they had someone that sounded like you, had the person give them sample and used AI to generate voice? What if the hired someone that could imitate your voice, then had the person give them sample and used AI to generate voice?

singleshot_ 2y ago

When you say that lawyers always cost a lot of money: I’d absolutely do this pro bono but more than likely you’re not in a state where I’m licensed. You can absolutely positively find a free lawyer if your issue is interesting enough. This is the most interesting issue of our day.

cityzen 2y ago

Ex-Google CEO says successful AI startups can steal IP and hire lawyers to ‘clean up the mess’ / “But if nobody uses your product, it doesn’t matter that you stole all the content,” Eric Schmidt said during a recent talk at Stanford that has been taken offline. Since that guy was CEO of Google it’s all good right??? https://www.theverge.com/2024/8/14/24220658/google-eric-schmidt-stanford-talk-ai-startups-openai https://www.theverge.com/2024/8/14/24220658/google-eric-schm...

johnnyanmac 2y ago
well that does nail down one huge societal issue that multiple domains need to address: fine and punishments are so low that they are simply an expense instead of a deathblow. We definitely need to overhaul a lot of these white collar fines. I just watched a video today and learned the federal maximum fine for being caught using child labor is capped at $15k per worker. No wonder child labor has skyrocketed over the decade.

veunes 2y ago

Investment in tools that can verify the authenticity of audio and video content is crucial

t0bia_s 2y ago

We should adopt and use to it. There will be more and more fake AI created content every day so we should be confrontend with it to learn how to react to it. Regulating prolong adoption and take resources.

oehpr 2y ago

I want to ask and answer two of my own questions here: 1. Why clone Jeff's voice? When I was messing with stable diffusion using Automatic1111's interface, I noticed it came with a big list of artists to add to the prompt to stylize the image in some way. There was a big row in the media about ai art reproducing artists work and many artists came forward feeling it was a personal attack. But... I mean the truth is more general than that. When I pressed a button to insert a random name into a prompt, my goal was not "yes give me this person's art for free", it was "style this somehow". I wasn't personally interested in any particular artist, I honestly would have preferred a bunch of sliders. Jeff here is clearly a good speaker. That's a practiced talent and voice actors exist because it's hard. Elecrow wanted a voice over and they wanted it to be as good as they could make it. Jeff is very good. So did they want Jeff? I think what they really wanted was a good and cogent narration with the tenor of a person. Not a machine making noises that sound like english. If they had an easy way to get that, we wouldn't be talking about it here. 2. What function does copyright serve? Well. I think a reasonable argument would be that if people were able to reproduce your work for free, you would quickly find yourself without a monetary incentive to make more of it. So. What happens if you combine answer 1 with answer 2? I think it leads to: "We should consider making it illegal to automatically reproduce the work of an artisan.", you know, the luddic argument. An argument that has been perceived to be, more or less, settled. So it seems to me: That for individuals, harms matter, and for society, it doesn't.

1shooner 2y ago
>I think what they really wanted was a good and cogent narration with the tenor of a person. Not a machine making noises that sound like english. If they had an easy way to get that, we wouldn't be talking about it here. I think the general assumption is that they wanted to, at the very least, strongly imply his endorsement of the product or video. Which I would say they did effectively. If I had happened on a clip of one of these videos outside the context of this controversy, I could have easily gotten the impression he was working with the vendor.
lesostep 2y ago
>> But... I mean the truth is more general than that. When I pressed a button to insert a random name into a prompt, my goal was not "yes give me this person's art for free", it was "style this somehow". yeah and that's the problem. The style of an artist is a developed thing. To think that one could borrow your style not through learning and caring, but through mathematically analyzing the width, and colors, and patterns and applying it to a random noise — that's kinda insulting. If nobody cares about my real work, why do they care about using my style, then? Develop your own an teach your AI on that, if there really isn't any difference. People say that AI learns how a human would. But a human wouldn't (couldn't!) learn like an AI can. He can't look at the pixels, can't mechanically churn through patterns. If someone can learn from art like AI learns art, I would also be opposed to them learning anything from me :D

Corrado 2y ago

So, do we "solve" this by impersonating a voice from a very litigious company? Let's say we clone Micky Mouse's voice to say some blasphemous things. Does that bring the full weight of Disney down on this? Will that force a change? Would it be a good change?

ycombinatrix 2y ago
isn't Mickey in the public domain now

rldjbpin 2y ago

from the discourse here, the main pain point really is the accessibility to do something like this thanks to the new models. IANAL and not sure about regional precedence on these topics, but there are plenty of ads where lookalikes or voice actors are used to use someone's likeness. they are mostly in satire, but there is yet to be a case where there was a litigation over this or prior approval needed. we have ai-based voice abuse in the political sphere, and where there was only one legislation for banning the use in voice calls for one country (https://news.ycombinator.com/item?id=39304736 https://news.ycombinator.com/item?id=39304736), another country actively used the same underlying tech to aid their own rallies (https://news.ycombinator.com/item?id=40532157 https://news.ycombinator.com/item?id=40532157). the tools are here to stay, but what is fair use needs to be defined more than ever.

johnnyanmac 2y ago
>they are mostly in satire Satire is one of the few use cases of fair use that hasn't been torn down. So that tracks. > there is yet to be a case where there was a litigation over this or prior approval needed. There's quite a few over impersonation. Most broadcast media knows how to skirt the rules though.

milabogdanova 2y ago

[dead]

m3kw9 2y ago

how do people know if its a very similar sounding voice, or identical?

johnnyanmac 2y ago
Not to be crude, but they have ears. and you'll hear uncanny likeness of a voice you follow everyday when it comes up. It's kind of like saying "how do we know that face is similar/identical"? Humans are surprisingly good at knowing when something feels off in other humans. Even if we lack the vocabulary to fully explain the difference.
- m3kw9 2y ago
  I think you unknowingly paired the speaking style, tone patterns, levels and spacing etc which isn’t actually part of their voice, but combining it with the voice print you get a clear picture. If someone cloned my voice and start speaking differently from me, different levels(soft loud) and have an accent, nobody would recognize it was me.
- echoangle 2y ago
  The problem is that there are a lot of humans, and a lot of humans with online presence. I wouldn’t really be confident in identifying a „stolen face“ online because you’re bound to look similar to someone when creating a face. Does that mean that the look was stolen or is it just coincidence? What is the likelihood of a random AI voice sounding similar to some real online creator?
  - johnnyanmac 2y ago
    >I wouldn’t really be confident in identifying a „stolen face“ online because you’re bound to look similar to someone when creating a face. Does that mean that the look was stolen or is it just coincidence? Luckily, national ID can easily solve that problem offline. It'd be a more interesting problem if some company searched far and wide to identify someone that has a similar face to [famous person]. And that's part of the issue in that there's no way to verify "is my voiced sampled here?". Opening the black box can help with that, but that's something companies are fighting to the death to not allow. > What is the likelihood of a random AI voice sounding similar to some real online creator? Given how much data was scraped, less likely than the fact that their data is in fact in the training set.