6 ms·
AI speech generator 'reaches human parity' – but it's too dangerous to release
- AlexDragusin 2y ago"Too dangerous to release" - I could use that line to promote my services :)
- jd115 2y agoI'm old enough to remember some number of months ago when GPT2 was described as "too dangerous to release".
- rmorey 2y ago“some number” would be 65. things change
- kbelder 2y ago>things change Often predictably.
- 01HNNWZ0MV43FF 2y agoRemember when the PlayStation 2 was "technically a supercomputer" and taking LSD a certain number of times made you insane? Great moments in marketing history
- klowner 2y agoOh yeah I vaguely recall news reports about how the PS2 was so powerful it could be used for missile guidance or some nonsense?
- abainbridge 2y agoAn article about that from 2000. https://www.theregister.com/2000/04/17/playstation_2_exports/ https://www.theregister.com/2000/04/17/playstation_2_exports... Brilliantly it says: Register readers with very long memories indeed will recall similar concerns being raised over Sir Clive Sinclair's ZX-81. The fear then was that the sneaky Sovs would try to buy heaps of ZX-81s for their Zilog Z80-A CPUs and might 1KB RAM to upgrade their nuclear missile guidance systems.
- jayGlow 2y agofunnily enough the ps3 was literally a super computer, at least if you hooked up enough of them https://phys.org/news/2010-12-air-playstation-3s-supercomputer.html https://phys.org/news/2010-12-air-playstation-3s-supercomput...
- Paul-Craft 2y agoHuh. Today, a college kid without even any underlying knowledge of the math can train their own GPT-2-level language model as a semester project.
- deleted 2y ago[deleted]
- devmor 2y agoI would say that this take was correct, just not in the way the detractors at the time intended. The danger was to the usefulness of the internet. I have yet to see any benefit to society from GPT's improvements, but I do see the internet quickly becoming more and more unusable due to the inundation of machine-generated spam on nearly every communications platform.
- goatlover 2y agoBy more unusable, do you mean more dead? As in LLMs are helping make the Dead Internet Theory real?
- devmor 2y agoIn the most "doomer" possible view of my beliefs, yes, that'd be an accurate description. I don't know if that will actually come true, of course - human society is pretty resilient in the face of its own self-caused adversity, but currently it does already extract a significant mental and emotional cost dealing with filtering out the "human mimics" on many platforms - especially search engines.
- Slyfox33 2y agoRiiiight
- pphysch 2y agoSpeech generation has gotten really good, but there's simply no way to faithfully recreate someone's vocal idiosyncracies and cadence with just "a few seconds" of real audio. That's where the models tend to fall short.
- michaelbuckbee 2y agoThis was my thought as well, but someone pointed out to me that regional accent identification captures a large percentage of cadence and inflection differences (specific word choices and turns of phrase obviously would still not be there).
- Melomololotolo 2y agoI don't think it's hard to get more than a few seconds of voice from many people. 'hi, sry to call you I'm Cindy and I'm from your insurance. I'm calling regarding your car crash ...'.
- criddell 2y agoFew seconds means less than a minute. That’s not nothing. Look at a clock and talk for a minute — it’s longer than you might think. Do you think you could give a recording of a minute of someone talking to a talented impressionist and they could impersonate that person to some degree? It doesn’t seem that far fetched to me.
- pphysch 2y ago"Few" doesn't mean <60 it typically means ~5 or <10. Getting high-quality audio for an arbitrary private citizen via public means isn't that easy, especially for folks like me that don't post video on public social media and use automated call screening and never say a word until the caller has been vetted.
- mcpar-land 2y agoI can believe a speech generator too good to release, but not even a perfect algorithm can get every one of your inflections and verbal tics with just a few seconds of sample material. Makes me think the whole thing is bs. I instantly see any "ooh our thing we are making on purpose is so dangerous oohhh" as an attempt at regulatory capture until I see proof of the danger.
- chx 2y agoThings are progressing just as https://youtu.be/xoVJKj8lcNQ https://youtu.be/xoVJKj8lcNQ predicted. > so 2024 will be the last human election and what we mean by that is not that it's just going to be an AI running as president in 2028 but that will really be although maybe um it will be you know humans as figureheads but it'll be Whoever greater compute power will win We saw already AI voices influencing elections in India https://restofworld.org/2023/ai-voice-modi-singing-politics/ https://restofworld.org/2023/ai-voice-modi-singing-politics/ > AI-generated songs, like the ones featuring Prime Minister Narendra Modi, are gaining traction ahead of India’s upcoming elections. [...] Earlier this month, an Instagram video of Modi “singing” a Telugu love song had over 2 million views, while a similar Tamil-language song had more than 2.7 million. A Punjabi song racked up more than 17 million views.
- BadHumans 2y agoUnfortunately in the US there is a political party that is attacking education and doesn't want people to learn critical thinking skills at a time when critical thinking is sorely needed. They happen to really "love the poorly educated" for some reason.
- Liquix 2y agoclassified tech is generally at least 10 years ahead of anything the public has access to. judging by how bizarre and polarizing the previous two US elections have been, i wouldn't be surprised if this prediction had already played out and we just didn't know it yet
- zazazache 2y agoWhat is the point of them trying to create this? That something like this would mostly be used to create disinformation and create chaos is easily understood before making something like this. Truly irresponsible
- Paul-Craft 2y agoYou're right, of course. Unfortunately, however, we're all just actors in a giant, multi-player, iterated Prisoner's Dilemma here. If I decide not to pursue human-level automated speech generation, or I end up developing it and don't release because it's "too dangerous," someone else will just come in behind me and take all that market share I could have captured. It's like we're stuck in some movie that came out in 1994[0], or something. Except, in this version, everything is gonna up sooner or later, anyway. Might as well profit from it along the way, right? Le sigh. --- [0]: https://www.imdb.com/title/tt0111257/ https://www.imdb.com/title/tt0111257/
- HeatrayEnjoyer 2y agoMove fast and break things (including organized society). I can't even think of non malicious uses that are anything more than novelty or small conveniences. Meanwhile the malicious use cases are innumerable. In a just world building this would be a severe felony, punished with prison and destruction of all of the direct and indirect source material.
- ninjanomnom 2y agoAt least one good use is for video games where the text of some dialogue is determined when you run the game. For example in a game I work on player chat is local and voiced by tts configured by the player for their character.
- mensetmanusman 2y agoCancer that takes someone’s voice.
- ryandrake 2y agoThis is what happens when you have an industry full of people "looking for challenging problems to solve" without an ethical foundation to warn them that just because you can build something doesn't mean you should.
- digitalsushi 2y agoweakest link, if they dont release this, someone else will release one. every time someone noble invents another gen ai toy/weapon, they lock it down with post filters so it cant be used for evil, and then a second person forks it, pops the safeties off, and tells the world to go nuts. social solutions take too long to use against the tech, but tech solutions are fallible. to be defeatist about it, there's going to be a golden window of time here where some really nasty scams have no impedance.
- cuddlyogre 2y agoIf you can't trust the mail, the phone, or your computer, make them come to your house with the sheriff. It's almost as if a consumer protection agency should be created and funded to protect consumers.
- bitshiftfaced 2y agoToo dangerous for PR reasons at least until after November.
- AnimalMuppet 2y agoIf November is the reason, well, this isn't the last November with that concern...
- bitshiftfaced 2y agoFrom a big tech POV, a political scandal would be most PR-damaging right before a big US election. The time from right after to two years before or so, it wouldn't be as big a deal. Once you open Pandora's box, then a scandal in the next election cycle wouldn't be as big of a deal in terms of the company's responsibility/optics since the tech is already out there and it's a "new normal" at that point.
- rentonl 2y agoOf course, this technology must only stay in the hands of our trusted corporate overlords.
- devmor 2y agoUnless of course, you're willing to pay a monthly subscription fee for access to it. Then the danger has been mitigated.
- thevillagechief 2y agoAh, the old "it's too dangerous to release" marketing move. Why even tell us about it?
- cuddlyogre 2y agoFor the extra dollars they'll obtain when it is no longer too dangerous to release.
- HeatrayEnjoyer 2y agoNot everything is a conspiracy. It is very plausible they're speaking genuinely.
- fooqux 2y agoAnd it's even more plausible this is just a marketing play to build hype. Take, for example, you just made some new super-pathogen in your basement lab. It could kill everyone on the planet. This is obviously pretty dangerous, so do you: A) silently dispose of it and hope nobody else ever makes the mistake of creating it. or B) keep it in the freezer and hold a press release about how you made it but it's too dangerous to share any details.
- HeatrayEnjoyer 2y agoNo, it's not more plausible.
- illiac786 2y agoI am utterly impressed by the logic of this argument.
- Tao3300 2y agoThis isn't the first time someone has spoken like this. It won't be the last. It's 100% marketing.
- abroadwin 2y agoRelevant research post from Microsoft: https://www.microsoft.com/en-us/research/project/vall-e-x/vall-e-2/ https://www.microsoft.com/en-us/research/project/vall-e-x/va...
- carapace 2y agohttps://arxiv.org/abs/2406.05370 https://arxiv.org/abs/2406.05370 > Our experiments on the LibriSpeech and VCTK datasets show that VALL-E 2 surpasses previous systems in speech robustness, naturalness, and speaker similarity. It is the first of its kind to reach human parity on these benchmarks. Moreover, VALL-E 2 consistently synthesizes high-quality speech, even for sentences that are traditionally challenging due to their complexity or repetitive phrases https://www.microsoft.com/en-us/research/project/vall-e-x/vall-e-2/ https://www.microsoft.com/en-us/research/project/vall-e-x/va... > This page is for research demonstration purposes only. Currently, we have no plans to incorporate VALL-E 2 into a product or expand access to the public. If you go back and look at older cities they almost all have the same pattern: walls and gates. I figure now that the Internet is a badlands roamed by robots pretending to be people as they attempt to rob you for their masters, we'll see the formation of cryptologically-secured enclaves. Maybe? Who knows? At this point I'm pretty much going to restrict online communication to encrypted authenticated channels. (Heck, I should sign this comment, eh? If only as a performance?) Hopefully it remains difficult to build an AI that can guess large numbers. ;P
- spywaregorilla 2y agoI really want something that can do a voice change and match the emotion and articulation of a voice clip that I provide. I don't care (or want) it to be based off a real person and the manners in which they would tend to articulate a sentence. Are there any decent open models out there?
- woodson 2y agoTry StyleTTS2. You will still have to experiment with the settings a little to get the right level of adherence to the reference speaker’s voice and the emotion content.
- spywaregorilla 2y agoWithout looking at this, are you sure that this can do speech to speech? Maybe my flaw in searching has been disregarding anything that's called "text to speech" as not also "speech to speech"?
- woodson 2y agoAh my bad, you’re right. I think it doesn’t do V2V directly, but can use reference audio to guide TTS.
- deleted 2y ago[deleted]
- hi_dang_ 2y agoThe classic Steven Seagal “my hands are weapons and I need a license for them” rhetoric. What a crock of shit.
- mensetmanusman 2y agoA gun can help rob a bank. A speech generator can help rob 1000 banks.
- exe34 2y agoit might help if the dumb dumbs at the bank would stop trying to make me say "my voice is my password". I've been careful to only say "no fcuk off you fcuking numpty who came up with this idea after voice cloning hit the mainstream".
- htrp 2y agohttps://arxiv.org/pdf/2406.05370 https://arxiv.org/pdf/2406.05370 The model in question is Microsoft Vall-E2 without the click bait headline.
- coeneedell 2y agoThese samples are terrible when compared to commercially released models like from eleven labs or playht. This is an extension of an interesting architecture but currently those more traditionally based models are way more convincing.
- BoredPositron 2y agoI can't wait until the free base models get better. The floods on tiktok, shorts and stories with the standard eleven labs voice is getting nauseating.
- dspillett 2y agoPeople saying “too dangerous to release” usually means one (or more) of three things: 1. “… but if you and your big rich company were to acquihire us you'd get access…” — though as this is MS it probably isn't that! 2. That is only works as well as claimed in specific circumstances, or has significant flaws, so they don't want people looking at it too closely just yet. The wordage “in benchmarks used by Microsoft” might point to this. 3. That a competitor is getting close to releasing something similar, or has just done so, and they don't want to look like they were in second place or too far behind.
- exe34 2y agoI too have an agi in my basement, but it's too dangerous to release! wanna give me some cash?
- ChrisArchitect 2y agoProject page: https://www.microsoft.com/en-us/research/project/vall-e-x/vall-e-2/ https://www.microsoft.com/en-us/research/project/vall-e-x/va... > "This page is for research demonstration purposes only. Currently, we have no plans to incorporate VALL-E 2 into a product or expand access to the public."
- thedonkeycometh 2y ago[dead]