10 ms·

Faux Rogan

220 points by mgdo 7y ago

MeltySmelty 7y ago
laughably easy to spot the fake. uncanny valley out the wazoooooo
kepler 7y ago
Solid work they both sound exactly the same to me. The Faux Rogan is a slightly faster speaker, so it was pretty easy to differentiate between the two.
dogma1138 7y ago
Got all right it seems that the faux Rogan talks a bit too fast but anyone who doesn’t listen to his podcast regularly likely won’t notice it. I wonder how long it would take until Joe brings those guys to his podcast that is something I’ll be waiting for.
siedes 7y ago
It was easy to figure out which are the fakes, but I am very impressed and spooked at the idea of this technology improving, then getting into the wrong hands.
- sho_hn 7y ago
  I'm not too spooked. I think as a society we're just going to learn to distrust recorded+reproduced media by default. Already we don't trust photos much for their potential to be a manipulation. You could call this a loss, sure - but maybe we should just never have in the first place. Photo manipulation is a very old practice. There's already businesses that sell camera apps (or cameras? my memory is a bit dim) that save a photo along with a cryptographic hash to prove authenticity. Their customers are for example insurance companies, which require their clients to take pictures of damaged property etc. for claim filings that way.
  - blotter_paper 7y ago
    Yup. Media will be signed cryptographically and your belief in the media will correspond to your trust in the signer. It will destroy the usefulness and damage of anonymous publication of recorded evidence. Bad for transparency, good for privacy.
    - YaxelPerez 7y ago
      We can't even get people to look for a green lock icon in their URL bar. I really doubt the kinds of people that are most vulnerable to these attacks will suddenly develop an interest in cryptography. And if the fake confirms someone's worldview or serves their interests, confirmation bias will take over and no "fancy math" is going to change their mind.
      - blotter_paper 7y ago
        I don't think we'll adapt "suddenly," I expect we'll have to experience some major news stories getting redacted due to deepfaked evidence before this reaction becomes mainstream, but I think it's coming. I also don't think an interest in cryptography is necessary, it's not like I expect people to read public keys directly. There will be some system that shows something along the lines of "Confirmed by $USER_NAME" under the media. You don't need an interest in video codecs to appreciate YouTube. And I have seen people reject faked evidence, such as fake screencaps of tweets, even when it backed up their worldview. We're not perfect at it, but I think we're getting better.
  - cco 7y ago
    Do you not see the problem in societies being unable to share trusted and accurate information about reality? I mean that is 100% required for any interpersonal relationship between two people let alone a community.
    - deleted 7y ago
      [deleted]
    - 7y ago
- arebours 7y ago
  > It was easy to figure out which are the fakes Yeah but would you figure it out if you just listened to it expecting Joe Rogan?
  - siedes 7y ago
    It would be slightly harder. What I'm looking out for here is voice distortion and the unnatural flow of speech, and it's present in every Faux recording. It's just unnatural compared to how JR normally sounds.
craftinator 7y ago
I didn't do so well. I was 60% correct. The non-word noises were really good with FR, pauses and breaths did a great job of convincing me. The one thing that I picked up on quickly though, was that FR speech was a bit more slurry and fast than JR speech. Interesting...
sho_hn 7y ago
My reliable tells for the Faux Rogan were a certain amount of noise/distortion to the audio, and the real Rogan doing more complex things with intonation/prosody. The latter requires a direct comparison, and the former must be fixable with filtering? That makes it pretty close to good enough for an original reading.
- codetrotter 7y ago
  Same here. Have only watched one or two episodes of his podcast in the past, so was basing my judgement on what sounded natural in general rather than any specific recollection of what he is supposed to sound like. I voted on each clip without listening to other clips first. I misidentified two that were real as being fake. Wonder if those might have been clips where he was reading from a script with a flatter voice than usual like another top-level comment ITT talks about. If audio of this quality was being presented in a context where I didn’t know it might be fake, I probably would have chalked it up to issues with the recording mainly, and would have believed that the audio was real.
  - sho_hn 7y ago
    An interesting question (ignoring how you need Real Rogan to train Fake one for a moment) is whether the Fake Rogan could be as popular, or if the extra personality the real voice has is vital to making him a popular speaker. I suspect that question strikes to things like gaining the tools to transpose speaker affect independent from voice characteristics etc. Imagine a remix culture where a future popular podcast voice is a mashup of different older popular speakers and things like that ...
    - soulofmischief 7y ago
      Reminds me of Aldous Huxley's Brave New World when the characters attend a synthetic orchestra. The book goes into vivid detail about the level of craftsmanship achievable by a machine composer and orchestra, yet somehow it all seems so shallow, like the soul of it has long gone.
    - 7y ago
overgard 7y ago
Gotta admit, I listen to his podcasts sometimes and I only hit 50%.
starpilot 7y ago
Got 1 wrong, and I've never listened to Rogan. Seemed the fake one talked too monotone.
- axaxs 7y ago
  I noticed that, and also that the spacing in words was too uniform.
- cm2187 7y ago
  What I am curious about is how much selection was involved in the fake ones. Like is it the cream of the top of the fake ones that sound the most natural or is it a typical output?
- daenz 7y ago
  The "warbling" factor stood out to me. It's like you could hear the interpolation between audio samples.
  - mrmondo 7y ago
    Yeah, there was some strange timing between sentences and words at times.
arebours 7y ago
Damn it's good. I was able to spot most of the fakes but hadn't I known there are any I don't believe I would have. I'm curious about longer samples.
nwallin 7y ago
I went 6/8. However, I call shenanigans. Some of the real clips were him reading from a script for the advertisements at the beginning of his show. Joe has a really flat/unnatural delivery when he's reading from a piece of paper. In addition, if they were including the advertisement reading in the training material, that would definitely mess up the final model. Advertisement Joe doesn't like Joe.
- wallace_f 7y ago
  I got them all correct easily. You can hear the emotion in the real voice. Someone should test this on criminal psychopaths and see if they are significantly better or worse at it than the general pop. I would bet there will be two major populations of psychopath in solidly each category.
  - ekianjo 7y ago
    > I got them all correct easily. You can hear the emotion in the real voice. Yup, it's fairly obvious there is a difference between a somewhat more robotic intonation vs someone who puts emphasis on certain parts of each word. Still, they nailed the sound/uniqueness of his voice pretty well.
  - hn_throwaway_99 7y ago
    Uh oh, I got nearly all of them wrong. I hope I'm not a psychopath :( Somewhat in my defense, although I've seen a few clips with Rogan, I'm not very familiar with him. I agree with the GP's comment, I thought the real Joe Rogan reading ads sounded very staccato and robotic. Near the end I got a little better with the "OK, robotic sounding Joe is the real one." That said, I'm an introvert and I don't enjoy social situations with people I don't know, so while perhaps not a psychopath it's plausible the my poor ability to infer emotional intonation is at play both here and in social situations.
    - ydb 7y ago
      I only missed 1 in 8, and I attribute that to a bit too much wine on a Saturday night. I heard "farm fresh" as "farm fish" and thought I was a genius for catching an algorithm error... However, as I mentioned in this comment[1] I don't think you have much to worry about. The correct criteria for diagnosing fake vs real is actually not being robotic. I knew immediately what was real because there was emphasis and affectation, as opposed to the monotone counterpart. Finally, a minor quib: "psychopathy" is not about an ability to infer intonation (quite the opposite). I'm an introvert like you (Hell, I'm probably considered a hermit) and I was easily able to identify the doppelganger. [1] https://news.ycombinator.com/item?id=19951679 https://news.ycombinator.com/item?id=19951679
remir 7y ago
Impressive stuff, and this is only 2019 tech. Now, imagine this kind of technology in 10, 20 or 30 years. Things will get weird for sure.
- tachyonbeam 7y ago
  We also have GANs that can generate realistic-looking faces now (https://www.youtube.com/watch?v=XOxxPcy5Gr4 https://www.youtube.com/watch?v=XOxxPcy5Gr4). How long before we can generate fake videos fluidly? I would expect that pretty soon, we'll start seeing a lot of fake online/dating profiles with generated pictures and text. Probably a lot of chatbots posing as attractive women trying to scam people. Seems it should also be possible to condition a GAN to generate pictures that look like someone specific. These bots could try to pass themselves off as people you know.
  - kibibu 7y ago
    If you don't need a physical relationship, an intelligent-enough GAN-bot might be appealing for a lot of people.
    - wallace_f 7y ago
      Life is going to be weird if virtual girls/dolls go from fringe otaku culture to realistic, relatively normal household items. People also seem to be wanting to get more drugged up, plastic, and bio-engineered. Maybe in the future the average person is the meeting somewhere in the middle.
      - tachyonbeam 7y ago
        I'm personally somewhat worried about the way modern society enables people to have less and less physical interactions. Imagine a future where it's commonplace to attend an online university, work from home, get goods, food and groceries delivered by a robot (don't even have to talk to a driver). Now, if we had chatbots that passed the Turing test, and always basically told you what you wanted to hear without ever challenging you, then what? I'm worried we're going the way of Japan and its 'herbivore' culture. A world where people have given up on dating because it's too complicated. Other human beings are uncomfortable to interact with, so everyone retreats into their own little bubbles and gets more and more of their needs met by machines... On an individual basis, it a seductive idea to have an idealized robotic partner that does exactly what you want when you want it and doesn't have any needs of its own, will never leave you, but what will that do to humanity as a whole?
__s 7y ago
Identity is a private key. Sign everything you wish attributed to your identity Unfortunately the UI/UX just isn't there yet
mitchtbaum 7y ago
XCKD: PGP https://xkcd.com/1181/ https://xkcd.com/1181/ https://www.explainxkcd.com/wiki/index.php/1181:_PGP https://www.explainxkcd.com/wiki/index.php/1181:_PGP
chubot 7y ago
I went 6 for 8 -- two times it convinced me that Faux Joe Rogan was real.
mrmondo 7y ago
Good job, However - I guess the answer correctly with all but one - “You are much less likely to injure yourself if you do it correctly” - which I can now hear the different (or perhaps confirmation bias). There’s something artificial about the timing with the pauses between sentences on the ML based ones that was the main giveaway for me, also while his voice is damn near spot on - there are some vocal inflections or perhaps “emotion” in a few words that also hinted to me. Hope that helps.
- wisty 7y ago
  I got all but that one correct to. I think the real tell is that Real Rogan has a lot of variance, Faux Rogan sounds too much like the average Rogan.
agrinman 7y ago
By submitting our guesses are we helping them train a fake joe rogan?
- throw20102010 7y ago
  Not likely helping with the training- unless I'm underestimating the scale of people visiting the site. Also unlikely because they had to have a well trained model to launch the site and make the YouTube video that everyone has seen. It's possible that they will do fine tuning in certain areas depending on if some generated clips get consistently answered the same way. More likely is that you're helping them write the results section of their research paper, with a sentence like this, "In a blind trial of 5,000 internet users, over 93% of people were unable to tell the difference between generated audio and real audio at a statistically significant level (P<=0.035)." Note: if we assume a binomial distribution, you need to get 7 or 8 correct to reach the magical P<=.05 barrier. If you assume that there are a fixed 4 generated clips and 4 real clips, then you need to get them all right (https://en.wikipedia.org/wiki/Fisher%27s_exact_test https://en.wikipedia.org/wiki/Fisher%27s_exact_test). I think it's fair to use the binomial distribution because the website does not tell you the number of real and generated clips before you take the test.
- soulofmischief 7y ago
  I don't think so, at first glance it looked all client-side.
daenz 7y ago
So when are we going to get an audiobook service that reads books in your own voice (or a celebrity narrator of your choosing)?
- ilaksh 7y ago
  I feel like this isn't perfect but it's good enough to use for that as an alternative when they can't get a person to do it. Seems like an ideal goal for them might be to have audible.com aquire them for a boatload of money.
- puranjay 7y ago
  Audiobooks in your own voice would be terrible because your voice in your own head is always deeper and richer than your actual voice
  - soulofmischief 7y ago
    Alright, audio books where the cast is my high school circle of friends.
- duncanawoods 7y ago
  It surprised me but one thing Edge is really good at is ebook support with a read-aloud feature. The text rendering is good, better than Calibre, and the cursor tracking following the voice is very satisfying. Some of the British voices are close to perfect. If you want to listen at 2x then it's often better than a human because at speed, there is no value in dramatic intonation, instead impeccably consistent pronunciation and pace are what helps intake which would otherwise be grating at standard speed. What is really nice is that when you start feel "beaten up" by a high pace reading, you can switch voices and brighten up the experience. I find switching back and forth between male and female voices helps avoid ear fatigue and keep up the pace. I immediately become more receptive to the content when a new voice begins.
- SlowRobotAhead 7y ago
  I don’t think I want Idris Elba reading 50 Shades to my wife :/
drenvuk 7y ago
8/8. There's distortion in every single faux rogan recording. The real ones are smooth.
m0zg 7y ago
8 out of 8. Look for intonation and you'll guess correctly every time. Machines don't have it (yet).
larsiusprime 7y ago
I got 8/8. Intonation was an easy give away for me, the fake recordings had obvious tells because they didn't pause naturally or emphasized words in a weird way. EDIT: Still impressive tho. I imagine the typical use case for this sort of tech is not just trying to fool people who have been explicitly forewarned.
rangibaby 7y ago
It’s easy to tell, the computer generated voices have what sounds like MP3 compression artifacts. I noticed the same thing on the bus in Kyoto
- cs02rm0 7y ago
  I thought the same, though the second one along seemed to have dodgy audio too yet that was real.
kyberias 7y ago
Each of the real Rogan samples had some detail that didn't sound like AI could reproduce it. The AI samples were much more monotonous.
WestCoastJustin 7y ago
How can I do this with my voice? Any suggestions? I run a site called https://sysadmincasts.com/ https://sysadmincasts.com/ with tons of voiceovers and would love to automate this (without it sounding like crap). There are always places where I need to update the audio with small tweaks. I imagine it is still pretty computerized today but I think we'll eventually get there. The google WaveNet stuff is pretty good but still not there yet [1]. I imagine eventually, you could have some type of transcript that's annotated with speech synthesis markup language (SSML). Then, you have a CI/CD pipeline that would run this text-to-speech engine and regenerate the audio. I could then pair this up with the video. I honestly wonder if we are a year or two away from this being possible. [1] https://cloud.google.com/text-to-speech/ https://cloud.google.com/text-to-speech/
- soulofmischief 7y ago
  I'd start with recording hundreds of hours of you having typical conversations with a wide variety of context, emotion and intensity.
- blhack 7y ago
  https://lyrebird.ai/ https://lyrebird.ai/
  - CharlesW 7y ago
    Wow, the samples on their front page are absolutely awful. Do they have good ones?
    - james_s_tayler 7y ago
      Yeah, I feel like this has to be a joke compared to the level of quality Faux Rogan is pumping out.
- petrochukm 7y ago
  Hey! > I honestly wonder if we are a year or two away from this being possible. We've launched a service for adding voiceovers 2 months ago... https://wellsaidlabs.com/ https://wellsaidlabs.com/ Techcrunch: https://techcrunch.com/2019/03/07/wellsaid-aims-to-make-natural-sounding-synthetic-speech-a-credible-alternative-to-real-humans/ https://techcrunch.com/2019/03/07/wellsaid-aims-to-make-natu... GeekWire: https://www.geekwire.com/2019/ai2s-incubator-gives-birth-wellsaid-startup-synthesizes-amazingly-realistic-voices/?fbclid=IwAR0uHTuvoJ9xf7VRWUSG4jFVfspY407ppRUzRSwbYFLqh6w_fYe-YJy-CBo https://www.geekwire.com/2019/ai2s-incubator-gives-birth-wel... > How can I do this with my voice? To prevent abuse of our technology, we need to review your use-case before creating you a custom voice. > The google WaveNet stuff is pretty good but still not there yet [1]. Google WaveNet is not built for high-quality voice-overs but rather for cheap and fast text-to-speech.
DavidSJ 7y ago
I got 7/8 and I don't even listen to him.
- markdown 7y ago
  Same here. Only got the "Healthy local ingredients, farm fresh ingredients" one wrong. And I've only ever listed to this person once (the Musk interview).
milleramp 7y ago
I went 5/8, Perhaps it’s because I listen to the podcast on 1.4 speed??
- chrisfinne 7y ago
  Me too. Hearing his voice at 1x seems weird.
zuminator 7y ago
I was able to get them all correct, but only because I knew to listen for an artificial voice. The faux Rogans clipped their consonants in a just-barely inauthentic manner, and real Joe's voice slooows down mid-phrase for dramatic emphasis. By the way I am not a follower of his, and was initially surprised as I was expecting the voice of actor Seth Rogen.
- ryanjshaw 7y ago
  Same, 8/8, voted after listening for at most 1 second to each clip and never having listened to the guy before. The clipping makes the rate of speech sound jarringly unnatural to me.
aembleton 7y ago
I got them all correct! It felt like the cadence of the Faux Rogan was just 'off' and I could tell from that. Everything else seemed spot on.
55555 7y ago
> "It's important that we listen to what others have to say about this work." Who are you, my father? Am I the only one who finds this trend really cringey?
Hoasi 7y ago
This is really well done. I could tell which was Faux Rogan by listening to the pacing of the sentences, not so much to voice itself.
j7ake 7y ago
It'd be cool if the algorithm is being tweaked dynamically and it is progressively getting harder and harder for people to get 8/8.
- tomlu 7y ago
  This is probably the hardest setting, yet most people seemed to have no trouble getting 8/8.
kapilkaisare 7y ago
I felt the Faux Rogan spoke just a mite quicker than the Real Rogan.
- is0metry 7y ago
  Agreed. That was how I was able to tell.
rahimnathwani 7y ago
This repo has an implementation of a similar model, and a very clear set of usage instructions: https://github.com/syang1993/gst-tacotron https://github.com/syang1993/gst-tacotron I haven't tried it yet, but if you're looking to do something similar, this repo (and the papers on which the implementation is based) might be a good starting point.
scanr 7y ago
This suggests that we are pretty close to not being able to rely on voice biometrics. Does anyone in the field know more?
rtkwe 7y ago
I actually had an oddly easy time of it... but not because of any audio differences all the Real Rogan's were talking about food and the rest were talking about something else.
sebringj 7y ago
there were slight audio anomalies in the fake ones, easy to spot.
peterwwillis 7y ago
You could do a pretty good update of 1984 with this tech. A person goes to a rally where the President gives a speech decrying the evil of global warming, and then later online all the reviews have clips of him saying global warming doesn't exist, and the viewer just changes their mind to believe what they saw in person wasn't real. An occasional hacker-led SEO attack will push up the real video from the original speech, but because it's so rarely published, everyone just assumes it's a fake designed as propaganda. Virtually everything we fear could be turned into something good for us, and media generated by AI becomes an engine that transforms people's thoughts. A bug in the AI software accidentally makes everyone believe we should all destroy our computers to prevent getting some virus, only later to discover that the computers were distorting our thoughts and that nothing was as it seemed.
pippin 7y ago
Hi HN, I'm one of the creators from Dessa of this project. If you haven't listened to it, we just released a longer clip of the RealTalk model[1]. In my opinion it's even more compelling than these clips. One of fascinating parts of building this has been the questions we received while showing it to people. I'll note a few anecdotes specifically: "What is the difference between this and a real voice?" "Can I learn to discern fakes over time?" "Would we relate differently to a generative voice model posthumously, compared to current media forms like videos?" These aren't questions that we necessarily have answers to yet, but they're important discussions to have. [1] https://www.youtube.com/watch?v=DWK_iYBl8cA&feature=youtu.be https://www.youtube.com/watch?v=DWK_iYBl8cA&feature=youtu.be
- failrate 7y ago
  All of these generated works are very impressive. I think it is completely irresponsible to advance the state of the art in this field without simultaneously developing techniques to demonstrate that the generated work is artificial. Please develop validation tests while developing your generative techniques.
  - gridlockd 7y ago
    Haven't read what they're doing, but chances are they are using an adversial neural network. The job of the adversarial network is to tell apart real from fake. The job of the neural network is to fool the adverserial network. Both are trained in tandem. One could imagine training another adverserial network that isn't used to train the network itself, and so will pick up on nuances that the original adversary doesn't pick up on. Anyone could do that, I don't think it's the author's responsibility. Somewhat related: https://keenlab.tencent.com/en/2019/03/29/Tencent-Keen-Security-Lab-Experimental-Security-Research-of-Tesla-Autopilot/ https://keenlab.tencent.com/en/2019/03/29/Tencent-Keen-Secur...
    - petrochukm 7y ago
      Doubt it. Generative-adversarial models have had a lot of success in image generation; however, the same cannot be said for speech synthesis. Unless they have figured out a new technique, they are probably using Tacotron 2 (https://ai.googleblog.com/2017/12/tacotron-2-generating-human-like-speech.html https://ai.googleblog.com/2017/12/tacotron-2-generating-huma...). Google's Tacotron 2 already achieved human-parity TTS without adversarial training as measured by MOS.
djohnston 7y ago
8/8. you can detect the faux rogan because there are slight cracks in the audio, almost like a record pop but much more subtle
petrochukm 7y ago
For context, it's important to know that these are probably cherry picked samples. The authors make no mention of attempting randomly select these samples. For as long as text-to-speech has existed, there have been impressive demos backed by cherry picking. The 3 Dessa team members did not in 3 months of work create anything innovative probably. Rayhane Mamah, one of the Dessa team members, had previously published a Tacotron 2 (Google's 2017 research) implementation (https://github.com/Rayhane-mamah/Tacotron-2 https://github.com/Rayhane-mamah/Tacotron-2) that has similar noise/distortion and intonation/prosody issues as their "RealTalk model". Following on the above, Google's TTS research already demonstrated human-parity as measured by MOS score in early 2018. That research was deployed as Google Duplex in mid 2018. Google's TTS research also showed the deficiencies of this technology. Without the invention of AGI, the TTS models do not understand the underlying text; therefore, it'll be unable to do more "complex things with intonation/prosody". Furthermore, the models suffer from overfitting. The model performance degrades significantly when performing TTS on text not typically seen in the training data.
smolder 7y ago
I got them all right. There are some noticeable artifacts in each of the generated clips that give them away. If it weren't for that, they sound pretty natural, so it would be difficult.
Fnoord 7y ago
Not sure, but I suppose this person [1] is the Real Rogan. (I didn't know this person so maybe it helps someone.) [1] https://en.wikipedia.org/wiki/Joe_Rogan https://en.wikipedia.org/wiki/Joe_Rogan
chriselles 7y ago
I have listened to about 10 Joe Rogan podcasts. I only got a 4 out of 8. I’m assuming the longer the audio clips the easier it would be to detect the fake/AI?

220 points by mgdo 7y ago

laughably easy to spot the fake. uncanny valley out the wazoooooo

Solid work they both sound exactly the same to me. The Faux Rogan is a slightly faster speaker, so it was pretty easy to differentiate between the two.

dogma1138 7y ago

Got all right it seems that the faux Rogan talks a bit too fast but anyone who doesn’t listen to his podcast regularly likely won’t notice it. I wonder how long it would take until Joe brings those guys to his podcast that is something I’ll be waiting for.

siedes 7y ago

It was easy to figure out which are the fakes, but I am very impressed and spooked at the idea of this technology improving, then getting into the wrong hands.

sho_hn 7y ago
I'm not too spooked. I think as a society we're just going to learn to distrust recorded+reproduced media by default. Already we don't trust photos much for their potential to be a manipulation. You could call this a loss, sure - but maybe we should just never have in the first place. Photo manipulation is a very old practice. There's already businesses that sell camera apps (or cameras? my memory is a bit dim) that save a photo along with a cryptographic hash to prove authenticity. Their customers are for example insurance companies, which require their clients to take pictures of damaged property etc. for claim filings that way.
- blotter_paper 7y ago
  Yup. Media will be signed cryptographically and your belief in the media will correspond to your trust in the signer. It will destroy the usefulness and damage of anonymous publication of recorded evidence. Bad for transparency, good for privacy.
  - YaxelPerez 7y ago
    We can't even get people to look for a green lock icon in their URL bar. I really doubt the kinds of people that are most vulnerable to these attacks will suddenly develop an interest in cryptography. And if the fake confirms someone's worldview or serves their interests, confirmation bias will take over and no "fancy math" is going to change their mind.
    - blotter_paper 7y ago
      I don't think we'll adapt "suddenly," I expect we'll have to experience some major news stories getting redacted due to deepfaked evidence before this reaction becomes mainstream, but I think it's coming. I also don't think an interest in cryptography is necessary, it's not like I expect people to read public keys directly. There will be some system that shows something along the lines of "Confirmed by $USER_NAME" under the media. You don't need an interest in video codecs to appreciate YouTube. And I have seen people reject faked evidence, such as fake screencaps of tweets, even when it backed up their worldview. We're not perfect at it, but I think we're getting better.
- cco 7y ago
  Do you not see the problem in societies being unable to share trusted and accurate information about reality? I mean that is 100% required for any interpersonal relationship between two people let alone a community.
  - deleted 7y ago
    [deleted]
  - 7y ago
arebours 7y ago
> It was easy to figure out which are the fakes Yeah but would you figure it out if you just listened to it expecting Joe Rogan?
- siedes 7y ago
  It would be slightly harder. What I'm looking out for here is voice distortion and the unnatural flow of speech, and it's present in every Faux recording. It's just unnatural compared to how JR normally sounds.

craftinator 7y ago

I didn't do so well. I was 60% correct. The non-word noises were really good with FR, pauses and breaths did a great job of convincing me. The one thing that I picked up on quickly though, was that FR speech was a bit more slurry and fast than JR speech. Interesting...

sho_hn 7y ago

My reliable tells for the Faux Rogan were a certain amount of noise/distortion to the audio, and the real Rogan doing more complex things with intonation/prosody. The latter requires a direct comparison, and the former must be fixable with filtering? That makes it pretty close to good enough for an original reading.

codetrotter 7y ago
Same here. Have only watched one or two episodes of his podcast in the past, so was basing my judgement on what sounded natural in general rather than any specific recollection of what he is supposed to sound like. I voted on each clip without listening to other clips first. I misidentified two that were real as being fake. Wonder if those might have been clips where he was reading from a script with a flatter voice than usual like another top-level comment ITT talks about. If audio of this quality was being presented in a context where I didn’t know it might be fake, I probably would have chalked it up to issues with the recording mainly, and would have believed that the audio was real.
- sho_hn 7y ago
  An interesting question (ignoring how you need Real Rogan to train Fake one for a moment) is whether the Fake Rogan could be as popular, or if the extra personality the real voice has is vital to making him a popular speaker. I suspect that question strikes to things like gaining the tools to transpose speaker affect independent from voice characteristics etc. Imagine a remix culture where a future popular podcast voice is a mashup of different older popular speakers and things like that ...
  - soulofmischief 7y ago
    Reminds me of Aldous Huxley's Brave New World when the characters attend a synthetic orchestra. The book goes into vivid detail about the level of craftsmanship achievable by a machine composer and orchestra, yet somehow it all seems so shallow, like the soul of it has long gone.
  - 7y ago

overgard 7y ago

Gotta admit, I listen to his podcasts sometimes and I only hit 50%.

starpilot 7y ago

Got 1 wrong, and I've never listened to Rogan. Seemed the fake one talked too monotone.

axaxs 7y ago
I noticed that, and also that the spacing in words was too uniform.
cm2187 7y ago
What I am curious about is how much selection was involved in the fake ones. Like is it the cream of the top of the fake ones that sound the most natural or is it a typical output?
daenz 7y ago
The "warbling" factor stood out to me. It's like you could hear the interpolation between audio samples.
- mrmondo 7y ago
  Yeah, there was some strange timing between sentences and words at times.

arebours 7y ago

Damn it's good. I was able to spot most of the fakes but hadn't I known there are any I don't believe I would have. I'm curious about longer samples.

nwallin 7y ago

I went 6/8. However, I call shenanigans. Some of the real clips were him reading from a script for the advertisements at the beginning of his show. Joe has a really flat/unnatural delivery when he's reading from a piece of paper. In addition, if they were including the advertisement reading in the training material, that would definitely mess up the final model. Advertisement Joe doesn't like Joe.

wallace_f 7y ago
I got them all correct easily. You can hear the emotion in the real voice. Someone should test this on criminal psychopaths and see if they are significantly better or worse at it than the general pop. I would bet there will be two major populations of psychopath in solidly each category.
- ekianjo 7y ago
  > I got them all correct easily. You can hear the emotion in the real voice. Yup, it's fairly obvious there is a difference between a somewhat more robotic intonation vs someone who puts emphasis on certain parts of each word. Still, they nailed the sound/uniqueness of his voice pretty well.
- hn_throwaway_99 7y ago
  Uh oh, I got nearly all of them wrong. I hope I'm not a psychopath :( Somewhat in my defense, although I've seen a few clips with Rogan, I'm not very familiar with him. I agree with the GP's comment, I thought the real Joe Rogan reading ads sounded very staccato and robotic. Near the end I got a little better with the "OK, robotic sounding Joe is the real one." That said, I'm an introvert and I don't enjoy social situations with people I don't know, so while perhaps not a psychopath it's plausible the my poor ability to infer emotional intonation is at play both here and in social situations.
  - ydb 7y ago
    I only missed 1 in 8, and I attribute that to a bit too much wine on a Saturday night. I heard "farm fresh" as "farm fish" and thought I was a genius for catching an algorithm error... However, as I mentioned in this comment[1] I don't think you have much to worry about. The correct criteria for diagnosing fake vs real is actually not being robotic. I knew immediately what was real because there was emphasis and affectation, as opposed to the monotone counterpart. Finally, a minor quib: "psychopathy" is not about an ability to infer intonation (quite the opposite). I'm an introvert like you (Hell, I'm probably considered a hermit) and I was easily able to identify the doppelganger. [1] https://news.ycombinator.com/item?id=19951679 https://news.ycombinator.com/item?id=19951679

remir 7y ago

Impressive stuff, and this is only 2019 tech. Now, imagine this kind of technology in 10, 20 or 30 years. Things will get weird for sure.

tachyonbeam 7y ago
We also have GANs that can generate realistic-looking faces now (https://www.youtube.com/watch?v=XOxxPcy5Gr4 https://www.youtube.com/watch?v=XOxxPcy5Gr4). How long before we can generate fake videos fluidly? I would expect that pretty soon, we'll start seeing a lot of fake online/dating profiles with generated pictures and text. Probably a lot of chatbots posing as attractive women trying to scam people. Seems it should also be possible to condition a GAN to generate pictures that look like someone specific. These bots could try to pass themselves off as people you know.
- kibibu 7y ago
  If you don't need a physical relationship, an intelligent-enough GAN-bot might be appealing for a lot of people.
  - wallace_f 7y ago
    Life is going to be weird if virtual girls/dolls go from fringe otaku culture to realistic, relatively normal household items. People also seem to be wanting to get more drugged up, plastic, and bio-engineered. Maybe in the future the average person is the meeting somewhere in the middle.
    - tachyonbeam 7y ago
      I'm personally somewhat worried about the way modern society enables people to have less and less physical interactions. Imagine a future where it's commonplace to attend an online university, work from home, get goods, food and groceries delivered by a robot (don't even have to talk to a driver). Now, if we had chatbots that passed the Turing test, and always basically told you what you wanted to hear without ever challenging you, then what? I'm worried we're going the way of Japan and its 'herbivore' culture. A world where people have given up on dating because it's too complicated. Other human beings are uncomfortable to interact with, so everyone retreats into their own little bubbles and gets more and more of their needs met by machines... On an individual basis, it a seductive idea to have an idealized robotic partner that does exactly what you want when you want it and doesn't have any needs of its own, will never leave you, but what will that do to humanity as a whole?

__s 7y ago

Identity is a private key. Sign everything you wish attributed to your identity Unfortunately the UI/UX just isn't there yet

mitchtbaum 7y ago

XCKD: PGP https://xkcd.com/1181/ https://xkcd.com/1181/ https://www.explainxkcd.com/wiki/index.php/1181:_PGP https://www.explainxkcd.com/wiki/index.php/1181:_PGP

chubot 7y ago

I went 6 for 8 -- two times it convinced me that Faux Joe Rogan was real.

mrmondo 7y ago

Good job, However - I guess the answer correctly with all but one - “You are much less likely to injure yourself if you do it correctly” - which I can now hear the different (or perhaps confirmation bias). There’s something artificial about the timing with the pauses between sentences on the ML based ones that was the main giveaway for me, also while his voice is damn near spot on - there are some vocal inflections or perhaps “emotion” in a few words that also hinted to me. Hope that helps.

wisty 7y ago
I got all but that one correct to. I think the real tell is that Real Rogan has a lot of variance, Faux Rogan sounds too much like the average Rogan.

agrinman 7y ago

By submitting our guesses are we helping them train a fake joe rogan?

throw20102010 7y ago
Not likely helping with the training- unless I'm underestimating the scale of people visiting the site. Also unlikely because they had to have a well trained model to launch the site and make the YouTube video that everyone has seen. It's possible that they will do fine tuning in certain areas depending on if some generated clips get consistently answered the same way. More likely is that you're helping them write the results section of their research paper, with a sentence like this, "In a blind trial of 5,000 internet users, over 93% of people were unable to tell the difference between generated audio and real audio at a statistically significant level (P<=0.035)." Note: if we assume a binomial distribution, you need to get 7 or 8 correct to reach the magical P<=.05 barrier. If you assume that there are a fixed 4 generated clips and 4 real clips, then you need to get them all right (https://en.wikipedia.org/wiki/Fisher%27s_exact_test https://en.wikipedia.org/wiki/Fisher%27s_exact_test). I think it's fair to use the binomial distribution because the website does not tell you the number of real and generated clips before you take the test.
soulofmischief 7y ago
I don't think so, at first glance it looked all client-side.

daenz 7y ago

So when are we going to get an audiobook service that reads books in your own voice (or a celebrity narrator of your choosing)?

ilaksh 7y ago
I feel like this isn't perfect but it's good enough to use for that as an alternative when they can't get a person to do it. Seems like an ideal goal for them might be to have audible.com aquire them for a boatload of money.
puranjay 7y ago
Audiobooks in your own voice would be terrible because your voice in your own head is always deeper and richer than your actual voice
- soulofmischief 7y ago
  Alright, audio books where the cast is my high school circle of friends.
duncanawoods 7y ago
It surprised me but one thing Edge is really good at is ebook support with a read-aloud feature. The text rendering is good, better than Calibre, and the cursor tracking following the voice is very satisfying. Some of the British voices are close to perfect. If you want to listen at 2x then it's often better than a human because at speed, there is no value in dramatic intonation, instead impeccably consistent pronunciation and pace are what helps intake which would otherwise be grating at standard speed. What is really nice is that when you start feel "beaten up" by a high pace reading, you can switch voices and brighten up the experience. I find switching back and forth between male and female voices helps avoid ear fatigue and keep up the pace. I immediately become more receptive to the content when a new voice begins.
SlowRobotAhead 7y ago
I don’t think I want Idris Elba reading 50 Shades to my wife :/

drenvuk 7y ago

8/8. There's distortion in every single faux rogan recording. The real ones are smooth.

m0zg 7y ago

8 out of 8. Look for intonation and you'll guess correctly every time. Machines don't have it (yet).

larsiusprime 7y ago

I got 8/8. Intonation was an easy give away for me, the fake recordings had obvious tells because they didn't pause naturally or emphasized words in a weird way. EDIT: Still impressive tho. I imagine the typical use case for this sort of tech is not just trying to fool people who have been explicitly forewarned.

rangibaby 7y ago

It’s easy to tell, the computer generated voices have what sounds like MP3 compression artifacts. I noticed the same thing on the bus in Kyoto

cs02rm0 7y ago
I thought the same, though the second one along seemed to have dodgy audio too yet that was real.

kyberias 7y ago

Each of the real Rogan samples had some detail that didn't sound like AI could reproduce it. The AI samples were much more monotonous.

WestCoastJustin 7y ago

How can I do this with my voice? Any suggestions? I run a site called https://sysadmincasts.com/ https://sysadmincasts.com/ with tons of voiceovers and would love to automate this (without it sounding like crap). There are always places where I need to update the audio with small tweaks. I imagine it is still pretty computerized today but I think we'll eventually get there. The google WaveNet stuff is pretty good but still not there yet [1]. I imagine eventually, you could have some type of transcript that's annotated with speech synthesis markup language (SSML). Then, you have a CI/CD pipeline that would run this text-to-speech engine and regenerate the audio. I could then pair this up with the video. I honestly wonder if we are a year or two away from this being possible. [1] https://cloud.google.com/text-to-speech/ https://cloud.google.com/text-to-speech/

soulofmischief 7y ago
I'd start with recording hundreds of hours of you having typical conversations with a wide variety of context, emotion and intensity.
blhack 7y ago
https://lyrebird.ai/ https://lyrebird.ai/
- CharlesW 7y ago
  Wow, the samples on their front page are absolutely awful. Do they have good ones?
  - james_s_tayler 7y ago
    Yeah, I feel like this has to be a joke compared to the level of quality Faux Rogan is pumping out.
petrochukm 7y ago
Hey! > I honestly wonder if we are a year or two away from this being possible. We've launched a service for adding voiceovers 2 months ago... https://wellsaidlabs.com/ https://wellsaidlabs.com/ Techcrunch: https://techcrunch.com/2019/03/07/wellsaid-aims-to-make-natural-sounding-synthetic-speech-a-credible-alternative-to-real-humans/ https://techcrunch.com/2019/03/07/wellsaid-aims-to-make-natu... GeekWire: https://www.geekwire.com/2019/ai2s-incubator-gives-birth-wellsaid-startup-synthesizes-amazingly-realistic-voices/?fbclid=IwAR0uHTuvoJ9xf7VRWUSG4jFVfspY407ppRUzRSwbYFLqh6w_fYe-YJy-CBo https://www.geekwire.com/2019/ai2s-incubator-gives-birth-wel... > How can I do this with my voice? To prevent abuse of our technology, we need to review your use-case before creating you a custom voice. > The google WaveNet stuff is pretty good but still not there yet [1]. Google WaveNet is not built for high-quality voice-overs but rather for cheap and fast text-to-speech.

DavidSJ 7y ago

I got 7/8 and I don't even listen to him.

markdown 7y ago
Same here. Only got the "Healthy local ingredients, farm fresh ingredients" one wrong. And I've only ever listed to this person once (the Musk interview).

milleramp 7y ago

I went 5/8, Perhaps it’s because I listen to the podcast on 1.4 speed??

chrisfinne 7y ago
Me too. Hearing his voice at 1x seems weird.

zuminator 7y ago

I was able to get them all correct, but only because I knew to listen for an artificial voice. The faux Rogans clipped their consonants in a just-barely inauthentic manner, and real Joe's voice slooows down mid-phrase for dramatic emphasis. By the way I am not a follower of his, and was initially surprised as I was expecting the voice of actor Seth Rogen.

ryanjshaw 7y ago
Same, 8/8, voted after listening for at most 1 second to each clip and never having listened to the guy before. The clipping makes the rate of speech sound jarringly unnatural to me.

aembleton 7y ago

I got them all correct! It felt like the cadence of the Faux Rogan was just 'off' and I could tell from that. Everything else seemed spot on.

55555 7y ago

> "It's important that we listen to what others have to say about this work." Who are you, my father? Am I the only one who finds this trend really cringey?

Hoasi 7y ago

This is really well done. I could tell which was Faux Rogan by listening to the pacing of the sentences, not so much to voice itself.

j7ake 7y ago

It'd be cool if the algorithm is being tweaked dynamically and it is progressively getting harder and harder for people to get 8/8.

tomlu 7y ago
This is probably the hardest setting, yet most people seemed to have no trouble getting 8/8.

kapilkaisare 7y ago

I felt the Faux Rogan spoke just a mite quicker than the Real Rogan.

is0metry 7y ago
Agreed. That was how I was able to tell.

rahimnathwani 7y ago

This repo has an implementation of a similar model, and a very clear set of usage instructions: https://github.com/syang1993/gst-tacotron https://github.com/syang1993/gst-tacotron I haven't tried it yet, but if you're looking to do something similar, this repo (and the papers on which the implementation is based) might be a good starting point.

scanr 7y ago

This suggests that we are pretty close to not being able to rely on voice biometrics. Does anyone in the field know more?

rtkwe 7y ago

I actually had an oddly easy time of it... but not because of any audio differences all the Real Rogan's were talking about food and the rest were talking about something else.

sebringj 7y ago

there were slight audio anomalies in the fake ones, easy to spot.

peterwwillis 7y ago

You could do a pretty good update of 1984 with this tech. A person goes to a rally where the President gives a speech decrying the evil of global warming, and then later online all the reviews have clips of him saying global warming doesn't exist, and the viewer just changes their mind to believe what they saw in person wasn't real. An occasional hacker-led SEO attack will push up the real video from the original speech, but because it's so rarely published, everyone just assumes it's a fake designed as propaganda. Virtually everything we fear could be turned into something good for us, and media generated by AI becomes an engine that transforms people's thoughts. A bug in the AI software accidentally makes everyone believe we should all destroy our computers to prevent getting some virus, only later to discover that the computers were distorting our thoughts and that nothing was as it seemed.

pippin 7y ago

Hi HN, I'm one of the creators from Dessa of this project. If you haven't listened to it, we just released a longer clip of the RealTalk model[1]. In my opinion it's even more compelling than these clips. One of fascinating parts of building this has been the questions we received while showing it to people. I'll note a few anecdotes specifically: "What is the difference between this and a real voice?" "Can I learn to discern fakes over time?" "Would we relate differently to a generative voice model posthumously, compared to current media forms like videos?" These aren't questions that we necessarily have answers to yet, but they're important discussions to have. [1] https://www.youtube.com/watch?v=DWK_iYBl8cA&feature=youtu.be https://www.youtube.com/watch?v=DWK_iYBl8cA&feature=youtu.be

failrate 7y ago
All of these generated works are very impressive. I think it is completely irresponsible to advance the state of the art in this field without simultaneously developing techniques to demonstrate that the generated work is artificial. Please develop validation tests while developing your generative techniques.
- gridlockd 7y ago
  Haven't read what they're doing, but chances are they are using an adversial neural network. The job of the adversarial network is to tell apart real from fake. The job of the neural network is to fool the adverserial network. Both are trained in tandem. One could imagine training another adverserial network that isn't used to train the network itself, and so will pick up on nuances that the original adversary doesn't pick up on. Anyone could do that, I don't think it's the author's responsibility. Somewhat related: https://keenlab.tencent.com/en/2019/03/29/Tencent-Keen-Security-Lab-Experimental-Security-Research-of-Tesla-Autopilot/ https://keenlab.tencent.com/en/2019/03/29/Tencent-Keen-Secur...
  - petrochukm 7y ago
    Doubt it. Generative-adversarial models have had a lot of success in image generation; however, the same cannot be said for speech synthesis. Unless they have figured out a new technique, they are probably using Tacotron 2 (https://ai.googleblog.com/2017/12/tacotron-2-generating-human-like-speech.html https://ai.googleblog.com/2017/12/tacotron-2-generating-huma...). Google's Tacotron 2 already achieved human-parity TTS without adversarial training as measured by MOS.

djohnston 7y ago

8/8. you can detect the faux rogan because there are slight cracks in the audio, almost like a record pop but much more subtle

petrochukm 7y ago

For context, it's important to know that these are probably cherry picked samples. The authors make no mention of attempting randomly select these samples. For as long as text-to-speech has existed, there have been impressive demos backed by cherry picking. The 3 Dessa team members did not in 3 months of work create anything innovative probably. Rayhane Mamah, one of the Dessa team members, had previously published a Tacotron 2 (Google's 2017 research) implementation (https://github.com/Rayhane-mamah/Tacotron-2 https://github.com/Rayhane-mamah/Tacotron-2) that has similar noise/distortion and intonation/prosody issues as their "RealTalk model". Following on the above, Google's TTS research already demonstrated human-parity as measured by MOS score in early 2018. That research was deployed as Google Duplex in mid 2018. Google's TTS research also showed the deficiencies of this technology. Without the invention of AGI, the TTS models do not understand the underlying text; therefore, it'll be unable to do more "complex things with intonation/prosody". Furthermore, the models suffer from overfitting. The model performance degrades significantly when performing TTS on text not typically seen in the training data.

smolder 7y ago

I got them all right. There are some noticeable artifacts in each of the generated clips that give them away. If it weren't for that, they sound pretty natural, so it would be difficult.

Fnoord 7y ago

Not sure, but I suppose this person [1] is the Real Rogan. (I didn't know this person so maybe it helps someone.) [1] https://en.wikipedia.org/wiki/Joe_Rogan https://en.wikipedia.org/wiki/Joe_Rogan

chriselles 7y ago

I have listened to about 10 Joe Rogan podcasts. I only got a 4 out of 8. I’m assuming the longer the audio clips the easier it would be to detect the fake/AI?