12 ms·

AI that produces sound through analysis of a source video is impressive. Fooling humans is not. Since most of us have grown up on a steady diet of film and tele

• kdeldycke 10y ago
  The Turing test in this case might consist of feeding the algorithm with a mute video of a scene from Monthy Python and the Holy Grail, when coconut halves are used to simulate the sound of galloping horses.
• cleeus 10y ago
  Yes, and often sounds from huge sound libraries are used. I allways smile when I hear a sound from Unreal Tournament 4.36 in a movie or on TV.
  • Karunamon 10y ago
    Or Doom.. I remember very clearly the intro to Modern Marvels on the History Channel uses the Doom door opening noise. I guess they used the same SFX library or something?
    • bitwize 10y ago
      The library used by Doom is REALLY common. The sound DSBOSPIT (sound of boss demon spitting a telecube in Doom 2) in particular is so overused it's not even funny. You hear it everywhere: in budget movies when a house or plane explodes, sometimes in other video games.
  • zodPod 10y ago
    I hear the Protoss building creation sound used a lot as well!
• mfoy_ 10y ago
  Everyone knows that if you fall while dying you make a Wilhelm scream. It's a clearly documented scientific phenomenon with earliest records dating back to long, long ago.
• pescolly 10y ago
  From what I've observed the sound of horse hooves is made by taking two halves of a coconut and banging them together.
  • sporkologist 10y ago
    I heard that was done in the Monty Python movie because of needing to save money by not getting a real horse. So that kinda makes it even funnier.
• bravura 10y ago
  Just an aside, but I really like the videos that Hiss and Roar makes to accompany their sound libraries: For example, "Vegetable Violence": http://hissandaroar.com/sd001-vegetable-violence/ http://hissandaroar.com/sd001-vegetable-violence/ "Vegetable Violence is an organic sound effects library for creating your own orchestrated sonic mayhem. Vegetable rips, tears, squelches, hits, punches, stabs all recorded & mastered at 96kHz for stomach churning realism, this component library of gore sound effects is available for immediate download."
  • dogecoinbase 10y ago
    Aside to the aside -- if you like this and also like Italian horror films, you should check out the movie "Berberian Sound Studio".
    • daturkel 10y ago
      What about the reverse? I saw BSS but feel like I don't really know the world it refers to.
      • bazzargh 10y ago
        Watch Dario Argento's films - Profundo Rosso, Suspiria, Inferno, Tenebrae. Those should all be quite easy to get hold of. Then start in on this list... https://en.wikipedia.org/wiki/Giallo https://en.wikipedia.org/wiki/Giallo Suspiria in particular sticks in the mind. Great music, saturated colours, properly horrific horror. It's a bit more 'on-screen' than BSS, btw.
• stepvhen 10y ago
  The sound of the hamburger rain in "Cloudy with a Chance of Meatballs" was wet brown paper towels (you know, from school?) being flopped against a floor.
• IshKebab 10y ago
  Fooling humans is fairly impressive - they didn't just ask "is this real sound", they played the real sound and the synthesised sound and ask "which one is the real sound". Although it looks like they only used a sample of 3 people which is pretty small, I imagine the parametrically synthesised sound would have fooled no-one.
  • esnard 10y ago
    I would love to try this test. I guess I would be tricked too, but that's definitely something I want to try.
• lisper 10y ago
  I don't see what that matters. Just because the human baseline comes from what is essentially a virtual reality instead of actual reality it's still quite a challenge to generalize sounds from associated images. Just because the AI is an artificial foley artist rather than a model of the real world doesn't make it any less impressive to me.
  • Aelinsaar 10y ago
    More importantly, what are all of the foley artists of the world thinking right now? "Oh dear god no" would be my first guess. Suddenly the prospect of needing to beat a computer at your job is rearing its head.
    • tuewocnc 10y ago
      Good! I can't abide foley. Nature documentaries are ruined by it. A tiny ant eating a leaf, accompanied by horrible sounds of plastic-wrap being twisted. Why not record the actual sounds of an ant eating ? It's supposed to be a documentary. And if the ant doesn't make any sound then just leave some silence.
      • wtetzner 10y ago
        This wouldn't remove foley, it would just replace the humans doing it with computers.
        • tuewocnc 10y ago
          but what sound is a DNN going to associate with an insect eating ?
          • Aelinsaar 10y ago
            The answer is definitely: How did you train it? If you trained it by showing it tons of movies and television, then you'd get a modern foley artist in a box, right? If you want them to do something else, you'd need to get a bunch of stock footage with stock audio. Both are doable, but which was done here?
• drzaiusapelord 10y ago
  All of your examples sound a lot like what these things sound in real life. Yes, there's hack foley work but the reality is that these aren't arbitrary sounds. You don't need an actual horse to get horse sounds. I think you're shooting for some idealized authenticity argument here that just doesn't work. I work in a tourist area where horses walk on cobblestone. Yeah it sounds exactly like what the foley guys do with coconuts. Its uncanny. Also in the digital age, a lot of foley work are samples of real sounds. We don't have guys in sound booths making new sounds anymore with gadgets and old shoes and such, outside of edge cases. I know everyone likes to feel clever when they identify a popular sound, but most of the times those are intentional homages and you have to consider that millions of sounds you don't recognize. Its not all from some static library of 1930s foley artists punching meat and knocking together coconuts anymore.
• hellofunk 10y ago
  And not just our perceptions of sounds are skewed from a lifetime of manipulation, but so too our perceptions of the images themselves.
• fudged71 10y ago
  Or the sound of a Bald Eagle ;) http://www.npr.org/templates/story/story.php?storyId=156187375 http://www.npr.org/templates/story/story.php?storyId=1561873...
  • pluma 10y ago
    Somehow this factoid is one of the most American things I can imagine. It's almost allegorical.
• stcredzero 10y ago
  AI that produces sound through analysis of a source video is impressive. Fooling humans is not. Since most of us have grown up on a steady diet of film and television many of the sounds we have in our memories are the work of foley artists that add sound effects to sequences in post. Right on! The fact that most audiences seem to expect a shotgun racking sound in a scene with a shotgun that doesn't even have that mechanism, or that drawing a katana is so often accompanied by a metallic "shing" and rattling sound -- these indicate the degree to which large swathes of people are drastically disconnected from an immediate and physically connected sense of how sound relates to the world around them. I think this is also related to the degree to which I find many people are unaware of the kinesthetic feeling of how beats are emphasized, and how this changes the feel of music. The most vibrant intelligence involves a connection to the world in realtime. You can hear how machine parts interrelate just as much as you can see them. (You can even smell how they interrelate!) This disconnection even seems to be directly correlated with a loss of self awareness and flexibility in problem solving. It's like we're raising generations of brains over-trained on the simplistic and highly abstracted world of media tropes and vastly under-prepared for the messy complexity of the natural physical world.
  • andyjdavis 10y ago
    >or that drawing a katana is so often accompanied by a metallic "shing" Relevant demonstration and explanation of why the "shing" noise isn't a thing outside of movies. https://www.youtube.com/watch?v=yzbfuI0PMdA https://www.youtube.com/watch?v=yzbfuI0PMdA
    • JorgeGT 10y ago
      Condensed for people who can't view the video: when you want to stab someone, it is better to not alert them beforehand with a characteristic sound.
  • ludamad 10y ago
    I'd suspect that there is a wide range of what we're willing to accept in a given situation, reflecting our incomplete model of how sound works. However, this isn't the same as us accepting any substitute sound, the TV tropes continue because their absence feels awkward. An AI that correctly mimicks TV-acceptable sounds is just about as impressive (however, this isn't on our list of 'hardest problems', for sure).
• anigbrowl 10y ago
  You're right, but as someone who's done a lot of sound editing/foley work I can't help having mixed feelings at seeing yet another job skill automated away. Good part - in a few years this will be good enough for commercial use which will save sound editors all sorts of tedious dull work and free them up to do more exciting creative stuff. Bad part: the tedious dull work was also what paid the bills. The easier it is to do that stuff automatically, the less people are willing to pay for good quality work. Rather than now being able to make a living do the sort of fun really creative stuff like inventing new sounds for teleportation devices or dramatic natural phenomena, editors are more likely to be asked to work for free on the theory that they'll gain great exposure for their creativity. That's generally a very bad bargain. If past trends in the electronic dance music market are anything to go by, increasing automation will not reward true creative talent but rather just lead to an arms race to have the latest sound libraries, synthesizers etc. and just be the first to market with big splashy new sounds that offer superficial novelty. Ability to provide high-value equipment below normal rental cost frequently trumps considerations of talent in the film industry. Similarly there are plenty of crappy directors of photography out there who get hired regularly because they own a pile of nice lenses and related camera equipment, and hiring them plus their camera package looks economically attractive on paper because it's hard to quantify photographic talent.
  • slr555 10y ago
    I have so much respect for foley artists. Artist being the operative word. People don't appreciate the hard work and creativity that goes into making the perfect sound.
  • nashashmi 10y ago
    Great Line: the tedious dull work was also what paid the bills. The easier it is to do that stuff automatically, the less people are willing to pay for good quality work.
  • dahart 10y ago
    As someone who's worked in the film industry (visual fx & CG) I'm subject to the same problem, but all tedious jobs from the industrial revolution on have been automated away one by one. I can understand the lament for something you worked hard on, and this isn't to take that away from you, but most job skills do actually have less value in the market over time, right? The other way to look at it is that what counts as good quality work changes and improves continually over time, higher and higher quality becomes available for the same price. Jobs are continually being reinvented, and people always get to work on the interesting parts that can't be automated. Something that took many people to do one decade only takes one person the next decade. This has been true for hundreds of years from farmers to accountants to cooks to car makers ... This "problem" is here to stay, our economy hasn't crashed yet, and there are as many creative people as ever.
• moozilla 10y ago
  Also in the clips in the video is fooled all _three_ of the participants tested. Couldn't find anything in the paper about sample sizes... but hopefully it was more than three...