5 ms·

I noticed Netflix doing this while watching this series: https://about.netflix.com/news/beyond-goodbye-premieres-november-14 https://about.netflix.com/news/beyo

I noticed Netflix doing this while watching this series: https://about.netflix.com/news/beyond-goodbye-premieres-november-14 https://about.netflix.com/news/beyond-goodbye-premieres-nove... It was so jarring I couldn't watch it and switched to Japanese with subtitles. Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked.

SirFatty 2y ago
Yes, Kung Pow is a great example of how jarring it can be! :-D
- m463 2y ago
  It's funny how older movies like that are unintentionally prescient. Like the Kentucky Friend Movie "Eyewitness News" segment and smart TVs.
- diftraku 2y ago
  "YOU WANT FRIES WITH THAT?"
aaron695 2y ago
[dead]
chedabob 2y ago
It was quite jarring in Amazon's Citadel Diana. The voices were ever so slightly out of sync with the lip movements, and the audio sounded like the studio recording hadn't been processed to match the environment.
lambdaone 2y ago
You can see a lot of open mouth/tongue stuff being skipped. Dealing with the tongue and the inside of the mouth is a huge problem with this sort of visual dubbing. Using traditional techniques, you can model teeth and gums as rigid bodies, and faces as rubber sheets (to first approximation), but tongues, for which you typically have no visual reference in any given shot, are much more difficult to model, and continuously, subtly, on the move. "AI" is the general answer to this problem nowadays, but even ML-based systems struggle to deal with the tongue issue while trying to reconcile visual appearance with animation fidelity.
- basch 2y ago
  But the actors re-recorded their dialog in English in the studio. Why wouldn’t you video record their mouths? In fact, the old punch the mouth out and pop the new one in behind the whole can be surprisingly effective sometimes. I would think having an actual visual reference and doing more of a deepfake on the lips should provide excellent results, vs synthetic lip motion to match the audio only.
deleted 2y ago
[deleted]
js2 2y ago
> Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked. Everything old is new again. Classic Italian films were almost always dubbed by their original actors, to the point that Fellini sometimes didn't even write portions of the dialog till after filming was done. The actors just said random stuff during filming. Originally this was mainly a technology limitation (sets were too noisy), then it became a cost-savings measure and cultural tradition to how films were made in Italy. I always find it a bit of a distraction when I haven't watched an Italian film in a while, but eventually get used to it.
- petters 2y ago
  Dubbing by the original actors is very common when making movies. But usually the dialog is the exact same so it’s not noticed
  - js2 2y ago
    > But usually the dialog is the exact same so it’s not noticed That's not the case for the films I'm describing. Here's the start of La Strada: https://www.youtube.com/watch?v=7jSewbxyHYM https://www.youtube.com/watch?v=7jSewbxyHYM Quinn isn't even speaking Italian during the shooting: As was the common practice for Italian films at the time, shooting was done without sound; dialogue was added later along with music and sound effects. As a consequence, cast members generally spoke in their native language during filming: Quinn and Basehart in English, Masina and the others in Italian. https://en.wikipedia.org/wiki/La_Strada#Sound https://en.wikipedia.org/wiki/La_Strada#Sound It's even more clear in this scene: https://www.youtube.com/watch?v=f7fmvA_KG6U https://www.youtube.com/watch?v=f7fmvA_KG6U
    - MoonGhost 2y ago
      Caligula et Messaline 1981 exist in two close but significantly different versions played by the same actors at the same time.

SirFatty 2y ago

Yes, Kung Pow is a great example of how jarring it can be! :-D

m463 2y ago
It's funny how older movies like that are unintentionally prescient. Like the Kentucky Friend Movie "Eyewitness News" segment and smart TVs.
diftraku 2y ago
"YOU WANT FRIES WITH THAT?"

aaron695 2y ago

[dead]

chedabob 2y ago

It was quite jarring in Amazon's Citadel Diana. The voices were ever so slightly out of sync with the lip movements, and the audio sounded like the studio recording hadn't been processed to match the environment.

lambdaone 2y ago

You can see a lot of open mouth/tongue stuff being skipped. Dealing with the tongue and the inside of the mouth is a huge problem with this sort of visual dubbing. Using traditional techniques, you can model teeth and gums as rigid bodies, and faces as rubber sheets (to first approximation), but tongues, for which you typically have no visual reference in any given shot, are much more difficult to model, and continuously, subtly, on the move. "AI" is the general answer to this problem nowadays, but even ML-based systems struggle to deal with the tongue issue while trying to reconcile visual appearance with animation fidelity.

basch 2y ago
But the actors re-recorded their dialog in English in the studio. Why wouldn’t you video record their mouths? In fact, the old punch the mouth out and pop the new one in behind the whole can be surprisingly effective sometimes. I would think having an actual visual reference and doing more of a deepfake on the lips should provide excellent results, vs synthetic lip motion to match the audio only.

deleted 2y ago

[deleted]

js2 2y ago

> Their mouths looked so stiff and unnatural and out of sync with their expressions that all me and my wife could talk about was how bad it looked. Everything old is new again. Classic Italian films were almost always dubbed by their original actors, to the point that Fellini sometimes didn't even write portions of the dialog till after filming was done. The actors just said random stuff during filming. Originally this was mainly a technology limitation (sets were too noisy), then it became a cost-savings measure and cultural tradition to how films were made in Italy. I always find it a bit of a distraction when I haven't watched an Italian film in a while, but eventually get used to it.

petters 2y ago
Dubbing by the original actors is very common when making movies. But usually the dialog is the exact same so it’s not noticed
- js2 2y ago
  > But usually the dialog is the exact same so it’s not noticed That's not the case for the films I'm describing. Here's the start of La Strada: https://www.youtube.com/watch?v=7jSewbxyHYM https://www.youtube.com/watch?v=7jSewbxyHYM Quinn isn't even speaking Italian during the shooting: As was the common practice for Italian films at the time, shooting was done without sound; dialogue was added later along with music and sound effects. As a consequence, cast members generally spoke in their native language during filming: Quinn and Basehart in English, Masina and the others in Italian. https://en.wikipedia.org/wiki/La_Strada#Sound https://en.wikipedia.org/wiki/La_Strada#Sound It's even more clear in this scene: https://www.youtube.com/watch?v=f7fmvA_KG6U https://www.youtube.com/watch?v=f7fmvA_KG6U
  - MoonGhost 2y ago
    Caligula et Messaline 1981 exist in two close but significantly different versions played by the same actors at the same time.