6 ms·

Halfway through the article, he shows a picture with mouths for a-z, sh and th: https://cdn-images-1.medium.com/max/1600/0*gwbWfYm-iQIPEXXP%2E https://cdn-image

by twiss 9y ago

Halfway through the article, he shows a picture with mouths for a-z, sh and th: https://cdn-images-1.medium.com/max/1600/0*gwbWfYm-iQIPEXXP%2E https://cdn-images-1.medium.com/max/1600/0*gwbWfYm-iQIPEXXP%... There are others if you search images for "lip sync animation". It would be interesting to see if the result is understandable if you simply replace the emojis with those images.

_nalply 9y ago
Lipreading is different. For example the «sh» sound is made by slightly pursing the lips but not all speakers do this. In German it gives a very clear visual picture.
- falsedan 9y ago
  I would only purse my lips if I were performing a shush. In words like ‘hash’ I’d just close my mouth slightly & leave lips parted and wide from the ‘ha’ sound. I agree that the mouth shapes are garbage in this demo. A shame because that is the most interesting & important part of animating speech.
  - maury91 9y ago
    This demo is done with my nearly 0 knowledge in lips reading, the effect I was searching was to make it more realistic, but with the limited amount of usable emojis I cannot have it very realistic, just better than random. At the end of the article I propose some gotchas to improve it: use better images, use a better map. I'm happy that this article has triggered some people curiosity and can maybe evolve in something more advanced
  - _nalply 9y ago
    That's why I am bad at lipreading in English. People have mouth shapes all over the place. I'm sure I could learn lipreading a friend I meet often well, however this doesn't help lipreading a stranger. When traveling in the US I always have a notebook ready.
nurettin 9y ago
That image does not map to letters, it maps to the phonetic alphabet. English has different phonetics for words when compared to their written counterparts. For example "money" needs to be converted to "mahnee" before being converted to emoji counterpart. Otherwise it is gibberish.
- yesenadam 9y ago
  'mah-' is US English, not English :P It's 'mah' in one country. p.s. Seems every day on here I'm reminding people from the US that their country is not the world, their version of English is spoken and spelt in just one country, etc etc etc. Maybe I should give up doing that. The most classic maybe was a linguist (yes, amazing) who on her blog was gloating that her variety of US English was particularly rich, in having almost every vowel sound in English. I looked at her list.. it didn't include the 'o' sound in 'pot'..
  - viridian 9y ago
    Either you don't understand that he means the pheonetic alphabet, or you say money in a way that is foreign to almost any person in the UK as well as the US. Do you say mooney or mohney? If not I think your phoneme almost certainly must be mah.
    - namelost 9y ago
      British English has at least two pronunciations of money that I know of, /mʌni/ and /mʊni/
      - viridian 9y ago
        The former is the most common way for Americans to pronounce it, and matches the mouth shape of ah, and the latter, while not common in the US, also matches the ah mouth shape. I really don't know how yesenadam is pronouncing it such that his mouth doesn't take the ah shape.
        9y ago

by twiss 9y ago

_nalply 9y ago

Lipreading is different. For example the «sh» sound is made by slightly pursing the lips but not all speakers do this. In German it gives a very clear visual picture.

falsedan 9y ago
I would only purse my lips if I were performing a shush. In words like ‘hash’ I’d just close my mouth slightly & leave lips parted and wide from the ‘ha’ sound. I agree that the mouth shapes are garbage in this demo. A shame because that is the most interesting & important part of animating speech.
- maury91 9y ago
  This demo is done with my nearly 0 knowledge in lips reading, the effect I was searching was to make it more realistic, but with the limited amount of usable emojis I cannot have it very realistic, just better than random. At the end of the article I propose some gotchas to improve it: use better images, use a better map. I'm happy that this article has triggered some people curiosity and can maybe evolve in something more advanced
- _nalply 9y ago
  That's why I am bad at lipreading in English. People have mouth shapes all over the place. I'm sure I could learn lipreading a friend I meet often well, however this doesn't help lipreading a stranger. When traveling in the US I always have a notebook ready.

nurettin 9y ago

That image does not map to letters, it maps to the phonetic alphabet. English has different phonetics for words when compared to their written counterparts. For example "money" needs to be converted to "mahnee" before being converted to emoji counterpart. Otherwise it is gibberish.

yesenadam 9y ago
'mah-' is US English, not English :P It's 'mah' in one country. p.s. Seems every day on here I'm reminding people from the US that their country is not the world, their version of English is spoken and spelt in just one country, etc etc etc. Maybe I should give up doing that. The most classic maybe was a linguist (yes, amazing) who on her blog was gloating that her variety of US English was particularly rich, in having almost every vowel sound in English. I looked at her list.. it didn't include the 'o' sound in 'pot'..
- viridian 9y ago
  Either you don't understand that he means the pheonetic alphabet, or you say money in a way that is foreign to almost any person in the UK as well as the US. Do you say mooney or mohney? If not I think your phoneme almost certainly must be mah.
  - namelost 9y ago
    British English has at least two pronunciations of money that I know of, /mʌni/ and /mʊni/
    - viridian 9y ago
      The former is the most common way for Americans to pronounce it, and matches the mouth shape of ah, and the latter, while not common in the US, also matches the ah mouth shape. I really don't know how yesenadam is pronouncing it such that his mouth doesn't take the ah shape.
      - 9y ago