3 ms·
That's a good point. There are various xml markup formats for synthesizing text to speech that let you tag words for emphasis and pitch, but there's not granula
by DonHopkins 5y ago
That's a good point. There are various xml markup formats for synthesizing text to speech that let you tag words for emphasis and pitch, but there's not granular or expressive enough to mark up individual syllables of words, and it's not useful for singing, for example.
It would get really messy of you had to put tags around individual letters, and letters don't even map directly to syllables, so you'd need to mark up a phonetic transcription. At that point you might as well use a binary file format, not xml.
But for that kind of stuff (singing), there are great tools like Vocaloid (which is a HUGE thing in Japan):
https://www.vocaloid.com/en/ https://www.vocaloid.com/en/
https://en.wikipedia.org/wiki/Vocaloid https://en.wikipedia.org/wiki/Vocaloid
VOCALOID5 - Walkthrough
https://www.youtube.com/watch?v=UAtVGHl1AFM https://www.youtube.com/watch?v=UAtVGHl1AFM
(Check out the "Cool / Cute" slider at 8:22!)
Here's a much simpler and cruder tool I made years ago (when xml was all the rage) for editing and "rotoscoping" speech pitch envelopes, called the "Phoneloper" -- not quite as polished and refined as Vocaloid, but it was sure fun to make and play with:
Phoneloper Demo
The Phoneloper is a toy/tool for creating and editing expressive speech "Phonelopes" that Don Hopkins developed for Will Wright's Stupid Fun Club in 2003, using Python + Tkiter + CMU's "Flite" open source speech synthesizer. It modified Flite so it could export and import the diphone/pitch/timing as xml "Phonelopes", so you could synthesize a sentence to get an initial stream of diphones and a pitch envelope. Then you could edit them by selecting and dragging them around, add and delete control points from the pitch and amplitude tracks to inflect the speech, stretch the diphones to change their duration, etc. It was not "fully automatic", but you could load an audio file and draw its spectrogram in the background of the pitch track, so you stretch the diphones and "rotoscope" the pitch track to match it.
https://www.youtube.com/watch?v=qy5cqV8ypIs https://www.youtube.com/watch?v=qy5cqV8ypIs
- deleted 5y ago[deleted]
- xhevahir 5y ago> and it's not useful for singing, for example. There's this: http://zeehio.github.io/festival/doc/Singing-Synthesis.html http://zeehio.github.io/festival/doc/Singing-Synthesis.html It won't sound as good as Vocaloid, but it uses XML with singing synthesis.
- SemanticStrengh 5y agoDo you know greats vocaloids for songs? the best I know being the soul crushing https://youtu.be/W2TE0DjdNqI https://youtu.be/W2TE0DjdNqI
- notjulianjaynes 5y agoDECTalk (famously, the TTS used by Stephen Hawking) lets you mark up individual syllables, and it even had a singing feature circa 1985.[1] It is indeed messy though. This is the example code to make it sing Happy Birthday: >DECtalk Software Sample Singing Program [:phoneme on] [hxae<300,10>piy<300,10> brr<600,12>th<100>dey<600,10> tuw<600,15> yu<1200,14>_<120>] [hxae<300,10>piy<300,10> brr<600,12>th<100>dey<600,10> tuw<600,17> yu<1200,15>_<120>] [hxae<300,10>piy<300,10> brr<600,22>th<100>dey<600,19>dih<600,15>rdeh<600,14&g;ktao<600,12>k_<120>_<120>] [hxae<300,20>piy<300,20> brr<600,19>th<100>dey<600,15> tuw<600,17> yu<1200,15>] --- [1] https://vt100.net/manx/details/1,1230 https://vt100.net/manx/details/1,1230 (92mb pdf so I won't link directly to it, but info on singing feature appears on p. 116 for the curious)
- DonHopkins 5y agoHere's a historic DECTalk Duet song from Peter Langston (which is actually quite lovely): Eedie & Eddie (And The Reggaebots) - Some Velvet Morning (Peter Langston) https://www.youtube.com/watch?v=1l0Ko1GUiSo https://www.youtube.com/watch?v=1l0Ko1GUiSo Peter S. Langston - "Some Velvet Morning" (By Lee Hazelwood) - Performed By Eedie & Eddie And The Reggaebots http://www.wfmu.org/365/2003/169.shtml http://www.wfmu.org/365/2003/169.shtml Eedie & Eddie On The Wire http://www.langston.com/SVM.html http://www.langston.com/SVM.html Peter Langston's Home Page: http://www.langston.com/ http://www.langston.com/ His 1986 Usenix "2332" paper: http://www.langston.com/Papers/2332.pdf http://www.langston.com/Papers/2332.pdf How to use Eddie and Eedie to make free third party long distance phone calls (it's OK, Bellcore had as much free long distance phone service as they wanted to give away for free): https://news.ycombinator.com/item?id=22308781 https://news.ycombinator.com/item?id=22308781 >My mom refused to get touch-tone service, in the hopes of preventing me from becoming a phone phreak. But I had my touch-tone-enabled friends touch-tone me MCI codes and phone numbers I wanted to call over the phone, and recorded them on a cassette tape recorder, which I could then play back, with the cassette player's mic and speaker cable wired directly into the phone speaker and mic. >Finally there was one long distance service that used speech recognition to dial numbers! It would repeat groups of 3 or 4 digits you spoke, and ask you to verify they were correct with yes or no. If you said no, it would speak each digit back and ask you to verify it: Was the first number 7? ... >The most satisfying way I ever made a free phone call was at the expense of Bell Communications Research (who were up to their ears swimming in as much free phone service as they possibly could give away, so it didn't hurt anyone -- and it was actually with their explicitly spoken consent), and was due to in-band signaling of billing authorization: When you called (201) 644-2332, it would answer, say "Hello," pause long enough to let the operator ask "Will you accept a collect call from Richard Nixon?", then it would say "Yes operator, I will accept the charges." And that worked just fine for third party calls too! >Peter Langston (working at Bellcore) created and wrote a classic 1985 Usenix paper about "Eedie & Eddie", whose phone number still rings a bell (in my head at least, since I called it so often): [...] >(201) 644-2332 or Eedie & Eddie on the Wire: An Experiment in Music Generation. Peter S Langston. Bell communications Research, Morristown, New Jersey. >ABSTRACT: At Bell Communications Research a set of programs running on loosely coupled Unix systems equipped with unusual peripherals forms a setting in which ideas about music may be "aired". This paper describes the hardware and software components of a short automated music concert that is available through the public switched telephone network. Three methods of algorithmic music generation are described.
- follower 5y agoFWIW the "Speech Synthesis Markup Language" (SSML) standard does have sections for "3.2 Prosody and Style" https://www.w3.org/TR/speech-synthesis11/#S3.2 https://www.w3.org/TR/speech-synthesis11/#S3.2 & "3.1.10 phoneme Element" https://www.w3.org/TR/speech-synthesis11/#edef_phoneme https://www.w3.org/TR/speech-synthesis11/#edef_phoneme in it which can enable quite granular control. Of course, whether a particular speech synthesis system supports such features is another thing. It also has a `pitch_contour` attribute: https://www.w3.org/TR/speech-synthesis11/#pitch_contour https://www.w3.org/TR/speech-synthesis11/#pitch_contour Coincidentally enough I pretty much only know any of this because a few years ago I created a GUI for a client which enabled an assistive technology researcher to "draw" in the pitch contour required for a word/phrase (not unlike the project demonstrated in your video :) ) from which the SSML was then generated.