4 ms·

Show HN: ML paper podcast generator using GPT and Tortoise-TTS

81 points by yacine_ 4y ago

I built a pipeline that turns tweets about ML papers into a podcast. Code's up here. Happy hacking. https://github.com/yacineMTB/scribepod https://github.com/yacineMTB/scribepod

renderingprompt 4y ago
Very impressive
0x008 4y ago
that is ridiculously funny haha
riskable 4y ago
This awesome, haha! It's already more accurate and informative (with better sound quality) than 90% of the podcasts that exist :D
ax8080 4y ago
WOW! What do you use to generate voice? It's SO scary similar to real podcasts. I couldn't find it in a minute in the sources.
- ax8080 4y ago
  and it's funny how sometimes it makes ahhrhrhrhrhhhhh sounds what is the reason behind that?
  - nielsinho 4y ago
    It happens quite often with TorToiSe that it collapses in this way. Especially for unseen tokens that wouldn't have appeared in the training data, which likely consisted of a lot of transcribed stuff and read text like audio books. Trying to make it laugh by prompting it with "hahaha" (which you won't really see in mentioned data) often gets you demon and zombie noises.
- qup 4y ago
  Play.ht
- carlbarrdahl 4y ago
  It's making an api request to play.ht: https://github.com/yacineMTB/scribepod/blob/master/playht.ts#LL33C38-L33C53 https://github.com/yacineMTB/scribepod/blob/master/playht.ts...
  - tehsauce 4y ago
    How did they get to use the joe rogan voice though? It seems that one isn’t public?
    - nielsinho 4y ago
      It uses the TorToiSe TTS model for generation. It's simple to generate conditioning voice latents using short audio samples. Likely transcribed JRE episodes were part of the TorToiSe training data, explaining how it's so good at recreating his voice characteristics in particular.
    - deleted 4y ago
      [deleted]
  - windsignaling 4y ago
    I wonder why the title says that it uses Tortoise TTS? Also interesting that play.ht allows you to clone others' voices.
- yacine_ 4y ago
  That generation uses tortoise-tts. Play.ht has a model called peregrine, I've taken to using a script to call them out. Super cool company & API. I just haven't had time to get my next gen version out.
carlbarrdahl 4y ago
Interesting to read the prompts used to generate these conversations: https://github.com/yacineMTB/scribepod/blob/master/lib/processWebpage.ts#L27 https://github.com/yacineMTB/scribepod/blob/master/lib/proce... > Make the dialogue about this as long as possible.
- yacine_ 4y ago
  No more intuitive interpreter than the english language
  - LawTalkingGuy 4y ago
    I'm borrowing your podcast prompts and one issue I get is that Bob bounces between an asker of questions and a co-announcer. I'm currently adding a bullet point to your instructions which seems to be working so far. Only Alice has read the facts beforehand, Bob should never mention anything Alice hasn't said first. I'm playing with this for popular science articles and am using a two-stage process, one to extract the textual claims from the article and another to rank them for the inclusion into the podcast. I found that just "summarize this" boiled down the wrong things - that there were discoveries not what they were for instance. Please make a bulleted list of all factual claims in this article, do not summarize or include any opinions or non-objective claims. and then Combine similar or duplicate facts. Rate the facts by importance, objectivity, and checkability, and pick the top six for inclusion into a podcast for already semi-informed viewers. When fed an article about the JWST, produced these and others: The two ancient galaxies were found billions of light-years behind a giant galaxy cluster called Abell 2744. -Importance: High, the location of the galaxies helps to understand the structure of the universe -Objectivity: High, this is a factual claim that can be confirmed by the telescope's observations -Checkability: High, the location of the galaxies can be verified through scientific data and observations. The two galaxies existed just 350 to 450 million years after the Big Bang. -Importance: High, this information tells us about the timeline of the universe and when certain galaxies formed -Objectivity: High, this is a factual claim that is supported by scientific data and research -Checkability: High, the age of the galaxies can be verified through scientific observations and data. So far I'm just using the rankings myself to manually pick the facts to discuss but I'm going to prompt it to discuss them itself in context.
TheCaptain4815 4y ago
This is quite an amazing use of this tech.
pikseladam 4y ago
I'm quite impressed and also shocked. Just wow! I believe we will find more useful cases like that in the near future.
georgeburdell 4y ago
Horrifying. “Believe nothing of what you hear and only half of what you see” has new meaning to me.
- epistemer 4y ago
  Imagine if we actually focused on papers being reproducible and deleted papers that were not reproducible or at least classified research that can't currently be reproduced. LLMs trained on reproducible research across domains would be super human.

81 points by yacine_ 4y ago

I built a pipeline that turns tweets about ML papers into a podcast. Code's up here. Happy hacking. https://github.com/yacineMTB/scribepod https://github.com/yacineMTB/scribepod

renderingprompt 4y ago

Very impressive

0x008 4y ago

that is ridiculously funny haha

riskable 4y ago

This awesome, haha! It's already more accurate and informative (with better sound quality) than 90% of the podcasts that exist :D

ax8080 4y ago

WOW! What do you use to generate voice? It's SO scary similar to real podcasts. I couldn't find it in a minute in the sources.

ax8080 4y ago
and it's funny how sometimes it makes ahhrhrhrhrhhhhh sounds what is the reason behind that?
- nielsinho 4y ago
  It happens quite often with TorToiSe that it collapses in this way. Especially for unseen tokens that wouldn't have appeared in the training data, which likely consisted of a lot of transcribed stuff and read text like audio books. Trying to make it laugh by prompting it with "hahaha" (which you won't really see in mentioned data) often gets you demon and zombie noises.
qup 4y ago
Play.ht
carlbarrdahl 4y ago
It's making an api request to play.ht: https://github.com/yacineMTB/scribepod/blob/master/playht.ts#LL33C38-L33C53 https://github.com/yacineMTB/scribepod/blob/master/playht.ts...
- tehsauce 4y ago
  How did they get to use the joe rogan voice though? It seems that one isn’t public?
  - nielsinho 4y ago
    It uses the TorToiSe TTS model for generation. It's simple to generate conditioning voice latents using short audio samples. Likely transcribed JRE episodes were part of the TorToiSe training data, explaining how it's so good at recreating his voice characteristics in particular.
  - deleted 4y ago
    [deleted]
- windsignaling 4y ago
  I wonder why the title says that it uses Tortoise TTS? Also interesting that play.ht allows you to clone others' voices.
yacine_ 4y ago
That generation uses tortoise-tts. Play.ht has a model called peregrine, I've taken to using a script to call them out. Super cool company & API. I just haven't had time to get my next gen version out.

carlbarrdahl 4y ago

Interesting to read the prompts used to generate these conversations: https://github.com/yacineMTB/scribepod/blob/master/lib/processWebpage.ts#L27 https://github.com/yacineMTB/scribepod/blob/master/lib/proce... > Make the dialogue about this as long as possible.

yacine_ 4y ago
No more intuitive interpreter than the english language
- LawTalkingGuy 4y ago
  I'm borrowing your podcast prompts and one issue I get is that Bob bounces between an asker of questions and a co-announcer. I'm currently adding a bullet point to your instructions which seems to be working so far. Only Alice has read the facts beforehand, Bob should never mention anything Alice hasn't said first. I'm playing with this for popular science articles and am using a two-stage process, one to extract the textual claims from the article and another to rank them for the inclusion into the podcast. I found that just "summarize this" boiled down the wrong things - that there were discoveries not what they were for instance. Please make a bulleted list of all factual claims in this article, do not summarize or include any opinions or non-objective claims. and then Combine similar or duplicate facts. Rate the facts by importance, objectivity, and checkability, and pick the top six for inclusion into a podcast for already semi-informed viewers. When fed an article about the JWST, produced these and others: The two ancient galaxies were found billions of light-years behind a giant galaxy cluster called Abell 2744. -Importance: High, the location of the galaxies helps to understand the structure of the universe -Objectivity: High, this is a factual claim that can be confirmed by the telescope's observations -Checkability: High, the location of the galaxies can be verified through scientific data and observations. The two galaxies existed just 350 to 450 million years after the Big Bang. -Importance: High, this information tells us about the timeline of the universe and when certain galaxies formed -Objectivity: High, this is a factual claim that is supported by scientific data and research -Checkability: High, the age of the galaxies can be verified through scientific observations and data. So far I'm just using the rankings myself to manually pick the facts to discuss but I'm going to prompt it to discuss them itself in context.

TheCaptain4815 4y ago

This is quite an amazing use of this tech.

pikseladam 4y ago

I'm quite impressed and also shocked. Just wow! I believe we will find more useful cases like that in the near future.

georgeburdell 4y ago

Horrifying. “Believe nothing of what you hear and only half of what you see” has new meaning to me.

epistemer 4y ago
Imagine if we actually focused on papers being reproducible and deleted papers that were not reproducible or at least classified research that can't currently be reproduced. LLMs trained on reproducible research across domains would be super human.