3 ms·

Well, I don't see anyone really commenting so far that has listened to much JRE, but I've probably listened to 200 episodes or so, and while perhaps the tone /

Well, I don't see anyone really commenting so far that has listened to much JRE, but I've probably listened to 200 episodes or so, and while perhaps the tone / style / type of humor of the provided results you got weren't far off, the actual descriptions and prescriptions and content was just all wrong. Like those aren't actual summaries or representations of anything I've seen the quoted figures say So I guess, you made an effective fake news generator / textual deepfake that creates falsely attributable statements/quotes that at a surface level sound reasonable, but don't actually represent the person being modeled's real views. Cool I guess but also really dangerous if people start linguistically shadowboxing fake versions of their ideological opponents that don't even share the same views as the real opponent. I imagine a world where Jordan Peterson fanboys have arguments with a virtualized Fauci (that makes stylistically accurate but factually incorrect defenses of the vaccine). Neat experiment but I'd really caution being careful in how you describe your results.

EniasCailliau 4y ago
Appreciate the comment. Are you referring to the summaries at the bottom of the blog post? Happy to hear more (here or on enias@steamship.com) — if we drew an inaccurate excerpt we should fix it.
- rngname22 4y ago
  If the goal of the tldr tool were to extract a thesis statement of a broad summary of the content of the episode, it doesn't seem to do that, rather just a random selection of phrases or sentences taken out of context and stitched together. And the excerpts in the blog post don't really fit my mental model of what the guests would say, or aren't complete enough to give any real understanding. But the audio transcription and searchability and sentiment analysis stuff is neat. Just don't think your summarization feature is any good.
  - EniasCailliau 4y ago
    Appreciate the feedback. Whether or not a model’s summary represents the underlying text is absolutely something we all have to pay careful attention to as these things improve. One thing we’ve seen work well is systems that always link back to the source so you can see the content that underlied the synthesis. Not a full solution, but important to keep that provanance. I'll update the UI so the reader can listen to each chapter by themselves and add a disclaimer that these summaries are AI generated and may lack context or be false.

EniasCailliau 4y ago

Appreciate the comment. Are you referring to the summaries at the bottom of the blog post? Happy to hear more (here or on enias@steamship.com) — if we drew an inaccurate excerpt we should fix it.

rngname22 4y ago
If the goal of the tldr tool were to extract a thesis statement of a broad summary of the content of the episode, it doesn't seem to do that, rather just a random selection of phrases or sentences taken out of context and stitched together. And the excerpts in the blog post don't really fit my mental model of what the guests would say, or aren't complete enough to give any real understanding. But the audio transcription and searchability and sentiment analysis stuff is neat. Just don't think your summarization feature is any good.
- EniasCailliau 4y ago
  Appreciate the feedback. Whether or not a model’s summary represents the underlying text is absolutely something we all have to pay careful attention to as these things improve. One thing we’ve seen work well is systems that always link back to the source so you can see the content that underlied the synthesis. Not a full solution, but important to keep that provanance. I'll update the UI so the reader can listen to each chapter by themselves and add a disclaimer that these summaries are AI generated and may lack context or be false.