10 ms·
Show HN: PodText.ai – Search anything said on a podcast, highlight text to play
Hi HN, wanted to share a project that I’ve been working on recently.
PodText allows users to find anything said on a podcast. You can also listen and share clips to a specific part of the podcast audio, simply by highlighting the text of that part. Currently there are just over 25k podcast episodes and I’m adding a lot more in the coming weeks (yes my GPU bill is painful).
In order to monetize it, I’m building a sponsorship database to help sponsors find podcasts and vice versa. This will be sold in the form of a $99/month “PodText Business” subscription. I bet I could charge a lot more to large sponsors but I’ll tweak that as I talk to potential customers.
Right now the UI is very bare bones (doesn’t even have pagination) but I’ll polish it once the data pipeline is working well. Please let me know if you run into any bugs or have any questions about the site or business model.
PS: I'm a regular on HN using my real name but can't post under that account since my employer will fire me if they found out about this project :-)
- shantnutiwari 4y agoWould be nice to have some sort of help--at the moment Im confused what its all about
- Ferdows-ip 4y agoHow are you guys affording the transcription bills? 25,000 episodes at 1 hour long, could cost 30,000 USD at a conservative market rate. Are you guys self funded?
- Ferdows-ip 4y agoThe transcription price would be big for 25,000 episodes at 1 hour long. That's like 30,000 USD. How are you guys funding this?
- nitajain 4y agoVery cool, looking forward to testing the platform!
- moserch 4y agoThis is a game-changer for topic research. Thank you! I also agree that this will be a valuable resource for potential business sponsors and worth the money. One note, I like that it shows you the exact location of the text with the term you are searching for. Is there a way to see the entire transcript or listen to the entire show on your site?
- lappa 4y agoThis is a really interesting project with a lot of potential. If I were a sponsor looking for a podcast I would want my search process to look something like this: - Search for a term relevant to my line of business - See a list of podcasts ordered by % of utterances which contain my key phrase throughout their last N episodes - Annotation of how many listeners each podcast had in last N episodes
- anonbuilder 4y agoAppreciate the feedback! I'll keep these use cases in mind as I build out PodText Business
- drusepth 4y agoBuilding out the search a little more to support exact matches would also be super useful in this flow. For example, I've been on several podcasts talking about Notebook.ai, but searching for the name also matches "notebook", which results in an unusable signal-to-noise ratio (seeing every podcast that says the word "notebook"). Likewise, it'd be great to quote-search exact matches for "Andrew Brown", instead of seeing all podcasts that mention "Andrew" or "brown".
- llambda 4y agoThey could and probably should offer semantic search, which would be far more powerful than searching exact match keywords. If you could identify podcasts that often talk about a domain more broadly, you'll have a higher hit rate and overall a better audience fit.
- jcutrell 4y agoI’ve been creating a semantic search using embeddings tonight against my own podcast transcripts. I’d be happy to have my own content surfacing mechanism like this!
- mritchie712 4y agoThis is amazing. How do you search for two terms at once? e.g. "aboriginal" and "origin". Doesn't seem possible to require both terms are present.
- anonbuilder 4y agoThanks! The search function is built with Algolia, I'm sure they support boolean ops like "AND" but I'll need to dig into their API. I think if you search both terms, transcripts containing both should be ranked higher.
- kwerk 4y agoI’m doing a similar personal product. Highly recommend switching to Typesense before your Algolia trial is up. I’ve heard good things about Meilisearch but Typesense has been rock solid for me.
- anonbuilder 4y agoHaven't heard about Typesense - thanks for the pointer! Btw if you want to trade notes on our projects, feel free to email me: team@podtext.ai
- Oras 4y agoI second that. Typesense is fantastic. I used it for a job board with 3 million and it did great.
- pablomendes 4y agoYou might want to try semantic search instead of fiddling with keywords. Disclaimer: I'm building a plug-and-play semantic search API at https://kailualabs.com https://kailualabs.com
- anonbuilder 4y agoI've been thinking about how to improve search, would love to try a demo of your product!
- 71a54xd 4y agoHaha I built this for fun a few months ago to full text search my own favorite podcasts. What are you guys using to do full text search / indexing?
- anonbuilder 4y agoCurrently I'm using Algolia but others have pointed out some alternatives I'll have to check out. Would love to hear any feedback/ideas you have from your project!
- 71a54xd 4y agoAwesome, I just tweeted at your proj re: gpu
- dendrite9 4y agoSomething like this would be interesting to me, many podcasts I listen to are not here or on some of the other links shared by others. Sometimes I can jump around and find roughly the right space for the segment to share but often I can't. It doesn't feel helpful to share a 2 hour podcast for a minute of discussion.
- pablomendes 4y agoHappy to exchange notes with you on our learnings building https://podsearch.page https://podsearch.page
- johnthescott 4y agoheads up. the "How it Works" link is broken on home page of podsearch.page
- BonoboIO 4y agoWhat software do you use to transcribe the speech? Whisper?
- anonbuilder 4y agoYes, using Whisper running on banana.dev :)
- nshm 4y agoDo I understand correctly banana pricing is that it costs $1.87 per hour, so the hour of audio with large model costs you about $1? Thats probably a bit too expensive compared to cloud providers.
- georgel 4y ago> 1 hour of audio processing with Whisper on Banana costs <30 cents. From their page on Whisper: https://www.banana.dev/deploy-whisper https://www.banana.dev/deploy-whisper
- nshm 4y agoThats probably medium model
- thundergolfer 4y agoWhisper can be run on CPUs and produce high-quality transcripts. Doing it on CPU is like a 10th the price and more horizontally scalable. This is using CPUs: https://modal-labs--whisper-pod-transcriber-fastapi-app.modal.run/ https://modal-labs--whisper-pod-transcriber-fastapi-app.moda.... Transcribes a 1hr podcast in 1min for ~3-5cents.
- jcowdy 4y agoNice work! It would be great if after clicking on the text within the podcast that matches my term, I was brought to that section of the transcript rather than to the beginning.
- anonbuilder 4y agoThat's a great piece of feedback! I'll get this fixed soon, super useful suggestion
- Oras 4y agoThis is great, I was working on something similar in the last few days, but since it is hard to cover every podcast, I stopped to think of a way to niche down. I feel your pain with GPU and scalability to transcript podcasts. I was thinking of adding something like this for the UI https://github.com/johan-akerman/SpotifyTranscripts https://github.com/johan-akerman/SpotifyTranscripts in case you find it useful. Good luck! It is a really nice project.
- anonbuilder 4y agoThanks! I'll check out their UI, mine definitely needs a lot of work :)
- mbesto 4y agoSuper interesting - I did a search on a common business name like "Dropbox" and its clear some ads show up. Wonder if there is any way to parse these out so they don't show in the results?
- anonbuilder 4y agoThat's a great point, I'm parsing out ads to determine sponsorships anyway so filtering these from search should be straightforward. Thanks for the feedback :)
- skykooler 4y agoThis is neat, but would it be possible to search for a multi-word phrase? If I search for a sequence of words I just get results that match one or more of the words but not the phrase itself.
- sva_ 4y agoI thought about making something like this, but one important part - which seems to be missing here - is speaker diarization (identify who says what.) In a world of increasing automated content generation, the "who" might become just as important as the "what" of information.
- kevmo314 4y agoVery neat! Is it possible to browse by topic instead of by podcast? You mentioned 25k podcast episodes but right now I can only browse ~100 of them or so unless I come up with some keywords.
- who-shot-jr 4y agoHello, this looks fantastic, I was looking for something like this a while ago.
- SCUSKU 4y agoHere's my crack at a podcast transcription website: https://podscription.app https://podscription.app I made this while unemployed and the skills I learned from making it helped me land my new job!
- 71a54xd 4y agoAny chance you'd be open to sharing source? Also, curious if you used any tuning on Whisper to get transcription more accurate?
- SCUSKU 4y agoIt's not particularly pretty but the source can be found here: github.com/zachbellay/podscription
- thundergolfer 4y agoThis demo is open-source: https://github.com/modal-labs/modal-examples/tree/main/06_gpu_and_ml/whisper_pod_transcriber https://github.com/modal-labs/modal-examples/tree/main/06_gp.... https://modal-labs--whisper-pod-transcriber-fastapi-app.modal.run/ https://modal-labs--whisper-pod-transcriber-fastapi-app.moda...
- nshm 4y agohttps://podscript.ai/ https://podscript.ai/ too!
- mickelsen 4y agoalso adding https://steno.ai https://steno.ai to the list
- 71a54xd 4y agoWow, how'd you manage to get interviews split into different speakers?
- jono_wilson 4y ago(not the creator, but I've built something similar for personal use) This is a great library for determining which speaker is speaking during each time in an audio file (this is called speaker diarization); I imagine they used it or something like it. Works really well out of the box! https://github.com/pyannote/pyannote-audio https://github.com/pyannote/pyannote-audio
- 71a54xd 4y agothanks!
- mosselman 4y agoHow can I search for a full name or phrase? Something like "Steve Jobs"? Now I would get Steve and Jobs results.
- carlosjobim 4y agoCool idea and I wish you the best. But talking about monetizing might be too early at this point. After a couple of tests it seems like it is completely unusable: - It can't search non-english words - It will change your the spelling of any word to show completely irrelevant results
- idk1 4y agoNice idea but it didn't work for me... you need to look into why "poppadoms or bread" isn't turning up any results. It's one of the lines in every episode of the Off Menu Podcasts, one of the biggest podcasts in the uk.
- eagleinparadise 4y agoWould be nice to have a slider to adjust the timestamp period of time. Timestamps every 3 seconds is distracting to read 4 words per line. Being able to adjust to 10/30/60 second timestamps would be nice to give you a proper paragraph to read (i guess you may not be able to indent cleanly but you get my point)
- 71a54xd 4y agoThis should be possible, since Whisper forks can support exporting results with timestamps per word. Indexing entire sentences / clauses is a more complex problem though.
- gnicholas 4y agoSuper cool. This could help make podcasts much more social. Do you generate the text or use provided transcripts? Are there any legal issues associated with that?
- anon25783 4y agoSeems like Chapo Trap House and The Adam Friedland Show are inexplicably unrepresented not only here but on other similar webapps linked elsewhere in this thread... Strange, considering that those are two incredibly popular podcasts, at least by the metric of Patreon revenue.
- yeahsure22 4y agoPeople don’t like advertising with Nazis so it wouldn’t make sense to pay to transcribe that.
- dec0dedab0de 4y agoI couldn't find either of Jim Cornette podcasts, tiger belly, flagrant, brilliant idiots, and Matt and Shane's secret podcast which I think is number 1 on Patreon. I would assume they started with the podcasts they currently subscribe to, then asked their friends, then started going down one of the charts that rank by downloads. I never heard of the ones you mentioned, and you may not have heard of the ones I mentioned. There are so many podcasts that there are always going to be missing ones. Not to mention, if the goal is to sell ads, then high Patreon earners would not really be your target.
- nonethewiser 4y agoStop baiting partisan bickering
- tootie 4y agoPlease take it as encouragement that this is a real business and not discouragement that it's been done but there are similar products in the market. We use podscribe at my place. It's more about better targeting than discovery because we have an ad sales team but suffice to say there is an established market for this kind of thing.
- dec0dedab0de 4y agoHate to be that guy, but is this legal? I would assume to do something like this you would need to have permission from each of the copyright holders.
- 71a54xd 4y agoHave you tweaked Whisper in any specific ways to improve accuracy?
- lxm 4y agoIf you want to grow your audience user-wise, allowing people to add alerts on specific terms would be pretty cool. There's a set of people I always enjoy listening to whenever they're on a podcast, but it's hard to keep track of everything out there.
- primitivesuave 4y agoI was also exploring this idea a while back, congratulations on taking it past the finish line. Highly recommend you set up CloudFront for serving S3 static resources - right now you are linking directly to S3 which is much more expensive for content distribution, especially since you're serving large audio files.
- culi 4y agofeel free to post any corrections podtext.ai --this post www.listennotes.com --focused a lot more than transcripts, but probably the most complete of all here. steno.ai --most professional seeming and possibly second most complete podscription.app --good search, very limited inventory podscript.ai --pretty but a little buggy, doesn't show timestamps or transcripts in search results podsearch.page --only generates for specific podcasts. can't search across podcasts
- deleted 4y ago[deleted]
- rubslopes 4y agoIf anyone is interested, there's a podcast app called Snipd that integrates transcription. They also use AI to create chapters and snippets. It then creates a feed of snippets, like Instagram reels. Pretty neat. (Not my app; just a happy user.)
- culi 4y agoSurprised to see how many other small projects doing this same thing there are. Kinda seems like a solved problem with ListenNotes. Not affiliated, but I use the service a lot and they have a lot more features than just transcripts including publicly accessible APIs (some of which could probably be utilized by some of the projects posted here)
- jonmc12 4y agoI'd echo this. ListenNotes API is great and I thought podcast search was already solved for devs. I've always enjoyed this question on their FAQ that gives some tips for potential competitors - https://www.listennotes.com/api/faq/#faq2 https://www.listennotes.com/api/faq/#faq2 > There are at least 3,035,027 podcasts and 156,316,374 episodes on the Internet...
- culi 4y agoIn addition, it's free for users and the API's free plan is sufficient for most personal uses. Hard to see how any of these newcomers could compete unless they had funding behind them
- guiambros 4y agoReally interesting service; well done! Looking forward to seeing how it will evolve. Quick feedback: it seems you're using fuzzy search, which may or may not be what the user expects. For example, searching for "FPGA" also matches "figa", "fuga", "Fagan", "FPNA", "PGA" (tour), and a bunch of other irrelevant terms. Using quotes to indicate I'm looking for an exact match didn't help.
- paulryanrogers 4y agoDo you get permission from podcast creators? Because these transcripts are probably derivative works.
- mike_d 4y agoIt hasn't been litigated yet, but the closest analog is closed captioning. You can legally create closed captions for someone elses work, and you then own the copyright.
- Geee 4y agoI think the closest analog is audiobooks. You can't create audiobooks without permission from the original creator of the book. Your derivative work has it's own copyright, but it only applies to the work you've done, i.e. the audio. You can't steal someone's original work just by adding something to it. You get copyright on the addition only.
- mindwork 4y agowow, I had the same idea as my next home project. I put off working on it for too long. And this looks exactly as what I was hoping to achieve. Guess I have to find another idea. As mentioned in the comments there are so many great services in this space. I had this problem of vaguely remembering something on the podcast that I listen to, and I wanted to find it by keywords. Using podtext.ai I was able to do it. So I consider it solved
- sciencesama 4y agoMay be like zoom linking of different speakers !! Is there an open source of this ?? Google transcribes all its podcasts automatically!!
- marcelfahle 4y agoLooks very cool! Besides the search, which works great and feels like a powerful tool, I like how snappy the word highlighting is in the transcripts. Can you elaborate a bit on how that's done? Thank you, this is really great work!
- mike_d 4y agoAmazing. I was literally just building the same thing. I have a little over 50k podcast episodes with karaoke style word by word transcription. On to the next project I guess.
- ProcNetDev 4y agoThere are dozens of us! This looks nice and the UI is better than I would have done. Good job, OP. I guess I can stop running my GPU every night. I might still try to release a nice bundled-up docker container that can STT a podcast RSS into a text RSS. Some podcasts are enjoyable to listen to, but some I just want to skim the text.
- jerrygoyal 4y agoI don't like that attitude ;)
- ramraj07 4y agoTbh this is like the single most common idea every tech guy gets when they’re frustrated with the problem. Many like me move on when we realize how niche and potentially small a TAM it could be (of course any idea can generalize in creative ways). So no harm as long as it was fun. Also many options can co-exist!
- unmonoqueteclea 4y agoI did the same at https://voilib.com https://voilib.com ;)
- whazor 4y agoI thought for Whisper (https://github.com/openai/whisper https://github.com/openai/whisper) you actually do not need GPU's and can use a CPU?
- SCUSKU 4y agoYou can use CPU, it's but using GPU in my experience has been about a 20x speedup.
- snozolli 4y agoFrom Noiser's latest episode of "Short History Of", at 0:14, the phrase "he's a law student" is transcribed as "his aloer student". Apparently aloer is a real word, but not one I've ever heard, and not one that has anything to do with Indian history (the subject of said podcast episode).
- rohit89 4y agoHow do you discover new podcasts to index?