5 ms·
Do you have a writeup of how you did it, which tools you used? I have a similar project on a much more mundane topic, where I want to build a searchable index o
by sofixa 2mo ago
Do you have a writeup of how you did it, which tools you used? I have a similar project on a much more mundane topic, where I want to build a searchable index of Youtube videos, blog posts, etc. based on their content.
- primitivesuave 2mo agoI've been meaning to write something about how I did it, and have been putting it off for a long time. I wrote all the tools myself - by that, I mean that everything was a web app running in my browser. My advice would be to first set up an automation for yt-dlp to pull the media, then use Whisper to build a transcription pipeline. Chunk the transcript based on desired result granularity, then store embeddings in local Qdrant. It helps to use an orchestrator to handle all of this - my current recommendation is Dagster (dagster.io).