3 ms·
Hello Alex, I've built a similar app for my own usage, I don't plan to ship it online. How do you handle token limitation for summarizing lengthy videos?
by jcq3 4y ago
Hello Alex, I've built a similar app for my own usage, I don't plan to ship it online. How do you handle token limitation for summarizing lengthy videos?
- ax8080 4y agoI do many requests. Then I summarize the summaries. This approach is well described in this OpenAI article: https://openai.com/blog/summarizing-books/ https://openai.com/blog/summarizing-books/
- jcq3 4y agoUseful link thanks. This approach is not cost effective though, I guess you're working on solution to optimize your cost. Fine tuning? Embedding?
- ax8080 4y agoHonestly, I haven't worked on cost reduction yet. I have a backlog, but I haven't done anything from there. One idea is to use other models to shorten text by throwing out meaningless words. I estimate this will reduce the length of the text (and thus the GPT cost) by 30%.
- newswasboring 4y ago> One idea is to use other models to shorten text by throwing out meaningless words. Does this mean GPT does not need coherent sentences to understand?
- ax8080 4y agoMy feeling is that she understands much better than people do. If you understand a sentence from which some of the conjunction words have been taken out, she will understand 100%. And I think this will all be smoothed out on a transcript of 150,000 characters (that's the average size of a podcast).
- eob 4y agoHey jcq3 Anything in particular thing stopping you from shipping it online? Or just that it's not necessary for your use? (Context: I've been working on `Heroku for LLM apps` and trying to understand where the value/frictions are)