4 ms·

Hey @wolongong942! Nice- I'm glad you're finding it useful! VideoMentions Search is built entirely using serverless technologies, so it should scale really wel

Hey @wolongong942! Nice- I'm glad you're finding it useful! VideoMentions Search is built entirely using serverless technologies, so it should scale really well. There's no single server or database to act as a bottleneck. When you perform a search, your browser fires off a number of network requests to serverless function API endpoints that fetch the data from YouTube, then return a response. So if you do an "All time" search on a channel with 2 million videos, your laptop fan may kick on while your browser works hard firing off thousand and thousands of requests until it's fetched all 2 million videos, you run out of memory, or you manually hit the "Cancel" button - whichever comes first :) Your regex idea is interesting, and I'll consider implementing some more complex rules like that if enough people request it. For now, my answer would be to either perform separate searches (like one for "yc" and another for "ycombinator"), or perform one search, then use ctrl/cmd+f to search within the matches displayed on the page. Thanks for checking out my project, and for the great feedback!

messe 4y ago
What're the (approximate) costs for monthly usage? (Or if you don't know yet: what're you budgeting for?)
- kellenmace 4y ago
  Hey @messe! I built VideoMentions Search with a combination of custom code and a couple NPM packages that are able to scrape publicly accessible YouTube pages to get the channel and videos data the app relies on. So because it doesn't use the YouTube API at all, the total monthly cost is $0 :) That's why I'm able to offer free, unrestricted access to the tool and allow folks to perform unlimited searches. Thanks for checking out my project! I hope you find it useful!
silax 4y ago
This is such a boss response. Good inspiration to incorporate attacker attacking into systems I build going forward. "Heh I might've just bricked you" -> "I don't think so, but you might've bricked yourself." Next step- have the browser send a single request per video, directly to YouTube. Actually, make it 2 or 3 just for redundancy. Then do the caption processing in-browser. Or better yet, distribute an ~electron~ app that runs this locally in the background with a separate process to autorespawn if quit. Separately and seriously, you should triple your pricing for this.
- kellenmace 4y ago
  Hey @silax! Good thoughts. Some requests can't be sent from the browser directly to YouTube due to the Content Security Policy directives that YouTube has in place, though. For example, if you try to run this code to fetch a YouTube video page in the browser console from any non-YouTube site (like Hacker News or VideoMentions), you'll see the it errors out: (async () => { const response = await fetch('https://www.youtube.com/watch?v=irjc1nJ1eJs'); https://www.youtube.com/watch?v=irjc1nJ1eJs'); console.log(response); })(); That's why my app uses serverless API endpoints as a middleman. It works like this: 1. Browser fires off a request to the API endpoint. 2. The serverless function Node.js process spins up, fetches the data from YouTube, returns the response, then spins back down. 3. My app takes the video data in the response, saves it in memory, searches it for matches, and re-renders the UI to show the matches, if any. I built my app to do as much work as possible client-side, and to use serverless function API endpoints for anything that can't be done in-browser. Thanks for checking out my project!

messe 4y ago

What're the (approximate) costs for monthly usage? (Or if you don't know yet: what're you budgeting for?)

kellenmace 4y ago
Hey @messe! I built VideoMentions Search with a combination of custom code and a couple NPM packages that are able to scrape publicly accessible YouTube pages to get the channel and videos data the app relies on. So because it doesn't use the YouTube API at all, the total monthly cost is $0 :) That's why I'm able to offer free, unrestricted access to the tool and allow folks to perform unlimited searches. Thanks for checking out my project! I hope you find it useful!

silax 4y ago

This is such a boss response. Good inspiration to incorporate attacker attacking into systems I build going forward. "Heh I might've just bricked you" -> "I don't think so, but you might've bricked yourself." Next step- have the browser send a single request per video, directly to YouTube. Actually, make it 2 or 3 just for redundancy. Then do the caption processing in-browser. Or better yet, distribute an ~electron~ app that runs this locally in the background with a separate process to autorespawn if quit. Separately and seriously, you should triple your pricing for this.

kellenmace 4y ago
Hey @silax! Good thoughts. Some requests can't be sent from the browser directly to YouTube due to the Content Security Policy directives that YouTube has in place, though. For example, if you try to run this code to fetch a YouTube video page in the browser console from any non-YouTube site (like Hacker News or VideoMentions), you'll see the it errors out: (async () => { const response = await fetch('https://www.youtube.com/watch?v=irjc1nJ1eJs'); https://www.youtube.com/watch?v=irjc1nJ1eJs'); console.log(response); })(); That's why my app uses serverless API endpoints as a middleman. It works like this: 1. Browser fires off a request to the API endpoint. 2. The serverless function Node.js process spins up, fetches the data from YouTube, returns the response, then spins back down. 3. My app takes the video data in the response, saves it in memory, searches it for matches, and re-renders the UI to show the matches, if any. I built my app to do as much work as possible client-side, and to use serverless function API endpoints for anything that can't be done in-browser. Thanks for checking out my project!