6 ms·
https://www.findlectures.com https://www.findlectures.com Right now it has an index of ~70k conference talks / lectures / speeches. I'm working on improving it
by garysieling 10y ago
https://www.findlectures.com https://www.findlectures.com
Right now it has an index of ~70k conference talks / lectures / speeches. I'm working on improving it to get slide text and audio quality (for ranking), and getting more historical content.
- WmyEE0UsWAwC2i 10y agoI didn't know i wanted this. How do you scrap all that data?
- garysieling 10y agoI started out scraping sites manually, and started automating more pieces (a lot of sites use wordpress, so they are pretty structured). I'm working on a talk on the subject, so I'll have an article soon that explains better :)
- garysieling 10y agoI wrote up some more info here on data acquisition here (just notes right now) - https://www.findlectures.com/articles/2017/01/22/Software-Architecture-of-FindLectures-com/ https://www.findlectures.com/articles/2017/01/22/Software-Ar...
- WmyEE0UsWAwC2i 10y agoThanks for the write up!
- zump 10y agoHow do you pay for the bandwidth costs for scraping?
- garysieling 10y agoI'm doing everything over my home network
- zump 10y agoANd how much does that cost?
- debamitro 10y agowhat is the stack you used?
- garysieling 10y agoIt's mostly Node right now. I started documenting it in more detail (just notes right now) - https://www.findlectures.com/articles/2017/01/22/Software-Architecture-of-FindLectures-com/ https://www.findlectures.com/articles/2017/01/22/Software-Ar...