4 ms·
I find that I frequently want to dump a table from some webpage into a sqlite db file and run queries on it. Most sites don't offer a CSV or XLS download option
by curun1r 9y ago
I find that I frequently want to dump a table from some webpage into a sqlite db file and run queries on it. Most sites don't offer a CSV or XLS download option, so a simple script to scrape the data usually ends up being the simplest option.
- ospider 9y agoI wrote web scrapers as my daily work. For so many times, people(coworkers) just came to me saying "Hi, can you scrape this site for me?", however, what they want might just be `extracting` some table out of a login-protected page. It made no sense to simulate the login in a script or something and extract it by xpath or css. So I just came up with a Chrome Extension(100 SLOC) that tries to extracting the main table from a web page, then you can copy the table to Excel or anything you like. It's kinda like Readability, but only for tables, not articles. [1] https://github.com/yifeikong/listify/ https://github.com/yifeikong/listify/