4 ms·
Very interesting, how tools like these can be so much helpful for journalists and generally transparency in government functions. Probably world changing, when
by ecthiender 11y ago
Very interesting, how tools like these can be so much helpful for journalists and generally transparency in government functions.
Probably world changing, when considering that even semi-technical folks can cook up tools to dig into things like this.
I know this tool was by a developer, but scrapinghub has web UI to make scrapers.
- unsettledtck 11y agoFull disclosure, I work for Scrapinghub and the web UI you speak of is Portia - our open source visual web scraper. It's for those who range from non-technical to technical but want a quick way to scrape data. I think it's extremely important to develop tools to democratize the acquisition of data regardless of technical background and skill. Glad you find the article and tool interesting!
- ecthiender 11y agoYes, totally agree with you on the great potential of tools for easy data acquisition. I have personally used Scrapy in the past, I find it to be a great tool. Congratulations on your work!
- unsettledtck 11y agoThank you. Glad you enjoy Scrapy, we're pretty fond of it ourselves!
- benologist 11y agoA similar thing happened in Costa Rica - “You can’t visit 160,000 people,” she notes. “But you can easily interrogate 160,000 records.” http://foreignpolicy.com/2015/05/27/the-data-sleuths-of-san-jose-costa-rica-corruption/ http://foreignpolicy.com/2015/05/27/the-data-sleuths-of-san-...