5 ms·

Scraping Hacker News on a Schedule with TaskPipes

39 points by f292 11y ago

fulafel 11y ago
This general category of tool seems very useful. Apparently there are many Yahoo Pipes followers. http://www.makeuseof.com/tag/12-best-yahoo-pipes-alternatives-look/ http://www.makeuseof.com/tag/12-best-yahoo-pipes-alternative... Support for email and HTML tables seem to be some distinguishing features of this one.
- giarc 11y ago
  I thought I remember reading that Yahoo was discontinuing the pipes tool?
draugadrotten 11y ago
Looks pretty but I can't find pricing information.
- bold 11y ago
  I was looking for prices as well, or any other indicator for that matter that the tool will be available for a reasonable time at least.
  - iag 11y ago
    +1 I love these tools but before I invest more of my time leveraging this in my workflow I'd want to know that it'll be around for another few months at least...
- ins0 11y ago
  Chatted with one of the guys We're starting to move TaskPipes out of beta at the moment, but our pricing will start at $50/month for the service
johnong 11y ago
I tried NewsBlur as an alternative to Yahoo Pipes. Like it because it has an Android app as well.
chdir 11y ago
I've seen something similar more than once on HN but my google-fu is failing. What other similar services are out there that'll let you smartly scrape a page into a spreadsheet. In the past, I've tried Google spreadsheets with "ImportXML" option but got frustrated after a bit and resorted to python.
- dhruvkar 11y ago
  Kimonolabs.com Allows you to scrape then import that data into a google spreadsheet. But its not the only way to consume the data.
  - chdir 11y ago
    Thanks, that's the one I had in mind. Also found : import.io, scrapinghub.com & 80legs.com
tynman 11y ago
What Open Source projects are similar? It seems like data transformation tasks like this are a universal problem. I'd just do map/reduce, but that's the advantage that comes with being a developer. Great tools for non-developers to do limited programming tasks are incredibly useful.
juanescobarcom 11y ago
Check out Diffbot @ http://www.diffbot.com/ http://www.diffbot.com/ -- you can setup repeat crawls and extract data into .csv/Excel file format or JSON with Crawlbot API, extract data automatically with automatic APIs, or Custom API Toolkit. You can get a free trial account at https://www.diffbot.com/plans/trial https://www.diffbot.com/plans/trial to try it out, and you can get paid plans at http://www.diffbot.com/pricing/ http://www.diffbot.com/pricing/.

39 points by f292 11y ago

This general category of tool seems very useful. Apparently there are many Yahoo Pipes followers. http://www.makeuseof.com/tag/12-best-yahoo-pipes-alternatives-look/ http://www.makeuseof.com/tag/12-best-yahoo-pipes-alternative... Support for email and HTML tables seem to be some distinguishing features of this one.

giarc 11y ago
I thought I remember reading that Yahoo was discontinuing the pipes tool?

draugadrotten 11y ago

Looks pretty but I can't find pricing information.

bold 11y ago
I was looking for prices as well, or any other indicator for that matter that the tool will be available for a reasonable time at least.
- iag 11y ago
  +1 I love these tools but before I invest more of my time leveraging this in my workflow I'd want to know that it'll be around for another few months at least...
ins0 11y ago
Chatted with one of the guys We're starting to move TaskPipes out of beta at the moment, but our pricing will start at $50/month for the service

johnong 11y ago

I tried NewsBlur as an alternative to Yahoo Pipes. Like it because it has an Android app as well.

chdir 11y ago

I've seen something similar more than once on HN but my google-fu is failing. What other similar services are out there that'll let you smartly scrape a page into a spreadsheet. In the past, I've tried Google spreadsheets with "ImportXML" option but got frustrated after a bit and resorted to python.

dhruvkar 11y ago
Kimonolabs.com Allows you to scrape then import that data into a google spreadsheet. But its not the only way to consume the data.
- chdir 11y ago
  Thanks, that's the one I had in mind. Also found : import.io, scrapinghub.com & 80legs.com

tynman 11y ago

What Open Source projects are similar? It seems like data transformation tasks like this are a universal problem. I'd just do map/reduce, but that's the advantage that comes with being a developer. Great tools for non-developers to do limited programming tasks are incredibly useful.

juanescobarcom 11y ago

Check out Diffbot @ http://www.diffbot.com/ http://www.diffbot.com/ -- you can setup repeat crawls and extract data into .csv/Excel file format or JSON with Crawlbot API, extract data automatically with automatic APIs, or Custom API Toolkit. You can get a free trial account at https://www.diffbot.com/plans/trial https://www.diffbot.com/plans/trial to try it out, and you can get paid plans at http://www.diffbot.com/pricing/ http://www.diffbot.com/pricing/.