4 ms·

Show HN: Screen scraping for dummies app

http://netreputation.co.uk/extractor/ I am embarrassed about this version but I thought I would develop it based on feedback. The idea was to be able to create something that any one could start using very fast. Not necessarily programmers.

cadab 15y ago
Clickable . http://netreputation.co.uk/extractor/ http://netreputation.co.uk/extractor/
- combiclickwise 15y ago
  Thanks
justliving 15y ago
nice! What are your future plans for it? E.g. which features do you plan to add for my grand-mother :-) ?
- combiclickwise 15y ago
  thanks. couple of things I have in mind 1. more output formats like JSON, HTML, Doc, RSS, ATOM 2. allow programming in loops.. for example you want to be able to scrape many pages of data for a google search result then you need a hack in the url itself. This is difficult to accomplish but I am think I can add support to some of the pagination patterns to start with 3. to be able to give more control over the HTML, maybe a panel which helps extract the relevant one from the target page
  - justliving 15y ago
    sounds good! I like especially 1/ and would add a dead simple API to it! Is this part of your plans? Besides, it might be intersting to play around w/ your stuff and yahoo-pipes.
    - combiclickwise 15y ago
      Oh and please use it if you want to... I will make sure I reserve a free place for you in case I go the paid route later on :-) my email is in my profile so you can send me yours.
    - combiclickwise 15y ago
      yes. API is definitely a part of it and yes, I was thinking about yahoo pipes as well. :-)
fagatini 15y ago
That is quite nice, good work!
- combiclickwise 15y ago
  Thanks a lot
davewasthere 15y ago
It's kind of like a basic dapper? http://open.dapper.net/ http://open.dapper.net/
- combiclickwise 15y ago
  haven't used it much. I wanted a pattern engine so the intelligence finally rests with the user without the user needing to know regex or even programming.
wrath 15y ago
It would be nice to be able to use xpaths to get to the data I want to extract. Having done quite a few extractors in the past I find xpaths to be the easiest way to finding the correct data. Just a thought.
- combiclickwise 15y ago
  xpaths might be especially useful for programmers yes. I am thinking I will add support for it. Thanks :-)

cadab 15y ago

Clickable . http://netreputation.co.uk/extractor/ http://netreputation.co.uk/extractor/

combiclickwise 15y ago
Thanks

justliving 15y ago

nice! What are your future plans for it? E.g. which features do you plan to add for my grand-mother :-) ?

combiclickwise 15y ago
thanks. couple of things I have in mind 1. more output formats like JSON, HTML, Doc, RSS, ATOM 2. allow programming in loops.. for example you want to be able to scrape many pages of data for a google search result then you need a hack in the url itself. This is difficult to accomplish but I am think I can add support to some of the pagination patterns to start with 3. to be able to give more control over the HTML, maybe a panel which helps extract the relevant one from the target page
- justliving 15y ago
  sounds good! I like especially 1/ and would add a dead simple API to it! Is this part of your plans? Besides, it might be intersting to play around w/ your stuff and yahoo-pipes.
  - combiclickwise 15y ago
    Oh and please use it if you want to... I will make sure I reserve a free place for you in case I go the paid route later on :-) my email is in my profile so you can send me yours.
  - combiclickwise 15y ago
    yes. API is definitely a part of it and yes, I was thinking about yahoo pipes as well. :-)

fagatini 15y ago

That is quite nice, good work!

combiclickwise 15y ago
Thanks a lot

davewasthere 15y ago

It's kind of like a basic dapper? http://open.dapper.net/ http://open.dapper.net/

combiclickwise 15y ago
haven't used it much. I wanted a pattern engine so the intelligence finally rests with the user without the user needing to know regex or even programming.

wrath 15y ago

It would be nice to be able to use xpaths to get to the data I want to extract. Having done quite a few extractors in the past I find xpaths to be the easiest way to finding the correct data. Just a thought.

combiclickwise 15y ago
xpaths might be especially useful for programmers yes. I am thinking I will add support for it. Thanks :-)