5 ms·

The presentation is beautiful and the website is great, but the tech broke so I have no idea how or if this even works. This is a wonderful concept and one I've

The presentation is beautiful and the website is great, but the tech broke so I have no idea how or if this even works. This is a wonderful concept and one I've talked about doing with others. I was really excited to try this. I watched the demo video and it seemed straightforward. I went to try and use it on the demo page it provides, going through and adding things, but when I went to save it, I just received an error that something went wrong. Well, crap. That was a waste of time. Oh well, maybe it's just me. Alright, I'll give it another shot using the website they used in the demo. Opened up a Hacker News discussion page and started to give it a try. Immediately it was far less intelligent than the demo. Clicking on a title proceeded to select basically every link on the page. Somehow I clicked on some empty spots as well. Nothing was being intelligently selected like it was in the demo. Fine, that wasn't working tremendously well, but I wanted to at least see the final result. Same thing: just got an error that something went wrong and it couldn't save my work. Disappointing. I still might try it again when it works 'cause it's a great idea if they really pulled it off. So far: doesn't seem to be the case.

pranade 13y ago
Sorry you had a bad first experience. We've tested this on a lot of sites, and it works stably across a lot of different cases, but haven't solved it everywhere. Thanks for letting us know about the discussion page. We'll look into the bugs right now
- randomdrake 13y ago
  Having written a whole lot of crawling code throughout the years, I can totally understand how monumental of a task this is. This really does look like a cool product. Glad you're actively hunting down ways to improve the demo before asking for money :) Feel free to drop me a line if you want any specifics about the troubles I had.
- faceyspacey 13y ago
  hacker news has probably been scraped too many times. that's why it's not working. it didn't work for me, but other sites worked easily. ...i will comment that the user experience for first adding elements is not what i expected. once u understand it's great overall, but i'd have some popup tooltip type guide thing going on for the first time a user uses it explaining that you should choose one element first, and then check a matching element. ...Then explain the number bubbles next to the property field. You don't get what those numbers are at first. ..Then for the property field, somehow make it more obvious that you should fill it out. Basically you have the interface setup so any action can be done at any time, but it should be presented as a forced sequential set of steps to the end user, at least at first.
  - pranade 13y ago
    Thanks for the suggestions on tool tips... We can also expand the tutorials section to make this more clear.
sswaner 13y ago
It worked great for me (http://www.kimonolabs.com/kimonoapp/aws-status-check http://www.kimonolabs.com/kimonoapp/aws-status-check). It would be helpful if you shared what you tried to do when it failed. I don't work for or with Kimono, but am curious what does and does not work.
BinaryBird 13y ago
In my case, worked for some pages and not for others. Currently I'm using Feedity: http://feedity.com http://feedity.com for all business-centric data extraction and it has been working great (although not as flexible as kimono).
ricardobeat 13y ago
HN pages are possibly the worst case, very hard to infer structure from due to its 1998 coding standards. You'll have a better chance with an alternative interface like http://ihackernews.com/ http://ihackernews.com/ or http://hckrnews.com http://hckrnews.com (no comments though).
- roel_v 13y ago
  OK so you're saying that instead of using a scraper to deal with malformed data out there (the whole reason of its existence), instead we should use a format that is better suited for machine representation? That's like saying 'yeah I've got this car here to take you to places that are very far away, too far to walk; except it doesn't work very well if you want to go far away, so you're betting off just staying at home. Or walk along time.'.
  - ricardobeat 13y ago
    Scrapers exist to take data out of HTML, malformed or not, when an API or feed is not available. Yes, ideally HTML is written in a semantic, annotated machine-parseable way; that also enables smarter search engines, better accessibility, interoperability and so on. That's one of the main reasons behind the changes to the standard made in HTML5. A better example would be "hey I have this cool device that will improve your mileage by 100%, but it only works on cars built according to current emission standards...".
    - pranade 13y ago
      Kimono can handle many pages with malformed/ old HTML. Of course, it's still beta and there are still pages that break it, so we're improving it as we go with the help of early adopters like everyone on this thread. Of course, our goal is an ideal state it works everywhere perfectly :)
- voltagex_ 13y ago
  A lot of the time when I'm writing a scraper it's because of bad/old code or incorrect HTML. So if Kimono has issues with that its utility is reduced.
deleted 13y ago
[deleted]
motyar 13y ago
You must check http://webscrapemaster.com/ http://webscrapemaster.com/ Doing one thing well.

pranade 13y ago

Sorry you had a bad first experience. We've tested this on a lot of sites, and it works stably across a lot of different cases, but haven't solved it everywhere. Thanks for letting us know about the discussion page. We'll look into the bugs right now

randomdrake 13y ago
Having written a whole lot of crawling code throughout the years, I can totally understand how monumental of a task this is. This really does look like a cool product. Glad you're actively hunting down ways to improve the demo before asking for money :) Feel free to drop me a line if you want any specifics about the troubles I had.
faceyspacey 13y ago
hacker news has probably been scraped too many times. that's why it's not working. it didn't work for me, but other sites worked easily. ...i will comment that the user experience for first adding elements is not what i expected. once u understand it's great overall, but i'd have some popup tooltip type guide thing going on for the first time a user uses it explaining that you should choose one element first, and then check a matching element. ...Then explain the number bubbles next to the property field. You don't get what those numbers are at first. ..Then for the property field, somehow make it more obvious that you should fill it out. Basically you have the interface setup so any action can be done at any time, but it should be presented as a forced sequential set of steps to the end user, at least at first.
- pranade 13y ago
  Thanks for the suggestions on tool tips... We can also expand the tutorials section to make this more clear.

sswaner 13y ago

It worked great for me (http://www.kimonolabs.com/kimonoapp/aws-status-check http://www.kimonolabs.com/kimonoapp/aws-status-check). It would be helpful if you shared what you tried to do when it failed. I don't work for or with Kimono, but am curious what does and does not work.

BinaryBird 13y ago

In my case, worked for some pages and not for others. Currently I'm using Feedity: http://feedity.com http://feedity.com for all business-centric data extraction and it has been working great (although not as flexible as kimono).

ricardobeat 13y ago

HN pages are possibly the worst case, very hard to infer structure from due to its 1998 coding standards. You'll have a better chance with an alternative interface like http://ihackernews.com/ http://ihackernews.com/ or http://hckrnews.com http://hckrnews.com (no comments though).

roel_v 13y ago
OK so you're saying that instead of using a scraper to deal with malformed data out there (the whole reason of its existence), instead we should use a format that is better suited for machine representation? That's like saying 'yeah I've got this car here to take you to places that are very far away, too far to walk; except it doesn't work very well if you want to go far away, so you're betting off just staying at home. Or walk along time.'.
- ricardobeat 13y ago
  Scrapers exist to take data out of HTML, malformed or not, when an API or feed is not available. Yes, ideally HTML is written in a semantic, annotated machine-parseable way; that also enables smarter search engines, better accessibility, interoperability and so on. That's one of the main reasons behind the changes to the standard made in HTML5. A better example would be "hey I have this cool device that will improve your mileage by 100%, but it only works on cars built according to current emission standards...".
  - pranade 13y ago
    Kimono can handle many pages with malformed/ old HTML. Of course, it's still beta and there are still pages that break it, so we're improving it as we go with the help of early adopters like everyone on this thread. Of course, our goal is an ideal state it works everywhere perfectly :)
voltagex_ 13y ago
A lot of the time when I'm writing a scraper it's because of bad/old code or incorrect HTML. So if Kimono has issues with that its utility is reduced.

deleted 13y ago

[deleted]

motyar 13y ago

You must check http://webscrapemaster.com/ http://webscrapemaster.com/ Doing one thing well.