pranade

15 ms·

1. 121.▲
   by pranade 13y ago

   Great suggestion... thanks for this one. We're putting this on our list
2. 122.▲
   by pranade 13y ago

   We're in the middle of implementing a more power developer version of the tool to handle the use cases you're talking about. The beginning of this is surfaced under the "advanced" tab in the data model view where we show
3. 123.▲
   by pranade 13y ago

   You've got a valid point. We want to eventually create a space that allows responsible scraping - so webmasters can have access to analytics on what's being scraped and can explicitly turn off kimono APIs for their domains if they
4. 124.▲
   by pranade 13y ago

   Sorry you had a bad first experience. We've tested this on a lot of sites, and it works stably across a lot of different cases, but haven't solved it everywhere. Thanks for letting us know about the discussion page. We'll loo
5. 125.▲
   by pranade 13y ago

   Great, it should work well on webkit browsers, what version of Safari are you using?
6. 126.▲
   by pranade 13y ago

   Thanks, the support tickets really help us debug. We'll look into it and get back to you
7. 127.▲
   by pranade 13y ago

   What page were you trying to hit? We'll check it out
8. 128.▲
   by pranade 13y ago

   Great request... it's on our feature shortlist. Definitely a feature we want to implement as soon as we can (after we tackle some basics like pagination and getting images)
9. 129.▲
   by pranade 13y ago

   Access, legality and rate limiting issues come up a lot. We're working on a couple things to address them. The first is an intelligent job distribution system that consolidates scrapes across users and hits sites (and pages) at human-l
10. 130.▲
    by pranade 13y ago

    thanks. glad you like it. pagination, dynamic tabs (and crawling in general) is a big feature we really want to add soon. a lot of people are asking for it. the challenge will be integrating it with the current UEX which we're trying t
11. 131.▲
    by pranade 13y ago

    Thanks... yes, public data from governments is a great use case. Often a lot of apps built using scrapers will wind up driving up traffic/ sales a the source site so it's okay. We want to do responsible web scraping, so will respe
12. 132.▲
    by pranade 13y ago

    It's a great suggestion, thanks! ... image extraction would be cool, and it's on our shortlist of features to build next
13. 133.▲
    by pranade 13y ago

    We don't support logging in yet, but it's a feature we're working on adding. Scripting will also be cool, but it's right now further down our feature queue
14. 134.▲
    by pranade 13y ago

    JS-heavy sites can be tricky. We position it so it should execute after most of the on-page JS, so it handles a lot of cases. There are still sites that break it though.... we're trying to tackle these guys one by one right now, as we
15. 135.▲
    by pranade 13y ago

    Where are you getting the 404s? We will check into it now
16. 136.▲
    by pranade 13y ago

    If it changes the format significantly, the scraper will break, so for now you'll have to use the tool to rebuild. You will see on your API status page that it's down. As for robots.txt, we do respect it... for now we're leav
17. 137.▲
    by pranade 13y ago

    Thanks we were pretty surprised as well, but we're really grateful for the encouragement
18. 138.▲
    by pranade 13y ago

    Awesome, thanks for the kind words. And for catching that typo, will fix that now
19. 139.▲
    by pranade 13y ago

    Thanks guys, glad you like it. Welcome any feedback so we can make it better!
20. 140.▲
    Show HN: Kimono – Never write a web scraper again(kimonify.kimonolabs.com)

    717 points by pranade 13y ago|230 comments
21. 141.▲
    How Twitter's improving a country's morale(bbc.co.uk)

    1 points by pranade 13y ago|0 comments