5 ms·
This is excellent. Even it if doesn't work for scraping all sites, it simplifies the average use case so much that it's not even funny. Feature proposal: deal
by hcarvalhoalves 13y ago
This is excellent. Even it if doesn't work for scraping all sites, it simplifies the average use case so much that it's not even funny.
Feature proposal: deal with pagination.
- pranade 13y agothanks. glad you like it. pagination, dynamic tabs (and crawling in general) is a big feature we really want to add soon. a lot of people are asking for it. the challenge will be integrating it with the current UEX which we're trying to keep super simple.
- kyriakos 13y ago+1 for paging. really important
- scotty79 13y agoAnother feature, simple one: Allow to add some filters to the data stream. For example: only posts that contain word "bitcoin" in the name or only those with 50 upvotes or more.
- pranade 13y agoThanks for the suggestion... adding to the list :)
- walden42 13y agoMake sure to include regex matching =)
- pranade 13y agoThanks. We support regex matching now. Try dragging to select text, if there's a relevant regex pattern and kimono will find it (there's an example inthe blog post). You can preview (and soon, you'll be able to edit) the CSS and Regex also in advanced mode.
- scotty79 13y agoToo bad I couldn't edit selectors and regex-es at this step. I could implement the filters I needed myself manually like this.
- garyjob 13y agoBesides pagination, you might want to handle nested pages as well as deep linking too. https://krake.io/docs/define-data#next_page_object https://krake.io/docs/define-data#next_page_object