6 ms·
Hey HN, I posted this in a comment thread the other day and (to my surprise) it got a positive reception so added a few more updates and decided to post it prop
by welanes 7y ago
Hey HN, I posted this in a comment thread the other day and (to my surprise) it got a positive reception so added a few more updates and decided to post it proper.
The idea is to be able to choose a website, select the data you want, and make it available (as JSON, CSV or an API) with as little friction as possible.
Kimono was the gold standard for a while so did yoink some of their ideas, while doing some other things differently.
Still needs some work but as an MVP would appreciate any feedback. Cheers.
- bko 7y agoWhen I saw this service last week, I think you had a section about a paid service where you do the scraping on a server and send the results. Do you offer that? How do you get around anti-scraping technology, if it exists?
- welanes 7y agoYeah, that's offered although it's currently free. No particular tricks to avoid detection. It's Puppeteer under the hood with a few customizations which works well on the majority of sites tested so far. Given the cat-and-mouse game around web scraping you may never cover every website, and that's ok.
- nannal 7y ago>would appreciate any feedback Any option for a firefox build?
- lucasverra 7y agoalso will try when on FF
- welanes 7y agoYes, working on it now.
- giarc 7y agoWhy is one page scrape 2 credits? Why not just 1?
- rapind 7y agoUnrelated question. There's a "Made by Lanes" badge. What was made by lanes.io though? The web page?