5 ms·
Dude, you're making ScraperAPI requests (with your API key in plain text as a parameter) directly from the client app...this thing is a nice idea but it'd proba
by gerpsh 7y ago
Dude, you're making ScraperAPI requests (with your API key in plain text as a parameter) directly from the client app...this thing is a nice idea but it'd probably be a good idea to clean it up and repost.
- criddell 7y agoI thought these multi-region searches were against Craigslist's terms of service.
- smt88 7y agoCraigslist is famously aggressive about scraping, too. While helping with some university research, I found that their anti-scraping defenses were really difficult to get around. They've also been litigious about this in the past, so this project would undoubtedly get a cease-and-desist if it ever became a blip on their radar.
- MegaButts 7y agoI'm guessing that's because of AirBnB and how they got their start by essentially stealing posts from Craigslist.
- massivecali 7y agoI thought Padmapper got their start that way, by essentially providing a better apartment hunt interface than CL had at the time.
- rhizome 7y agoAnd they got the banhammer, too. Not too hard to see where the fulcrum of this "settlement" is: https://www.eff.org/deeplinks/2015/06/padmapper-and-3taps-settle-suit-craigslist-over-use-real-estate-facts https://www.eff.org/deeplinks/2015/06/padmapper-and-3taps-se...
- CalRobert 7y agoWhich was a shame, because padmapper stuck it on a map and linked to Craigslist, instead of just taking the content. If Google Maps did it nobody would think twice.
- giancarlostoro 7y agoCraigsList might make them think twice. I don't think they want to overly exhaust their own resources, what they have works for them.
- Reedx 7y ago> I found that their anti-scraping defenses were really difficult to get around What sort of defenses are they using?
- stevenwoo 7y agoI was a contractor at a company where one of the divisions did Craiglist scraping as part of the business model and they had a closet full of laptops doing the scraping part of the job - thirdhand what I heard was separate PC's were necessary for the anti-scraping workaround they were using.
- smt88 7y agoThey had fantastic detection of headless Chrome and curl requests, a thorough IP blacklist, aggressive rate limiting, and possibly some JS stuff.
- rhizome 7y agoAll scrapey automated stuff is. I doubt this will be alive next week, or tomorrow.
- throwaway190102 7y agoalready appears broken.
- nydel 7y agoi'm not seeing this, has it already been cleaned up, or am i missing it?
- jolmg 7y agoI see it.
- guessmyname 7y agoI clicked on the “search” button. Then it sent 503 XHR requests: https://i.imgur.com/R9D23iU.png https://i.imgur.com/R9D23iU.png
- chovy 7y agoI know. It’s just a poc
- smt88 7y agoEven a PoC needs basic security before being released publicly. It wouldn't even delay the release by a single day.
- everdev 7y agoProof of Concept is a vague term and doesn't give you any sort of protection or defense. If it's against Craigslist's terms you're taking on some risk by making and public and promoting it.
- eagsalazar2 7y agoWhat a shame to have invested so much in an app that will immediately be shut down by CL.