5 ms·
Open Source LinikedIn Scraper
Hi All,
I am planning to build a LinkedIn scraper, which would scrape LinkedIn profiles, company pages, groups, searches (sales nav and normal searches), likes, comments, etc, for free. I already a set piece of codebase that I have built out. I wanted to know if anyone would be interested in using an open-source LinkedIn scraper? Will this be good option?
- greenpinia 2y agoAs if there was not enough recruiter spam mail.
- altdataseller 2y agoIronically, sometimes I wish that LinkedIn vs HiQ verdict went the other way and made scraping LinkedIn illegal
- satvikpendem 2y agoThe judge ruled towards LinkedIn in the latest ruling of that case, which I suppose most don't know about due to it being an appeal.
- devildani 2y agoI did read about this. Not most know about this.
- altdataseller 2y agoI’m not a lawyer but does that mean scraping LinkedIn is indeed illegal? Or is it still yet to be determined?
- devildani 2y agoTLDR: The short answer is: it is legal to scrape LinkedIn properly. However, scraping LinkedIn a certain way could be illegal. Scraping data from public LinkedIn profiles? Ok. Scraping data from LinkedIn profiles with fake logged-in accounts? Not ok.
- devildani 2y agohttps://www.linkedin.com/feed/update/urn:li:share:7006059900271235072/ https://www.linkedin.com/feed/update/urn:li:share:7006059900...
- ldjkfkdsjnv 2y agoUse an LLM to do it
- devildani 2y agoI am trying this out, I haven't really written any code to build the application. Generating codes via LLMs.
- noashavit 2y agoThere are services that do that, might not be open source but with free tiers like common room. LinkedIn might not like you scraping the site as well, esp if you use sale navigator to get access to deeper info. Keep that in mind
- devildani 2y agoYes, I am aware about this and I won't be using my account for extraction. Users will put their session cookies for data extraction.
- paulcole 2y agoGreat idea. I love a tool that unexpectedly gets my account banned.
- brianjking 2y agolol
- xorl 2y agoFacts, they will ban your LinkedIn account for scraping. Good luck!
- paulcole 2y agoI work with someone in Spain and asked her to log into my Recruiter Lite account to review candidates. LinkedIn gave a pretty stern warning about sharing the account with anyone and that I’d be banned if it happened again. I’m guessing that the same thing would happen if you started letting a scraper act on your behalf.
- mdbmdb 2y agoI would be interested. I think LinkedIn aggressively tries to shut down things like you are trying to build though
- devildani 2y agoGood to know. Can you share your email id? I will ping you as soon as I have the first version of application ready.
- enjoyyourlife 2y agoFrom https://www.linkedin.com/robots.txt https://www.linkedin.com/robots.txt it says to contact whitelist-crawl@linkedin.com
- KomoD 2y agoThat's just for search engine crawlers, not random people who want to scrape all their data
- whodev 2y agoThis is why I private my social media accounts. There is already enough spam and I don't feel it needs to be any easier.
- vrepo54 2y agoI would be interested. How do you plan on dealing with IP address bans?