7 ms·

U.S. Supreme Court revives LinkedIn bid to shield personal data

204 points by anfilt 5y ago

MattGaiser 5y ago
> LinkedIn told the Supreme Court that hiQ's software "bots" can harvest data on a massive scale, far beyond what any individual person could do when viewing public profiles. So would it be legal if I funnelled the profiles into Mechanical Turk and had it manually entered?
fibers 5y ago
What would happen if they ban scraping altogether? Is this the end of google search?
- pionar 5y ago
  I doubt it. Google's crawling bots (like most search engine bots) follow a published standard for access rules for bots (robots.txt).
austincheney 5y ago
It is because I am a strong advocate for privacy that I believe Microsoft is wrong. If Microsoft really wanted to protect user privacy they would restrict access to user data on their platform. This is basic entry level security: apply access control. Microsoft wants to split hairs in order to both maximize revenue and prevent competitor access, which is nothing to do with privacy. If Microsoft wins companies will be able to use this verdict to destroy privacy for revenue gain.
- carbocation 5y ago
  I agree. I want to add that the headline is infuriatingly credulous of Microsoft's claims.
- bingidingi 5y ago
  Yes, important to note that Microsoft is going after a competitor here; there are a bunch of data brokers that have been scraping LinkedIn for years and this is the case they're going after. I hope the Supreme Court can see through this BS and tell them to go pound sand. (If Microsoft wanted to go after data brokers I'd be on board for that tho)
  - nojito 5y ago
    It was hiq that sued linkedin
    - mrastro 5y ago
      After being threaten by LinkedIn: "LinkedIn told hiQ in 2017 to stop scraping LinkedIn's public profiles or face liability under the anti-hacking law." I think it's fair to say LinkedIn is the instigator in this case.
      - Causality1 5y ago
        That seems like it would have been a breathtakingly easy problem to solve using technical means instead of legal threats. They could've required logins to view profile information. They could have blacklisted any IP address exhibiting automated behavior.
        onion2k 5y ago
        That sort of 'solution' is just a game of technical whack-a-mole as one party finds a new way to scrape the data and the other finds a new way to stop them though. If you want to permanently stop scrapping you need a non-technical solution that stops someone even trying.
        mixologic 5y ago
        But thats exactly the issue here. If they find a way to limit the data such that they can split who is "authorized" and who is not, then they have found something that falls under the now new reading of cfaa. It's not going to stop scrapers from entities outside of the jurisdiction of the US legal system, but it's going to end up icing companies in the US who do this. But I think Microsoft/Linked In is going to have a very difficult time redefining authorized when the data is publicly available to anonymous access.
- TeeMassive 5y ago
  And let's not forget that Microsoft probably is using that data when it comes to their own hiring decisions and recruiting efforts.
  - username90 5y ago
    They already sell the data to anyone who wants to pay, they have no moral high ground here at all, the only extra "protection" you'd get is that companies would have to pay Microsoft more money to abuse your data.
- jakelazaroff 5y ago
  Is hiQ a competitor to LinkedIn? They scrape people’s profiles so they can snitch to employers about employees whom they think are job hunting. AFAIK, that’s not a service LinkedIn provides. Moreover, I would like it both ways. Access control would be great, but it sidesteps the underlying issue. I’m fine with my profile being public — in fact, I want it to be — but I don’t want my data to be vacuumed up and weaponized against me. I agree that anti-hacking laws are not appropriate here. But what hiQ is doing is sleazy as heck, and we should absolutely try to throw out as much bathwater as possible without throwing out the baby too.
  - danuker 5y ago
    > I’m fine with my profile being public — in fact, I want it to be — but I don’t want my data to be vacuumed up and weaponized against me. LinkedIn is a authwalled garden only available to those with accounts. "We may or may not have a Jake Lazaroff. Who's asking?". Sounds like you need a personal website.
    - jjk166 5y ago
      When most people say they want things to be public, they typically mean "accessible to those who have a legitimate reason to access it, which may include some people I have not explicitly authorized." Also LinkedIn does have public profile settings which allow some or all of your profile to be visible without an account. https://www.linkedin.com/help/linkedin/answer/83/linkedin-public-profile-visibility?lang=en https://www.linkedin.com/help/linkedin/answer/83/linkedin-pu...
  - mrwww 5y ago
    I believe LinkedIn started providing some services similar to HiQ, which is one of their arguments. How can your profile be public but not vaccumed up? Google vacuums it to index of course. Or perhaps you mean only public on Linkedin? What the Russians or any other bad actor you care to think about already has done with your public profile is probably way way sleazier.
- deleted 5y ago
  [deleted]
lizdax 5y ago
Curious to see which direction the Supreme court goes with this -- given the last CFAA ruling. >LinkedIn told the Supreme Court that hiQ's software "bots" can harvest data on a massive scale, far beyond what any individual person could do when viewing public profiles. So can anyone with a couple thousand dollars to burn. By barring US companies from doing it all you're going to do is move the work to another country that doesn't care about US Law. The data will still be scrapped nonetheless.
- NaOH 5y ago
  >LinkedIn told the Supreme Court that hiQ's software "bots" can harvest data on a massive scale, far beyond what any individual person could do when viewing public profiles. That doesn't seem any different than when I registered a new business in my state. I did that online, and before I received the printed, mailed confirmation from the state I was getting credit card offers in the mail addressed to the business. I'm not glad how that played out, but I don't see any difference with hiQ collecting public information.
  - fncypants 5y ago
    Secretary of State / corporate information is often sold in bulk without any need for scraping without authorization. > Daily Filing Update – a file that includes all of the database updates for a specific day. Customers frequently begin by purchasing a Master Unload in order to create a database, and then subscribe to the Daily Filing Update so that they can download and update their databases on a daily basis to keep their data current with that maintained on the SOS BEST database.[0] [0] https://direct.sos.state.tx.us/help/help-corp.asp?pg=bulk https://direct.sos.state.tx.us/help/help-corp.asp?pg=bulk
- cwkoss 5y ago
  Typo quibble: the data will still be scraped - if it was scrapped it wouldn't be an issue!
cj 5y ago
2 questions I asked myself after reading this: 1) Do I side with LinkedIn or HiQ? 2) What case law precedent do I think we ought to set? I’m personally conflicted on which way to lean. I see pros and cons to both - this is an interesting case.
- wslack 5y ago
  Scraping for non-profit purposes feels fine. Scraping for commercial purposes feels dirty. I don't know if there's a way for the law to square that circle.
  - postalrat 5y ago
    How about a browser extension that scrapes only the the profile you are visiting and adds that information to an ATS. Just to prevent a lot of copy paste.
  - fncypants 5y ago
    If what you are feeling is that scraping for commercial purposes feels "unfair," that is because if it is not allowed it traditionally falls within a legal cause of action commonly called "unfair competition." In the same vein as trade secret theft. So the law does square that circle, but the line drawn is a fuzzy gray one. Its unfair to free ride off a competitor, but yet some kinds of actions that look like free riding are in fact allowed.
  - jrochkind1 5y ago
    When Google does it, it's commercial, right? Should you need a license from all indexed content to make a search engine? Isn't indexing for search what HiQ is doing too?
- cwkoss 5y ago
  I feel strongly that non-commercial scraping should be protected. I think commercial scraping should probably be allowed, but there should be some sort of mandatory maximum retention limit, so deleted (or updated access controls on) content eventually ages out (or gets transferred to a non-commercial custodian for archival purposes).
kizer 5y ago
Scraping is inevitable. It seems like it exists in this grey area of a “hacky” accepted practice. I think it should be expected today. If it’s public, it’s public. Of course copyright applies, but day after tomorrow people will just run copy through a GPT-3-like scrambler or rephrasher or something and publish it anyways. The reality is that anything public on the web will be able to be harvested and made use of —- sort of how it’s been and really how it should be.
gip 5y ago
The headline is misleading. What is happening is that LinkedIn is using user data for its own benefit only and blocking innovation. As a LinkedIn user I want everyone to have access to the data I'm sharing publicly through the service. I can't wait for entrepreneurs to take on LinkedIn. There are a lot of opportunities there.
- mLuby 5y ago
  I'd rather entrepreneurs build on top of LinkedIn. There's so much untapped potential to build upon aggregated markets (of professionals, of jobs, of housing, of singles, of restaurants, etc) rather than building yet another aggregator. The existing aggregator cements its dominant position, can charge rent to connected apps, and attracts rather than repels killer apps. The main problem I see is fears that connected apps will siphon away the aggregator's data, and/or combine that data with external data; both are attempts to supplant the aggregator. Maybe this fear can be assuaged by technical or legal means.
  - jb775 5y ago
    What do you do when you build an entire business on top of LinkedIn, then they decide to completely revoke your access? We need DE-centralization, so no-one in the world has this type of power over anyone.
    - gip 5y ago
      Totally agree. About 10 years ago I was the first hire of a startup built partially on top of LinkedIn. A Monday we got a cease and desist letter. We had to immeditely stop using their API and had to start crawling the data instead. That almost killed us.
      - altdataseller 5y ago
        What was this startup?
altdataseller 5y ago
If they do rule HiQ labs unable to scrape, can they also make sure ZoomInfo stops scraping LinkedIn profiles too? I have seen people’s profiles scraped word for word and posted on ZoomInfo. And sold to salespeople who cold call you. Let put an end to this.
TechBro8615 5y ago
Any ideas how Microsoft wants to frame this so it becomes a partisan issue? Microsoft Legal can’t change the facts of the case, but presumably Microsoft PR wants to shape the narrative into one agreeable to the conservative majority on the court. I have a feeling that regardless of the specifics, it will be a narrative where the little people lose and the big people get to keep playing. I would love to be proven wrong though. If I’m right, it’s a stark reminder that Microsoft need only sustain its open-source friendly, developer-empowering persona as long as it’s convenient and rational. When there’s territory at stake, that veneer of friendliness begins to crack.
rurabe 5y ago
This is a pretty bad headline. I don't know that i would characterize this as revived. The same 9th circuit who held last year that LinkedIn could not block hiQ from scraping public data, just got asked to reconsider the same case, except now there is additional precedent that SCOTUS says if you had permission to access the computer then it's not a violation of the CFAA (even if you are a shady corrupt cop). Hard to see this turning out any other way than the 9th circuit reaffirming their decision (or even strengthening it) and then it's up to LinkedIn to try SCOTUS again
- Goety 5y ago
  I disagree with that SCOTUS decision. It completely obliterated CFAA. Imagine if they said nurses/doctors could do that with their terminals and it didn't violate HIPAA. I will say there is a ridiculous amount of redtape around law enforcement using data. Loopholes with third party access is already something that exist. So if it's above board monitoring would be easier... But I'm not sure we have adequate monitoring let alone enforcement now. I feel this is a weak case to attack 3rd party data scrapers/brokers. The public generally recognizes the monster we created by having life changing data accessible to anyone with $50 and a bank account. I want to side with LinkedIn but realistically I'm becoming more and more jaded on the concept of open internet and iot of everything. I hate the alternative of an open internet worse. I would love to restrict data scrapers but at the same time should we restrict who has that data? I'd rather we shift how we use the internet and socially enforce boundaries on companies. I cant even open my fridge, use my microwave, stove without it being logged either by the electric company or bluetooth enabled appliance with TV and wifi temperature control software company where you hope an update doesnt brick the appliance. There is no way in my mind that data helps the consumer. It might help companies maximize profit but at what energy consumption/cost to the environment?
  - nulbyte 5y ago
    > I disagree with that SCOTUS decision. It completely obliterated CFAA. Imagine if they said nurses/doctors could do that with their terminals and it didn't violate HIPAA. I don't see the similarity between CFAA and HIPAA, here, and SCOTUS didn't obliterate the CFAA. Theybsimply said, if you are authorized to use a system, your use of the system isn't unauthorized. That's fairly straightforward. HIPAA, on the other hand, regulates disclosure of specific data. You can violate HIPAA even if you are authorized to use a system that holds covered data.
rurabe 5y ago
Note that LinkedIn could make this whole issue moot by putting their data behind a login tomorrow. The 9th circuit decision pretty clearly stated that. The only reason the data is public is for marketing purposes, to sign up more users.
- bryan_w 5y ago
  I thought they were prevented from doing that (at least at some point, by injunction)
  - mcny 5y ago
    Each Linked In user has the option to make as much or as little of their information public. I don’t see how any injunction can stop that. None of this data belongs to LinkedIn. It nudges us to make more of our data public. I don’t even want to get into what goes on in the mind of a Linked In “influencer”. I am convinced salvation cannot come from the courts and at the same time Congress has ceded all authority to the White House. The goal for everyone in our industry must be complete abolition of the CFAA but that can’t happen until we can get Congress to act.
    - rurabe 5y ago
      Yeah each user gets the choice right now as a consequence of LinkedIn's product design. As far as I know, there's no reason it couldn't be "Log in to see this profile". I believe the injunction bars LinkedIn from blocking hiQ specifically from accessing the publicly available information (which again is public by LinkedIn's choice). They made a point to draw a distinction between the public stuff and stuff behind a login/password.
- ID1452319 5y ago
  I want some of my LinkedIn profile to be public. I DON'T want any random company scraping that PII data, storing it, processing it and making money from it. Is that too much to ask?
  - 5y ago
jollybean 5y ago
You can walk around the store and 'see' whatever you want, but if you start recording prices and taking pictures, they may ask you to leave. I wonder if there's some relevant precedent there.
ineedasername 5y ago
Doesn't Google pretty much already scrape all of LinkedIn and partially republish in search results (and cached pages)? Microsoft doesn't have a problem with that.
- durnygbur 5y ago
  You've got to be in the cartel.
cletus 5y ago
The Supreme Court curtailed the broad scope of the CFAA recently in the van Buren case. I personally agree with that decision (and also find it funny as it continues the long trend of Clarence Thomas being on the wrong side of history). While van Buren's actions were obviously problematic he was an authorized user of the computer system. The issue is the vagueness of the CFAA has been a prosecutor's wet dream and a predictable source of overreach. Case in point: Aaron Swartz. So here's what's interesting: by taking up this case SCOTUS is potentially going in the other direction. The Appeals Court held that the CFAA didn't apply, allowing hiQ to continue. And here's where (in my layman's view) SCOTUS may choose to act: The Appeals Court stopped LinkedIn taking action to impede hiQ's access. I personally view this as overreach. There's a difference between not "hacking (in CFAA) terms a website and blocking the information provider impeding bots. As much as Microsoft/LinkedIn is a nightmare of data misuse and dark patterns (eg to obtain contacts), I think it would be a bad decision to let the current hiQ ruling stand. Disallowing sites from taking action to block scrapers from systematically taking all that information and building a competitor seems like a bad idea. But given the van Buren ruling, it seems unlikely SCOTUS will reverse course and declare hiQ's actions "hacking" (in CFAA terms). I suspect they'll curtail the remedy.
- dannyw 5y ago
  My counterpoint: companies shouldn't get to have their cake and eat it too, as in offering public content, but then preventing competitors from using that public content. To use another example, what if Google+ was a success, and Google blocks every other competing search engine from indexing even public posts on Google+? Is that a good outcome? There are already copyright protections to stop Shutterstock from ripping, say, all of Getty Image's photos. What we are talking about here is information so banal, so unoriginal, so uncreative, that copyright doesn't even apply.
  - gwoplock 5y ago
    > My counterpoint: companies shouldn't get to have their cake and eat it too, as in offering public content, but then preventing competitors from using that public content. I agree this case is taking an anti-competitive action and spinning it into a pro-privacy case. However, what's the line? Personally I believe if you apply the rules equally to competitors and non-competitors alike you can restrict programmatic access to public data. Since LinkedIn isn't doing that, HiQ shouldn't be restricted. In any case I think this is a poor use of the CFAA.
wly_cdgr 5y ago
I gotta say, making scraping illegal would be a great mercy towards web developers. It's just the worst
gnicholas 5y ago
I had a strange experience with LinkedIn today: I had to restart my computer, and when my browser started it re-opened all my tabs. This included about 50 LinkedIn tabs, which I opened sometime over the last 3 months (most of them were opened yesterday). LinkedIn logged me out and accused me of unauthorized use or some such thing, and then locked me out for a period. When I submitted a help request and explained what had happened (browser reopened tabs), they gave a boilerplate answer that was totally non responsive. I understand they want to defeat scrapers (whether I think they should or not), but outcomes like this are a pretty annoying way to do it. Can they really not tell when a browser is restarting and opening a bunch of tabs that the user has already viewed (and which are all 1st or 2nd degree contacts)? edit: added context around when tabs were opened, in response to a comment.
- Judgmentality 5y ago
  Just curious, how many tabs do you normally have open?
  - gnicholas 5y ago
    Maybe 100? I use a Chromium-compatible version of the Tree-Style-Tabs extension, which makes it easy to organize and navigate dozens of tabs.
- bpodgursky 5y ago
  It seems possible, but it also seems like a sorta painful problem to solve if locking an account requires dredging through 3+ months of a user's interactions. At a lot of companies that kind of historical data ends up getting batched and stored in some kind of timeseries datastore (aka S3) and then batch-operated on via Spark, MapReduce, Flink etc in hourly or daily jobs. So instead of implementing account locking and scraping detection at the realtime layer looking at current requests, you have to write some ugly daily aggregate & backfill job, which inevitably ends up super expensive and slow. Anyway, yes, but I can definitely sympathize with _not_ doing it this way.
  - gnicholas 5y ago
jb775 5y ago
What a joke that the courts within our society are so obviously controlled by big business. LinkedIn is essentially asking for legal protection so they can box out competition as they pimp out user data they don't even really own in the first place. The argument our society should be discussing is that individuals own the underlying data. All previous "user agreements" are null and void. All new user accounts should be bundled like non-fungible tokens, then every piece of data associated with that account can be tracked to compensate that user based on actual impressions. Companies can take their cut, but if a user isn't happy with the setup, they can revoke all access to their own data. Enough of this "you agreed to our service agreement" smoke and mirror bullshit...that's like saying the water company owns the water itself.
- pverghese 5y ago
  By that same token, all use of free services should be charged from the time of use of said services, since previous agreements are nill and void. By using said services you have provided your information given the understanding that your information will be used by the companies that provide the services
- 6gvONxR4sf7o 5y ago
  On the other hand, I want to use linkedin without my data being taken by other companies. It’s my data, not linkedin’s and certainly not hiq’s.
durnygbur 5y ago
> hiQ Labs So the recent LinkedIn "data leak" are public profiles scrapped by hiQ from LinkedIn and then "leaked" again from hiQ? or hiQ is selling the data dumps undercover? Either way hard to pity MS and LI, the website is a Potemkin village...
ramoq 5y ago
you have to watch the video of the actual court proceedings. a core issue is that LinkedIn knew hIQ was scraping their website and allowed them to do this until they started launching similar features. hIQ has a strong argument around this point and evidence proving so. video: https://youtu.be/tvLdJujOp8k https://youtu.be/tvLdJujOp8k
myfavoritedog 5y ago
For its part, hiQ uses the data for products that analyze employee skills or alert employers when they could be looking for a new job. Interesting that they've made a business out of this. I've always maintained to my colleagues that when I see a bunch of LinkedIn activity from them, I can tell that they're putting themselves on the job market.
myfavoritedog 5y ago
Apart from the CFAA issue, it seems strange that you can't regulate access patterns to your own web site in order to prevent mass scraping of user data.
failwhaleshark 5y ago
LinkedIn had a well-known profile viewing crack for years. Add this magic to the URL and poof, you don't need to login or be connected to them. It wasn't even close to a "hack" how trivial it was to bypass LinkedIn's (not) security.
ggggtez 5y ago
> public The entire case is just about that one word. If the profile is public, then you are authorized. We already went through this. CFAA is not a privacy law.
rhacker 5y ago
It seems absolutely nuts to me that this case ended where it did: https://en.wikipedia.org/wiki/Spokeo,_Inc._v._Robins https://en.wikipedia.org/wiki/Spokeo,_Inc._v._Robins And we're having a discussion about linkedin might possibly lose. In the spokeo case it's data that Robins said was untrue about him and Spokeo was allowed to keep sharing it - simply because he couldn't prove harm. Which, is laughable in TODAY's political climate where harm is waking up in the morning - because of all the data sharing happening everywhere and being unable to stop it.
MR4D 5y ago
To me, this is simple: If the pages are not behind a login, then they are PUBLIC, and open to any sort of data gathering. If the pages are behind a login, then they are PRIVATE, and have all the protections alongside it. Frankly, this bullcrap about "it's not public, but we want to leverage the usage of the open web as it suits us" to build our monopoly crap is getting old.[0] [0] - I don't care who the company is, whether Microsoft, Apple, Amazon, Twitter, Facebook, Google, or some no-name company

204 points by anfilt 5y ago

> LinkedIn told the Supreme Court that hiQ's software "bots" can harvest data on a massive scale, far beyond what any individual person could do when viewing public profiles. So would it be legal if I funnelled the profiles into Mechanical Turk and had it manually entered?

fibers 5y ago

What would happen if they ban scraping altogether? Is this the end of google search?

pionar 5y ago
I doubt it. Google's crawling bots (like most search engine bots) follow a published standard for access rules for bots (robots.txt).

austincheney 5y ago

It is because I am a strong advocate for privacy that I believe Microsoft is wrong. If Microsoft really wanted to protect user privacy they would restrict access to user data on their platform. This is basic entry level security: apply access control. Microsoft wants to split hairs in order to both maximize revenue and prevent competitor access, which is nothing to do with privacy. If Microsoft wins companies will be able to use this verdict to destroy privacy for revenue gain.

carbocation 5y ago
I agree. I want to add that the headline is infuriatingly credulous of Microsoft's claims.
bingidingi 5y ago
Yes, important to note that Microsoft is going after a competitor here; there are a bunch of data brokers that have been scraping LinkedIn for years and this is the case they're going after. I hope the Supreme Court can see through this BS and tell them to go pound sand. (If Microsoft wanted to go after data brokers I'd be on board for that tho)
- nojito 5y ago
  It was hiq that sued linkedin
  - mrastro 5y ago
    After being threaten by LinkedIn: "LinkedIn told hiQ in 2017 to stop scraping LinkedIn's public profiles or face liability under the anti-hacking law." I think it's fair to say LinkedIn is the instigator in this case.
    - Causality1 5y ago
      That seems like it would have been a breathtakingly easy problem to solve using technical means instead of legal threats. They could've required logins to view profile information. They could have blacklisted any IP address exhibiting automated behavior.
      - onion2k 5y ago
        That sort of 'solution' is just a game of technical whack-a-mole as one party finds a new way to scrape the data and the other finds a new way to stop them though. If you want to permanently stop scrapping you need a non-technical solution that stops someone even trying.
        mixologic 5y ago
        But thats exactly the issue here. If they find a way to limit the data such that they can split who is "authorized" and who is not, then they have found something that falls under the now new reading of cfaa. It's not going to stop scrapers from entities outside of the jurisdiction of the US legal system, but it's going to end up icing companies in the US who do this. But I think Microsoft/Linked In is going to have a very difficult time redefining authorized when the data is publicly available to anonymous access.
TeeMassive 5y ago
And let's not forget that Microsoft probably is using that data when it comes to their own hiring decisions and recruiting efforts.
- username90 5y ago
  They already sell the data to anyone who wants to pay, they have no moral high ground here at all, the only extra "protection" you'd get is that companies would have to pay Microsoft more money to abuse your data.
jakelazaroff 5y ago
Is hiQ a competitor to LinkedIn? They scrape people’s profiles so they can snitch to employers about employees whom they think are job hunting. AFAIK, that’s not a service LinkedIn provides. Moreover, I would like it both ways. Access control would be great, but it sidesteps the underlying issue. I’m fine with my profile being public — in fact, I want it to be — but I don’t want my data to be vacuumed up and weaponized against me. I agree that anti-hacking laws are not appropriate here. But what hiQ is doing is sleazy as heck, and we should absolutely try to throw out as much bathwater as possible without throwing out the baby too.
- danuker 5y ago
  > I’m fine with my profile being public — in fact, I want it to be — but I don’t want my data to be vacuumed up and weaponized against me. LinkedIn is a authwalled garden only available to those with accounts. "We may or may not have a Jake Lazaroff. Who's asking?". Sounds like you need a personal website.
  - jjk166 5y ago
    When most people say they want things to be public, they typically mean "accessible to those who have a legitimate reason to access it, which may include some people I have not explicitly authorized." Also LinkedIn does have public profile settings which allow some or all of your profile to be visible without an account. https://www.linkedin.com/help/linkedin/answer/83/linkedin-public-profile-visibility?lang=en https://www.linkedin.com/help/linkedin/answer/83/linkedin-pu...
- mrwww 5y ago
  I believe LinkedIn started providing some services similar to HiQ, which is one of their arguments. How can your profile be public but not vaccumed up? Google vacuums it to index of course. Or perhaps you mean only public on Linkedin? What the Russians or any other bad actor you care to think about already has done with your public profile is probably way way sleazier.
deleted 5y ago
[deleted]

lizdax 5y ago

Curious to see which direction the Supreme court goes with this -- given the last CFAA ruling. >LinkedIn told the Supreme Court that hiQ's software "bots" can harvest data on a massive scale, far beyond what any individual person could do when viewing public profiles. So can anyone with a couple thousand dollars to burn. By barring US companies from doing it all you're going to do is move the work to another country that doesn't care about US Law. The data will still be scrapped nonetheless.

NaOH 5y ago
>LinkedIn told the Supreme Court that hiQ's software "bots" can harvest data on a massive scale, far beyond what any individual person could do when viewing public profiles. That doesn't seem any different than when I registered a new business in my state. I did that online, and before I received the printed, mailed confirmation from the state I was getting credit card offers in the mail addressed to the business. I'm not glad how that played out, but I don't see any difference with hiQ collecting public information.
- fncypants 5y ago
  Secretary of State / corporate information is often sold in bulk without any need for scraping without authorization. > Daily Filing Update – a file that includes all of the database updates for a specific day. Customers frequently begin by purchasing a Master Unload in order to create a database, and then subscribe to the Daily Filing Update so that they can download and update their databases on a daily basis to keep their data current with that maintained on the SOS BEST database.[0] [0] https://direct.sos.state.tx.us/help/help-corp.asp?pg=bulk https://direct.sos.state.tx.us/help/help-corp.asp?pg=bulk
cwkoss 5y ago
Typo quibble: the data will still be scraped - if it was scrapped it wouldn't be an issue!

cj 5y ago

2 questions I asked myself after reading this: 1) Do I side with LinkedIn or HiQ? 2) What case law precedent do I think we ought to set? I’m personally conflicted on which way to lean. I see pros and cons to both - this is an interesting case.

wslack 5y ago
Scraping for non-profit purposes feels fine. Scraping for commercial purposes feels dirty. I don't know if there's a way for the law to square that circle.
- postalrat 5y ago
  How about a browser extension that scrapes only the the profile you are visiting and adds that information to an ATS. Just to prevent a lot of copy paste.
- fncypants 5y ago
  If what you are feeling is that scraping for commercial purposes feels "unfair," that is because if it is not allowed it traditionally falls within a legal cause of action commonly called "unfair competition." In the same vein as trade secret theft. So the law does square that circle, but the line drawn is a fuzzy gray one. Its unfair to free ride off a competitor, but yet some kinds of actions that look like free riding are in fact allowed.
- jrochkind1 5y ago
  When Google does it, it's commercial, right? Should you need a license from all indexed content to make a search engine? Isn't indexing for search what HiQ is doing too?
cwkoss 5y ago
I feel strongly that non-commercial scraping should be protected. I think commercial scraping should probably be allowed, but there should be some sort of mandatory maximum retention limit, so deleted (or updated access controls on) content eventually ages out (or gets transferred to a non-commercial custodian for archival purposes).

kizer 5y ago

Scraping is inevitable. It seems like it exists in this grey area of a “hacky” accepted practice. I think it should be expected today. If it’s public, it’s public. Of course copyright applies, but day after tomorrow people will just run copy through a GPT-3-like scrambler or rephrasher or something and publish it anyways. The reality is that anything public on the web will be able to be harvested and made use of —- sort of how it’s been and really how it should be.

gip 5y ago

The headline is misleading. What is happening is that LinkedIn is using user data for its own benefit only and blocking innovation. As a LinkedIn user I want everyone to have access to the data I'm sharing publicly through the service. I can't wait for entrepreneurs to take on LinkedIn. There are a lot of opportunities there.

mLuby 5y ago
I'd rather entrepreneurs build on top of LinkedIn. There's so much untapped potential to build upon aggregated markets (of professionals, of jobs, of housing, of singles, of restaurants, etc) rather than building yet another aggregator. The existing aggregator cements its dominant position, can charge rent to connected apps, and attracts rather than repels killer apps. The main problem I see is fears that connected apps will siphon away the aggregator's data, and/or combine that data with external data; both are attempts to supplant the aggregator. Maybe this fear can be assuaged by technical or legal means.
- jb775 5y ago
  What do you do when you build an entire business on top of LinkedIn, then they decide to completely revoke your access? We need DE-centralization, so no-one in the world has this type of power over anyone.
  - gip 5y ago
    Totally agree. About 10 years ago I was the first hire of a startup built partially on top of LinkedIn. A Monday we got a cease and desist letter. We had to immeditely stop using their API and had to start crawling the data instead. That almost killed us.
    - altdataseller 5y ago
      What was this startup?

altdataseller 5y ago

If they do rule HiQ labs unable to scrape, can they also make sure ZoomInfo stops scraping LinkedIn profiles too? I have seen people’s profiles scraped word for word and posted on ZoomInfo. And sold to salespeople who cold call you. Let put an end to this.

TechBro8615 5y ago

Any ideas how Microsoft wants to frame this so it becomes a partisan issue? Microsoft Legal can’t change the facts of the case, but presumably Microsoft PR wants to shape the narrative into one agreeable to the conservative majority on the court. I have a feeling that regardless of the specifics, it will be a narrative where the little people lose and the big people get to keep playing. I would love to be proven wrong though. If I’m right, it’s a stark reminder that Microsoft need only sustain its open-source friendly, developer-empowering persona as long as it’s convenient and rational. When there’s territory at stake, that veneer of friendliness begins to crack.

rurabe 5y ago

This is a pretty bad headline. I don't know that i would characterize this as revived. The same 9th circuit who held last year that LinkedIn could not block hiQ from scraping public data, just got asked to reconsider the same case, except now there is additional precedent that SCOTUS says if you had permission to access the computer then it's not a violation of the CFAA (even if you are a shady corrupt cop). Hard to see this turning out any other way than the 9th circuit reaffirming their decision (or even strengthening it) and then it's up to LinkedIn to try SCOTUS again

Goety 5y ago
I disagree with that SCOTUS decision. It completely obliterated CFAA. Imagine if they said nurses/doctors could do that with their terminals and it didn't violate HIPAA. I will say there is a ridiculous amount of redtape around law enforcement using data. Loopholes with third party access is already something that exist. So if it's above board monitoring would be easier... But I'm not sure we have adequate monitoring let alone enforcement now. I feel this is a weak case to attack 3rd party data scrapers/brokers. The public generally recognizes the monster we created by having life changing data accessible to anyone with $50 and a bank account. I want to side with LinkedIn but realistically I'm becoming more and more jaded on the concept of open internet and iot of everything. I hate the alternative of an open internet worse. I would love to restrict data scrapers but at the same time should we restrict who has that data? I'd rather we shift how we use the internet and socially enforce boundaries on companies. I cant even open my fridge, use my microwave, stove without it being logged either by the electric company or bluetooth enabled appliance with TV and wifi temperature control software company where you hope an update doesnt brick the appliance. There is no way in my mind that data helps the consumer. It might help companies maximize profit but at what energy consumption/cost to the environment?
- nulbyte 5y ago
  > I disagree with that SCOTUS decision. It completely obliterated CFAA. Imagine if they said nurses/doctors could do that with their terminals and it didn't violate HIPAA. I don't see the similarity between CFAA and HIPAA, here, and SCOTUS didn't obliterate the CFAA. Theybsimply said, if you are authorized to use a system, your use of the system isn't unauthorized. That's fairly straightforward. HIPAA, on the other hand, regulates disclosure of specific data. You can violate HIPAA even if you are authorized to use a system that holds covered data.

rurabe 5y ago

Note that LinkedIn could make this whole issue moot by putting their data behind a login tomorrow. The 9th circuit decision pretty clearly stated that. The only reason the data is public is for marketing purposes, to sign up more users.

bryan_w 5y ago
I thought they were prevented from doing that (at least at some point, by injunction)
- mcny 5y ago
  Each Linked In user has the option to make as much or as little of their information public. I don’t see how any injunction can stop that. None of this data belongs to LinkedIn. It nudges us to make more of our data public. I don’t even want to get into what goes on in the mind of a Linked In “influencer”. I am convinced salvation cannot come from the courts and at the same time Congress has ceded all authority to the White House. The goal for everyone in our industry must be complete abolition of the CFAA but that can’t happen until we can get Congress to act.
  - rurabe 5y ago
    Yeah each user gets the choice right now as a consequence of LinkedIn's product design. As far as I know, there's no reason it couldn't be "Log in to see this profile". I believe the injunction bars LinkedIn from blocking hiQ specifically from accessing the publicly available information (which again is public by LinkedIn's choice). They made a point to draw a distinction between the public stuff and stuff behind a login/password.
ID1452319 5y ago
I want some of my LinkedIn profile to be public. I DON'T want any random company scraping that PII data, storing it, processing it and making money from it. Is that too much to ask?
- 5y ago

jollybean 5y ago

You can walk around the store and 'see' whatever you want, but if you start recording prices and taking pictures, they may ask you to leave. I wonder if there's some relevant precedent there.

ineedasername 5y ago

Doesn't Google pretty much already scrape all of LinkedIn and partially republish in search results (and cached pages)? Microsoft doesn't have a problem with that.

durnygbur 5y ago
You've got to be in the cartel.

cletus 5y ago

The Supreme Court curtailed the broad scope of the CFAA recently in the van Buren case. I personally agree with that decision (and also find it funny as it continues the long trend of Clarence Thomas being on the wrong side of history). While van Buren's actions were obviously problematic he was an authorized user of the computer system. The issue is the vagueness of the CFAA has been a prosecutor's wet dream and a predictable source of overreach. Case in point: Aaron Swartz. So here's what's interesting: by taking up this case SCOTUS is potentially going in the other direction. The Appeals Court held that the CFAA didn't apply, allowing hiQ to continue. And here's where (in my layman's view) SCOTUS may choose to act: The Appeals Court stopped LinkedIn taking action to impede hiQ's access. I personally view this as overreach. There's a difference between not "hacking (in CFAA) terms a website and blocking the information provider impeding bots. As much as Microsoft/LinkedIn is a nightmare of data misuse and dark patterns (eg to obtain contacts), I think it would be a bad decision to let the current hiQ ruling stand. Disallowing sites from taking action to block scrapers from systematically taking all that information and building a competitor seems like a bad idea. But given the van Buren ruling, it seems unlikely SCOTUS will reverse course and declare hiQ's actions "hacking" (in CFAA terms). I suspect they'll curtail the remedy.

dannyw 5y ago
My counterpoint: companies shouldn't get to have their cake and eat it too, as in offering public content, but then preventing competitors from using that public content. To use another example, what if Google+ was a success, and Google blocks every other competing search engine from indexing even public posts on Google+? Is that a good outcome? There are already copyright protections to stop Shutterstock from ripping, say, all of Getty Image's photos. What we are talking about here is information so banal, so unoriginal, so uncreative, that copyright doesn't even apply.
- gwoplock 5y ago
  > My counterpoint: companies shouldn't get to have their cake and eat it too, as in offering public content, but then preventing competitors from using that public content. I agree this case is taking an anti-competitive action and spinning it into a pro-privacy case. However, what's the line? Personally I believe if you apply the rules equally to competitors and non-competitors alike you can restrict programmatic access to public data. Since LinkedIn isn't doing that, HiQ shouldn't be restricted. In any case I think this is a poor use of the CFAA.

wly_cdgr 5y ago

I gotta say, making scraping illegal would be a great mercy towards web developers. It's just the worst

gnicholas 5y ago

I had a strange experience with LinkedIn today: I had to restart my computer, and when my browser started it re-opened all my tabs. This included about 50 LinkedIn tabs, which I opened sometime over the last 3 months (most of them were opened yesterday). LinkedIn logged me out and accused me of unauthorized use or some such thing, and then locked me out for a period. When I submitted a help request and explained what had happened (browser reopened tabs), they gave a boilerplate answer that was totally non responsive. I understand they want to defeat scrapers (whether I think they should or not), but outcomes like this are a pretty annoying way to do it. Can they really not tell when a browser is restarting and opening a bunch of tabs that the user has already viewed (and which are all 1st or 2nd degree contacts)? edit: added context around when tabs were opened, in response to a comment.

Judgmentality 5y ago
Just curious, how many tabs do you normally have open?
- gnicholas 5y ago
  Maybe 100? I use a Chromium-compatible version of the Tree-Style-Tabs extension, which makes it easy to organize and navigate dozens of tabs.
bpodgursky 5y ago
It seems possible, but it also seems like a sorta painful problem to solve if locking an account requires dredging through 3+ months of a user's interactions. At a lot of companies that kind of historical data ends up getting batched and stored in some kind of timeseries datastore (aka S3) and then batch-operated on via Spark, MapReduce, Flink etc in hourly or daily jobs. So instead of implementing account locking and scraping detection at the realtime layer looking at current requests, you have to write some ugly daily aggregate & backfill job, which inevitably ends up super expensive and slow. Anyway, yes, but I can definitely sympathize with _not_ doing it this way.
- gnicholas 5y ago

jb775 5y ago

What a joke that the courts within our society are so obviously controlled by big business. LinkedIn is essentially asking for legal protection so they can box out competition as they pimp out user data they don't even really own in the first place. The argument our society should be discussing is that individuals own the underlying data. All previous "user agreements" are null and void. All new user accounts should be bundled like non-fungible tokens, then every piece of data associated with that account can be tracked to compensate that user based on actual impressions. Companies can take their cut, but if a user isn't happy with the setup, they can revoke all access to their own data. Enough of this "you agreed to our service agreement" smoke and mirror bullshit...that's like saying the water company owns the water itself.

pverghese 5y ago
By that same token, all use of free services should be charged from the time of use of said services, since previous agreements are nill and void. By using said services you have provided your information given the understanding that your information will be used by the companies that provide the services
6gvONxR4sf7o 5y ago
On the other hand, I want to use linkedin without my data being taken by other companies. It’s my data, not linkedin’s and certainly not hiq’s.

durnygbur 5y ago

> hiQ Labs So the recent LinkedIn "data leak" are public profiles scrapped by hiQ from LinkedIn and then "leaked" again from hiQ? or hiQ is selling the data dumps undercover? Either way hard to pity MS and LI, the website is a Potemkin village...

ramoq 5y ago

you have to watch the video of the actual court proceedings. a core issue is that LinkedIn knew hIQ was scraping their website and allowed them to do this until they started launching similar features. hIQ has a strong argument around this point and evidence proving so. video: https://youtu.be/tvLdJujOp8k https://youtu.be/tvLdJujOp8k

myfavoritedog 5y ago

For its part, hiQ uses the data for products that analyze employee skills or alert employers when they could be looking for a new job. Interesting that they've made a business out of this. I've always maintained to my colleagues that when I see a bunch of LinkedIn activity from them, I can tell that they're putting themselves on the job market.

myfavoritedog 5y ago

Apart from the CFAA issue, it seems strange that you can't regulate access patterns to your own web site in order to prevent mass scraping of user data.

failwhaleshark 5y ago

LinkedIn had a well-known profile viewing crack for years. Add this magic to the URL and poof, you don't need to login or be connected to them. It wasn't even close to a "hack" how trivial it was to bypass LinkedIn's (not) security.

ggggtez 5y ago

> public The entire case is just about that one word. If the profile is public, then you are authorized. We already went through this. CFAA is not a privacy law.

rhacker 5y ago

It seems absolutely nuts to me that this case ended where it did: https://en.wikipedia.org/wiki/Spokeo,_Inc._v._Robins https://en.wikipedia.org/wiki/Spokeo,_Inc._v._Robins And we're having a discussion about linkedin might possibly lose. In the spokeo case it's data that Robins said was untrue about him and Spokeo was allowed to keep sharing it - simply because he couldn't prove harm. Which, is laughable in TODAY's political climate where harm is waking up in the morning - because of all the data sharing happening everywhere and being unable to stop it.

MR4D 5y ago

To me, this is simple: If the pages are not behind a login, then they are PUBLIC, and open to any sort of data gathering. If the pages are behind a login, then they are PRIVATE, and have all the protections alongside it. Frankly, this bullcrap about "it's not public, but we want to leverage the usage of the open web as it suits us" to build our monopoly crap is getting old.[0] [0] - I don't care who the company is, whether Microsoft, Apple, Amazon, Twitter, Facebook, Google, or some no-name company