4 ms·

I think the two main questions raised in this case: 1. Is it always ok for someone to build a bot to do something which can be legally done by hand? Example: B

by whack 9y ago

I think the two main questions raised in this case: 1. Is it always ok for someone to build a bot to do something which can be legally done by hand? Example: Building a LinkedIn scraper that tracks all public data. Or using a GPS tracker to track a car, instead of manually following it 2. Is it anti-competitive practice, and a violation of anti-trust laws, for LinkedIn to allow the general public, and other companies like Google, access to its public data, but ban others such as HiQ? On question 1, I tend to lean towards LinkedIn's position. Just because something can be legally done by hand, shouldn't automatically mean that we should allow it to be done at massive scale by automated scripts. I wouldn't want companies having the right to surveil the movements of every citizen 24/7, just because they have the right to follow someone on foot, and I think a similar argument can be made against HiQ. On question 2 though, I agree with HiQ. LinkedIn's attempt to ban HiQ doesn't seem like an attempt to protect their users, but rather, an anti-competitive attempt to kill off a potential competitor, and secure the market for themselves.

leggomylibro 9y ago
>Is it always ok for someone to build a bot to do something which can be legally done by hand? Funny that you put it that way; when computers were in their infancy, people would scoff along the lines of, "...and they spent all that money/time teaching the damned thing to do something that you could get anyone off the street to do for $0.50/hr..." Of course, before computers were invented in the first place 'computer' was a job description for a person who computes. How things change in...what, 50-80 years?
lovich 9y ago
Haven't the courts already decided that if things can be done by hand then they can be automated? For example license plate scanners were ruled legal because officers can view that information in public, but the automation allows the police to follow everyones movements with only a few machines set up. This ship has sailed
- epicide 9y ago
  You're probably correct in how it will turn out, but laws can change. One precedent does not mean nothing will ever break it. It does make it harder, though.
- mtgx 9y ago
  On the other hand the Supreme Court ruled 9-0 that "just because law enforcement can follow someone manually" doesn't mean it can also put a GPS tracker on whoever it wants, without a warrant. https://www.reuters.com/article/us-usa-police-gps/supreme-court-rules-police-need-warrant-for-gps-tracking-idUSTRE80M1E120120124 https://www.reuters.com/article/us-usa-police-gps/supreme-co... I think we as the society are ultimately responsible for setting certain "rights" in stone. For instance, should the government be able to record everything anyone says "in a public place", even while on the phone or having a "private conversation" (but in public) with a friend, through highly advanced CCTV cameras? Sure you could argue that "because it's a public place, then yes, we can conclude that the government should be allowed to do that". But that doesn't necessarily have to end the discussion. We, the society, can decide that "HELL NO, that's not acceptable, and we'll put anyone who tries to do that in prison."
- matt4077 9y ago
  Automated license plate scanning is probably legal for local municipalities (Neil v Fairfax County, SC appeal pending). That doesn't mean it's legal for private entities to do so, nor is necessarily legal for such data to be, for example, aggregated nation-wide. See https://www.theatlantic.com/politics/archive/2014/02/mass-surveillance-of-all-car-trips-is-nearly-upon-us/283922/ https://www.theatlantic.com/politics/archive/2014/02/mass-su... for Conor Friedersdorf's article which is attributed with bringing down the last attempt for a "Homeland Security" database of where you were last summer.
  - mjevans 9y ago
    The correct solution for this concern is to make it legal to obscure the IDENTIFYING part of the license plates (but require that state and 'tabs' are still exposed) while it is parked. Moving the obscurity would constitute modification of private property and should require either a valid documented probable cause or a warrant for the search.
derefr 9y ago
> I wouldn't want companies having the right to surveil the movements of every citizen 24/7, just because they have the right to follow someone on foot I feel like there's an inherent contradiction there, and the right thing to do isn't to "lean into" the contradiction, but rather to resolve it in the other direction. That is: we probably need to change what is legal for private individuals to do, if we want to effect change in what is legal for automation to do. After all, at its most basic, we've got "automation" like Mechanical Turk, or people who you can hire to stand in a line for you to buy a new iPhone. Any law against automation won't work out if exceptions like that still exist; and those exceptions can't be stopped except by changing what deals it is legal for a human to make.
- coldtea 9y ago
  >I feel like there's an inherent contradiction there, and the right thing to do isn't to "lean into" the contradiction, but rather to resolve it in the other direction. That is: we probably need to change what is legal for private individuals to do, if we want to effect change in what is legal for automation to do. It can't be solved this way, because somethings can not be made illegal or if they can, they cannot be enforced at the individual doing it manually level. Some individual can always sit at a street corner and read the license plates of the cars that pass. How one can make that illegal? But systematically mass-collecting such data could easily be made illegal. >After all, at its most basic, we've got "automation" like Mechanical Turk, or people who you can hire to stand in a line for you to buy a new iPhone. That still has costs and limitations -- so it's nowhere near any competition for automation. I'd say making their automated version illegal is a good starting point.
  - derefr 9y ago
    > Some individual can always sit at a street corner and read the license plates of the cars that pass. How one can make that illegal? Have you heard of conspiracy laws? We disincentivize gangs from using drug mules, by making it hard for them to find willing drug mules, by making it illegal to be a willing drug mule—the drug mule will, if they were complicit in the act, be charged with conspiracy to commit a felony. This (apparently) works to decrease the prevalence of drug-muling, even though it's very hard to detect drug mules (which is the whole point of drug mules.) The law only really gets applied when you end up finding a mule through some other investigation (i.e. busting the gang itself.) But that still happens often enough to scare all the other potential willing drug mules. This is how I'd see individuals sitting on street-corners counting license plates being charged: not for doing anything that is illegal prima facie, but rather for their willing complicity in a conspiracy to commit the novel crime of, say, building a database through espionage/surveillance without a license. In other words: if the company's business model is based on a crime, then they're a criminal organization; and it's illegal to profit from dealings with a criminal organization, so you're doing something illegal by doing what they say, even if the thing they're asking you to do isn't illegal.
coldtea 9y ago
>1. Is it always ok for someone to build a bot to do something which can be legally done by hand? I'd say no. Technology is a multiplier, and if the same thing that took painstaking work and devoting resources to be done manually can be done automatically, it can be the difference between a democracy and a police state. (E.g. the police targeting some suspects by tailing them and have some people listen to their conversations, and everybody in the country monitored 24/7). The kind of crappy arguments usually taken as OK in courts however see this otherwise, as if 1000x automated something is the same as 1x.
pishpash 9y ago
Yes, the two questions are in fact quite orthogonal. - Should data truly intended to be public be scrapable by HiQ (or anyone)? The answer has to be yes. - Should you be able to use "public" data to seek out essentially private information behind the user's back (like what HiQ is doing)? The answer has to be no. On this latter question, there is a notion of continuous consent and discoverability. There should be a feedback trail all the way back to the original provider of data (the user) about who is accessing the data and for what purpose, even if (or especially because) it is public; and this fact should be made known to the accessor. There has to be a semblance of symmetry in how a public exchange of information should take place. Both the provider and the accessor should be in the open if it is truly public. Then the user can take actions to affirmatively provide continuous consent or withdraw.
dalbasal 9y ago
Well... 2 could easily be both, or even just what linkedin said. If HR gets an alert when you start sniffing around for new opportunities, people will sniff elsewhere. Linkedin don't want that and (coincidentally) neither do users. Either way, I don't think it's the interesting question. On question 1, I think you are really on the money. Concise. We're used to thinking in terms of principles when it comes to our principles, especially laws. Realistically though, our moral codes are not like physics. They're approximations that seem to work for the most part, for the situations we know about. Being a somewhat anal retentive species, we don't generally like this. It feels like the principles are unsound.
kelnos 9y ago
As a LinkedIn user, though, I don't want HiQ to have access to my data. I'm certainly not going to claim LinkedIn is an angel of a company and has its users' best interests at heart, but its users voluntarily gave LI their information with the expectation that it would be used according to LI's privacy policy and ToS. If that doesn't include "allowing 3rd parties to scrape the site without LI's or the user's consent", then under what grounds does HiQ believe that they have the legal right to use and transform my data for commercial purposes?
skybrian 9y ago
Traditionally this is controlled by robots.txt, which does let you control which search engines can scrape your site. It's not enforceable, but generally considered fair play to respect it. I think it's well within a website's rights to control who scrapes them. The problem with LinkedIn (and other social media sites) is that they don't let their users control this, in turn. It should be up to each user to decide whether they want to appear in search engines, and which ones. It's pretty bizarre that this was never brought up in the article. The idea that users should control their own information was entirely ignored.
cvsh 9y ago
>1. Is it always ok for someone to build a bot to do something which can be legally done by hand? Example: Building a LinkedIn scraper that tracks all public data. Or using a GPS tracker to track a car, instead of manually following it This is a thorny question. What if I build a scraper that requires me to press a button in an app to actually run a scrape? Is that "by hand"? This seems hard to effectively regulate without sliding into a criminalization of a lot of benign things programmers do on the web.

by whack 9y ago

leggomylibro 9y ago

>Is it always ok for someone to build a bot to do something which can be legally done by hand? Funny that you put it that way; when computers were in their infancy, people would scoff along the lines of, "...and they spent all that money/time teaching the damned thing to do something that you could get anyone off the street to do for $0.50/hr..." Of course, before computers were invented in the first place 'computer' was a job description for a person who computes. How things change in...what, 50-80 years?

lovich 9y ago

Haven't the courts already decided that if things can be done by hand then they can be automated? For example license plate scanners were ruled legal because officers can view that information in public, but the automation allows the police to follow everyones movements with only a few machines set up. This ship has sailed

epicide 9y ago
You're probably correct in how it will turn out, but laws can change. One precedent does not mean nothing will ever break it. It does make it harder, though.
mtgx 9y ago
On the other hand the Supreme Court ruled 9-0 that "just because law enforcement can follow someone manually" doesn't mean it can also put a GPS tracker on whoever it wants, without a warrant. https://www.reuters.com/article/us-usa-police-gps/supreme-court-rules-police-need-warrant-for-gps-tracking-idUSTRE80M1E120120124 https://www.reuters.com/article/us-usa-police-gps/supreme-co... I think we as the society are ultimately responsible for setting certain "rights" in stone. For instance, should the government be able to record everything anyone says "in a public place", even while on the phone or having a "private conversation" (but in public) with a friend, through highly advanced CCTV cameras? Sure you could argue that "because it's a public place, then yes, we can conclude that the government should be allowed to do that". But that doesn't necessarily have to end the discussion. We, the society, can decide that "HELL NO, that's not acceptable, and we'll put anyone who tries to do that in prison."
matt4077 9y ago
Automated license plate scanning is probably legal for local municipalities (Neil v Fairfax County, SC appeal pending). That doesn't mean it's legal for private entities to do so, nor is necessarily legal for such data to be, for example, aggregated nation-wide. See https://www.theatlantic.com/politics/archive/2014/02/mass-surveillance-of-all-car-trips-is-nearly-upon-us/283922/ https://www.theatlantic.com/politics/archive/2014/02/mass-su... for Conor Friedersdorf's article which is attributed with bringing down the last attempt for a "Homeland Security" database of where you were last summer.
- mjevans 9y ago
  The correct solution for this concern is to make it legal to obscure the IDENTIFYING part of the license plates (but require that state and 'tabs' are still exposed) while it is parked. Moving the obscurity would constitute modification of private property and should require either a valid documented probable cause or a warrant for the search.

derefr 9y ago

> I wouldn't want companies having the right to surveil the movements of every citizen 24/7, just because they have the right to follow someone on foot I feel like there's an inherent contradiction there, and the right thing to do isn't to "lean into" the contradiction, but rather to resolve it in the other direction. That is: we probably need to change what is legal for private individuals to do, if we want to effect change in what is legal for automation to do. After all, at its most basic, we've got "automation" like Mechanical Turk, or people who you can hire to stand in a line for you to buy a new iPhone. Any law against automation won't work out if exceptions like that still exist; and those exceptions can't be stopped except by changing what deals it is legal for a human to make.

coldtea 9y ago
>I feel like there's an inherent contradiction there, and the right thing to do isn't to "lean into" the contradiction, but rather to resolve it in the other direction. That is: we probably need to change what is legal for private individuals to do, if we want to effect change in what is legal for automation to do. It can't be solved this way, because somethings can not be made illegal or if they can, they cannot be enforced at the individual doing it manually level. Some individual can always sit at a street corner and read the license plates of the cars that pass. How one can make that illegal? But systematically mass-collecting such data could easily be made illegal. >After all, at its most basic, we've got "automation" like Mechanical Turk, or people who you can hire to stand in a line for you to buy a new iPhone. That still has costs and limitations -- so it's nowhere near any competition for automation. I'd say making their automated version illegal is a good starting point.
- derefr 9y ago
  > Some individual can always sit at a street corner and read the license plates of the cars that pass. How one can make that illegal? Have you heard of conspiracy laws? We disincentivize gangs from using drug mules, by making it hard for them to find willing drug mules, by making it illegal to be a willing drug mule—the drug mule will, if they were complicit in the act, be charged with conspiracy to commit a felony. This (apparently) works to decrease the prevalence of drug-muling, even though it's very hard to detect drug mules (which is the whole point of drug mules.) The law only really gets applied when you end up finding a mule through some other investigation (i.e. busting the gang itself.) But that still happens often enough to scare all the other potential willing drug mules. This is how I'd see individuals sitting on street-corners counting license plates being charged: not for doing anything that is illegal prima facie, but rather for their willing complicity in a conspiracy to commit the novel crime of, say, building a database through espionage/surveillance without a license. In other words: if the company's business model is based on a crime, then they're a criminal organization; and it's illegal to profit from dealings with a criminal organization, so you're doing something illegal by doing what they say, even if the thing they're asking you to do isn't illegal.

coldtea 9y ago

>1. Is it always ok for someone to build a bot to do something which can be legally done by hand? I'd say no. Technology is a multiplier, and if the same thing that took painstaking work and devoting resources to be done manually can be done automatically, it can be the difference between a democracy and a police state. (E.g. the police targeting some suspects by tailing them and have some people listen to their conversations, and everybody in the country monitored 24/7). The kind of crappy arguments usually taken as OK in courts however see this otherwise, as if 1000x automated something is the same as 1x.

pishpash 9y ago

Yes, the two questions are in fact quite orthogonal. - Should data truly intended to be public be scrapable by HiQ (or anyone)? The answer has to be yes. - Should you be able to use "public" data to seek out essentially private information behind the user's back (like what HiQ is doing)? The answer has to be no. On this latter question, there is a notion of continuous consent and discoverability. There should be a feedback trail all the way back to the original provider of data (the user) about who is accessing the data and for what purpose, even if (or especially because) it is public; and this fact should be made known to the accessor. There has to be a semblance of symmetry in how a public exchange of information should take place. Both the provider and the accessor should be in the open if it is truly public. Then the user can take actions to affirmatively provide continuous consent or withdraw.

dalbasal 9y ago

Well... 2 could easily be both, or even just what linkedin said. If HR gets an alert when you start sniffing around for new opportunities, people will sniff elsewhere. Linkedin don't want that and (coincidentally) neither do users. Either way, I don't think it's the interesting question. On question 1, I think you are really on the money. Concise. We're used to thinking in terms of principles when it comes to our principles, especially laws. Realistically though, our moral codes are not like physics. They're approximations that seem to work for the most part, for the situations we know about. Being a somewhat anal retentive species, we don't generally like this. It feels like the principles are unsound.

kelnos 9y ago

As a LinkedIn user, though, I don't want HiQ to have access to my data. I'm certainly not going to claim LinkedIn is an angel of a company and has its users' best interests at heart, but its users voluntarily gave LI their information with the expectation that it would be used according to LI's privacy policy and ToS. If that doesn't include "allowing 3rd parties to scrape the site without LI's or the user's consent", then under what grounds does HiQ believe that they have the legal right to use and transform my data for commercial purposes?

skybrian 9y ago

Traditionally this is controlled by robots.txt, which does let you control which search engines can scrape your site. It's not enforceable, but generally considered fair play to respect it. I think it's well within a website's rights to control who scrapes them. The problem with LinkedIn (and other social media sites) is that they don't let their users control this, in turn. It should be up to each user to decide whether they want to appear in search engines, and which ones. It's pretty bizarre that this was never brought up in the article. The idea that users should control their own information was entirely ignored.

cvsh 9y ago

>1. Is it always ok for someone to build a bot to do something which can be legally done by hand? Example: Building a LinkedIn scraper that tracks all public data. Or using a GPS tracker to track a car, instead of manually following it This is a thorny question. What if I build a scraper that requires me to press a button in an app to actually run a scrape? Is that "by hand"? This seems hard to effectively regulate without sliding into a criminalization of a lot of benign things programmers do on the web.