5 ms·

The perceived hypocrisy sort of goes away when you stop thinking about it as a collaboration or a community of equals, and instead think of it as a competition,

The perceived hypocrisy sort of goes away when you stop thinking about it as a collaboration or a community of equals, and instead think of it as a competition, which is what it is. You would not say of a football team "oh, it's okay for you to try to score a goal on me, but if I try to score a goal on you, suddenly you're blocking the ball?!" Naturally, they're going to say "web scraping uses resources, stop it!" but then keep web scraping in the background. To be clear, it's bad behavior, it's just not hypocritical behavior, as it's completely in keeping with what amoral corporations locked in constant battle would be expected to do: maximize benefits to themselves while minimizing benefits to others.

Kareem71 3y ago
The problem is as this article points out is democratically elected courts should not be choosing winners in a capitalistic competition
- karaterobot 3y ago
  Only commenting on how we should expect corporations to act, or more accurately why we should not be surprised at their behavior.
backtoyoujim 3y ago
There are clearly more than two teams in these issues. It is not a game, and it is not football. It is a public policy issue also outpaces "competition" which is merely a subject change.
- karaterobot 3y ago
  Football is metaphorical in this case.
philipov 3y ago
Hypocrisy doesn't require one to believe what they say and utter it in good faith but fall short of those ideals in practice. Equivocating about football teams doesn't change that one is trying to impose standards on another without holding oneself equally to them. It is still hypocrisy, even if they do it amorally in bad faith. Especially then. What matters is what policy you espouse; you don't get a pass for not really believing what you say. The implication is that hypocrites are acting in bad faith.
gnomewascool 3y ago
That's a very interesting comparison (thanks!), but I'm not sure if it's the correct framing. Making scraping technically difficult would be equivalent to trying to score a goal (so still not great, for the rest of the world, but probably not hypocritical). Trying to prevent certain classes of behaviours via legal means is more like trying to prevent certain types of play, by appealing to the referee, while still doing them yourself. Clearly, this often does happen in sports, but _is_ generally seen as hypocritical.
- jrm4 3y ago
  For quite sometime I've always felt that "sports analogies" are overwhelmingly the BEST way to frame most microeconomic disputes. Much better than the far inferior Darwin-esque "Survival of the fittest" metaphors that imply some natural order to certain types of greed and bad behavior. There's NOTHING natural about our economic systems. They're all COMPLETELY made up, let's treat them that way. (and yes, here it is about 'lobbying the ref')
  - kubanczyk 3y ago
    Sport in general is a cultural phenomenon and it seems that all cultural phenomena share a lot of similarities. Genetics however is not only a useful model, it's hard science. You can experimentally find out whether some characteristic is e.g. Mendelian (I'd doubt the greed is, as normally defined). It got me thinking that to cross the two domains, there is also a meta-concept of cultural viruses ("memes") to which Dawkins applied Darwinian model. Definitely not hard science, but they kind of counter your point that "there's NOTHING natural about our economic systems".
    - jrm4 3y ago
      I mean, it's true that you might see phenomena that echo natural systems or things, but I suppose what I'm getting at is: unlike natural systems, the "rules" can be changed. Honestly, what helped me a lot is: Economic systems are more like video games than "nature." Sure, video games can look like nature, but also are extremely malleable.
jjoonathan 3y ago
> football team In football, the rules have been extensively tuned to promote a fair fight. Perhaps we should do a bit more of that sort of thing in corporate law.
hattmall 3y ago
As the article states the issue is with courts not companies. We need a state actor to pass a law similar to the weapons laws of other states that guarantee a right to scrape. Then all the scraping companies setup shop in that state. If a service doesn't want their data scraped they need to make sure that it doesn't get sent into that state. Ideally a large enough state that companies wouldn't want to block.
- nosecreek 3y ago
  Agreed that legal clarity is important - especially for smaller players. I've built a significant hobby site that relies fairly heavily on scraping (grocery price comparison site). I believe what I'm doing is morally okay, and also that big players wouldn't run into any issues, but when it's just me (or even if it was a small company) the legal 'grey area' makes it a much bigger risk.
- tomcam 3y ago
  I like what you’re saying, but how do you provide for the existence of evil or simply incompetent scrapers who drag the system down due to incompetence?
autoexec 3y ago
> Naturally, they're going to say "web scraping uses resources, stop it!" that's the expected cost of publishing something to the public internet. People are going to access it. No one has a right to complain when people access something that was put there for the public to see. Scrappers can be dicks about it too, they can get lazy and endlessly hammer away at some server or repeatedly pull down the same content because they messed up, but we don't need need litigation for that. If something raises to the level of DoS that's already covered under existing laws. > it's completely in keeping with what amoral corporations locked in constant battle would be expected to do: maximize benefits to themselves while minimizing benefits to others. Maybe we need to rethink giving some of these corporations the privilege of corporate personhood if they are just going to make things worse for everyone else while only enriching themselves. We don't need to allow parasites and pillagers to take whatever they want at our expense.
- mrkeen 3y ago
  > Scrappers can be dicks about it too It's not always about individual bad actors. You can have lots of small players causing problems. I wonder how many python developers there are right now trying to make their own offline copy of stackoverflow.com. Wikipedia has a great defence against this. They ask you not to scrape, and at the same time, provide torrents of the data (https://meta.wikimedia.org/wiki/Data_dump_torrents https://meta.wikimedia.org/wiki/Data_dump_torrents)
  - rzzzt 3y ago
    Stack Exchange also provides one: https://archive.org/details/stackexchange https://archive.org/details/stackexchange There was a hiccup around June but that seems resolved now: https://meta.stackexchange.com/questions/389922/june-2023-data-dump-is-missing https://meta.stackexchange.com/questions/389922/june-2023-da...
    - genewitch 3y ago
      yeah i thought this was the case. There's a couple of other sites that - i hesitate to state this completely factual phrase - leave dumps for the public to pick up.
runesofdoom 3y ago
The problem with that sort "that's what amoral corporations do" reasoning is that corporations are permitted to exist because of the idea that they do contribute to the net public good. Once that idea is out the window, then there's no reason for society not to treat corporations as the hungry Lovecraftian nightmares they are and obliterate them with fire and steamship.
KieranMac 3y ago
I think the difference is that defeating the other team is the point of sports, whereas at least ostensibly the law is supposed to provide a set of coherent rules for businesses to compete against each other. Trademarks are defined according to certain legal rules, and if you have one, this is how they provide you with a limited monopoly in a certain context. Allowing businesses to define property law through contracts lets people define the rules however they want. And that leads to irrational results.
KirillPanov 3y ago
> You would not say of a football team You also would not let one football team buy up 60% of the other football teams in the world and merge them into one MEGA-TEAM. But apparently we still cannot summon the willpower to enforce simple and straightforward 100+-year-old antitrust laws (so instead we make new ones and just won't enforce those either).
- vkou 3y ago
  > You also would not let one football team buy up 60% of the other football teams in the world and merge them into one MEGA-TEAM. Why not? If that mega-team wants to destroy their revenue stream (People want to watch competitive football, as opposed to curb-stomps), they are free to do that.
  - sobkas 3y ago
    And you believe that MEGA-TEAM won't get bailed out? Because otherwise bloated corpse of MEGA-TEAM will smother football as we know it?
    - vkou 3y ago
      What do you mean bailed-out? The whole point of competitive sports is for there to be, well, meaningful competition. All the money in that industry only exists if there's meaningful chances for either team to win. Winning too hard is a detriment to the winner. That is incredibly different from capitalism, where the whole point of it is for one of the participants to win the competition, to the deteriment of the rest of us, which is why society comes up with elaborate rules for preventing that.

Kareem71 3y ago

The problem is as this article points out is democratically elected courts should not be choosing winners in a capitalistic competition

karaterobot 3y ago
Only commenting on how we should expect corporations to act, or more accurately why we should not be surprised at their behavior.

backtoyoujim 3y ago

There are clearly more than two teams in these issues. It is not a game, and it is not football. It is a public policy issue also outpaces "competition" which is merely a subject change.

karaterobot 3y ago
Football is metaphorical in this case.

philipov 3y ago

Hypocrisy doesn't require one to believe what they say and utter it in good faith but fall short of those ideals in practice. Equivocating about football teams doesn't change that one is trying to impose standards on another without holding oneself equally to them. It is still hypocrisy, even if they do it amorally in bad faith. Especially then. What matters is what policy you espouse; you don't get a pass for not really believing what you say. The implication is that hypocrites are acting in bad faith.

gnomewascool 3y ago

That's a very interesting comparison (thanks!), but I'm not sure if it's the correct framing. Making scraping technically difficult would be equivalent to trying to score a goal (so still not great, for the rest of the world, but probably not hypocritical). Trying to prevent certain classes of behaviours via legal means is more like trying to prevent certain types of play, by appealing to the referee, while still doing them yourself. Clearly, this often does happen in sports, but _is_ generally seen as hypocritical.

jrm4 3y ago
For quite sometime I've always felt that "sports analogies" are overwhelmingly the BEST way to frame most microeconomic disputes. Much better than the far inferior Darwin-esque "Survival of the fittest" metaphors that imply some natural order to certain types of greed and bad behavior. There's NOTHING natural about our economic systems. They're all COMPLETELY made up, let's treat them that way. (and yes, here it is about 'lobbying the ref')
- kubanczyk 3y ago
  Sport in general is a cultural phenomenon and it seems that all cultural phenomena share a lot of similarities. Genetics however is not only a useful model, it's hard science. You can experimentally find out whether some characteristic is e.g. Mendelian (I'd doubt the greed is, as normally defined). It got me thinking that to cross the two domains, there is also a meta-concept of cultural viruses ("memes") to which Dawkins applied Darwinian model. Definitely not hard science, but they kind of counter your point that "there's NOTHING natural about our economic systems".
  - jrm4 3y ago
    I mean, it's true that you might see phenomena that echo natural systems or things, but I suppose what I'm getting at is: unlike natural systems, the "rules" can be changed. Honestly, what helped me a lot is: Economic systems are more like video games than "nature." Sure, video games can look like nature, but also are extremely malleable.

jjoonathan 3y ago

> football team In football, the rules have been extensively tuned to promote a fair fight. Perhaps we should do a bit more of that sort of thing in corporate law.

hattmall 3y ago

As the article states the issue is with courts not companies. We need a state actor to pass a law similar to the weapons laws of other states that guarantee a right to scrape. Then all the scraping companies setup shop in that state. If a service doesn't want their data scraped they need to make sure that it doesn't get sent into that state. Ideally a large enough state that companies wouldn't want to block.

nosecreek 3y ago
Agreed that legal clarity is important - especially for smaller players. I've built a significant hobby site that relies fairly heavily on scraping (grocery price comparison site). I believe what I'm doing is morally okay, and also that big players wouldn't run into any issues, but when it's just me (or even if it was a small company) the legal 'grey area' makes it a much bigger risk.
tomcam 3y ago
I like what you’re saying, but how do you provide for the existence of evil or simply incompetent scrapers who drag the system down due to incompetence?

autoexec 3y ago

> Naturally, they're going to say "web scraping uses resources, stop it!" that's the expected cost of publishing something to the public internet. People are going to access it. No one has a right to complain when people access something that was put there for the public to see. Scrappers can be dicks about it too, they can get lazy and endlessly hammer away at some server or repeatedly pull down the same content because they messed up, but we don't need need litigation for that. If something raises to the level of DoS that's already covered under existing laws. > it's completely in keeping with what amoral corporations locked in constant battle would be expected to do: maximize benefits to themselves while minimizing benefits to others. Maybe we need to rethink giving some of these corporations the privilege of corporate personhood if they are just going to make things worse for everyone else while only enriching themselves. We don't need to allow parasites and pillagers to take whatever they want at our expense.

mrkeen 3y ago
> Scrappers can be dicks about it too It's not always about individual bad actors. You can have lots of small players causing problems. I wonder how many python developers there are right now trying to make their own offline copy of stackoverflow.com. Wikipedia has a great defence against this. They ask you not to scrape, and at the same time, provide torrents of the data (https://meta.wikimedia.org/wiki/Data_dump_torrents https://meta.wikimedia.org/wiki/Data_dump_torrents)
- rzzzt 3y ago
  Stack Exchange also provides one: https://archive.org/details/stackexchange https://archive.org/details/stackexchange There was a hiccup around June but that seems resolved now: https://meta.stackexchange.com/questions/389922/june-2023-data-dump-is-missing https://meta.stackexchange.com/questions/389922/june-2023-da...
  - genewitch 3y ago
    yeah i thought this was the case. There's a couple of other sites that - i hesitate to state this completely factual phrase - leave dumps for the public to pick up.

runesofdoom 3y ago

The problem with that sort "that's what amoral corporations do" reasoning is that corporations are permitted to exist because of the idea that they do contribute to the net public good. Once that idea is out the window, then there's no reason for society not to treat corporations as the hungry Lovecraftian nightmares they are and obliterate them with fire and steamship.

KieranMac 3y ago

I think the difference is that defeating the other team is the point of sports, whereas at least ostensibly the law is supposed to provide a set of coherent rules for businesses to compete against each other. Trademarks are defined according to certain legal rules, and if you have one, this is how they provide you with a limited monopoly in a certain context. Allowing businesses to define property law through contracts lets people define the rules however they want. And that leads to irrational results.

KirillPanov 3y ago

> You would not say of a football team You also would not let one football team buy up 60% of the other football teams in the world and merge them into one MEGA-TEAM. But apparently we still cannot summon the willpower to enforce simple and straightforward 100+-year-old antitrust laws (so instead we make new ones and just won't enforce those either).

vkou 3y ago
> You also would not let one football team buy up 60% of the other football teams in the world and merge them into one MEGA-TEAM. Why not? If that mega-team wants to destroy their revenue stream (People want to watch competitive football, as opposed to curb-stomps), they are free to do that.
- sobkas 3y ago
  And you believe that MEGA-TEAM won't get bailed out? Because otherwise bloated corpse of MEGA-TEAM will smother football as we know it?
  - vkou 3y ago
    What do you mean bailed-out? The whole point of competitive sports is for there to be, well, meaningful competition. All the money in that industry only exists if there's meaningful chances for either team to win. Winning too hard is a detriment to the winner. That is incredibly different from capitalism, where the whole point of it is for one of the participants to win the competition, to the deteriment of the rest of us, which is why society comes up with elaborate rules for preventing that.