4 ms·

X updates its terms to ban crawling and scraping

19 points by vinnyglennon 3y ago

jsnell 3y ago
Dupe: https://news.ycombinator.com/item?id=37433759 https://news.ycombinator.com/item?id=37433759
jandrese 3y ago
Good luck with that.
dom96 3y ago
Warning for those considering this: I got banned from Reddit for scraping it. I was nice enough to include my username in the user agent (I foolishly thought they would be nice and give me a warning instead of straight up banning my 10+ year old account if they didn't like what I was doing).
kmeisthax 3y ago
Y'know, if tech companies think crawling the entire Internet to train text and image generators is a bad thing then maybe they should stop also trying to train text and image generators on Other People's Content. Alternatively if AI is just that good then anyone should be allowed to scrape and train and the resulting models should be public. If AI companies don't think they should pay for training data then I shouldn't have to pay for their models. As it currently stands Xitter[0] is trying to have their cake and eat it, too. Reddit as well. The idiots running these companies are burning everything to the ground for the sake of trying to build a new monopoly around training data rights that largely don't exist. [0] Pronounced "shitter"
taytus 3y ago
INAL but wasn't scrapping legal? They can ban it in their T&C but that doesn't make it illegal.
- jandrese 3y ago
  This allows them to ban accounts that they suspect are scraping. Not that they really needed an excuse to ban accounts before, but this is about sending a message.
ActionHank 3y ago
So when companies do this I genuinely wonder: * How will they know? * What will they do? They've literally spent years convincing the world that the data on their platform is freely open to the public therefore it's ok if they collect as much data per person that they can.
yonatan8070 3y ago
This is so confusing, every time I see a headline about Twitter I think to myself "what does X.Org have to do with this?"
tzs 3y ago
When I read the title I thought "crawling and scraping...no problem, I can do crawling separate from scraping". But alas, the terms actually ban crawling or scraping.

19 points by vinnyglennon 3y ago

jsnell 3y ago

Dupe: https://news.ycombinator.com/item?id=37433759 https://news.ycombinator.com/item?id=37433759

jandrese 3y ago

Good luck with that.

dom96 3y ago

Warning for those considering this: I got banned from Reddit for scraping it. I was nice enough to include my username in the user agent (I foolishly thought they would be nice and give me a warning instead of straight up banning my 10+ year old account if they didn't like what I was doing).

kmeisthax 3y ago

Y'know, if tech companies think crawling the entire Internet to train text and image generators is a bad thing then maybe they should stop also trying to train text and image generators on Other People's Content. Alternatively if AI is just that good then anyone should be allowed to scrape and train and the resulting models should be public. If AI companies don't think they should pay for training data then I shouldn't have to pay for their models. As it currently stands Xitter[0] is trying to have their cake and eat it, too. Reddit as well. The idiots running these companies are burning everything to the ground for the sake of trying to build a new monopoly around training data rights that largely don't exist. [0] Pronounced "shitter"

taytus 3y ago

INAL but wasn't scrapping legal? They can ban it in their T&C but that doesn't make it illegal.

jandrese 3y ago
This allows them to ban accounts that they suspect are scraping. Not that they really needed an excuse to ban accounts before, but this is about sending a message.

ActionHank 3y ago

So when companies do this I genuinely wonder: * How will they know? * What will they do? They've literally spent years convincing the world that the data on their platform is freely open to the public therefore it's ok if they collect as much data per person that they can.

yonatan8070 3y ago

This is so confusing, every time I see a headline about Twitter I think to myself "what does X.Org have to do with this?"

tzs 3y ago

When I read the title I thought "crawling and scraping...no problem, I can do crawling separate from scraping". But alas, the terms actually ban crawling or scraping.