5 ms·

Anyone else got served a page with garbage content meant for AI scrapers?

perching_aix 6mo ago
Oh, that's kinda fun. I got the same that I get for every Mastodon (and Anubis-protected) link: a page telling me that it won't work without JavaScript. I guess since AI scrapers these days do run some amount of JS, that is some second layer of defense? At least for Twitter there are proxies that work without JS. For Mastodon, none that I'm aware of. I usually just audibly sigh and remark that they shall "keep their secrets then", and move on.
- rglullis 6mo ago
  We are not talking about the same thing, it seems. I can understand a web page that doesn't work without javascript. What I do not understand is someone going through all this work of putting an AI-scraper tarpit on Mastodon, a system that fundamentally needs to have its data distributed to other servers. It's just signalling and posturing, because that content is available on any server that has someone following the account. (Tip to AI scrapers: if you want to slurp all the data from the fediverse, just create an account on mastodon.social and pull the data from the "Federated timeline" stream.)
  - NitpickLawyer 6mo ago
    > What I do not understand It's not you. It's the people that were somehow convinced that serving crap is gonna "hurt" the models. These are people who have 0 clue on how models are trained and how they work, but have been riled up by others who similarly don't understand the technical details, but have strong biases against them. This is ignorance signalling at its finest. And, as expected, it's hurting their (regular) users more than they'll hurt the model trainers. Oh well..
stingraycharles 6mo ago
I got it as well, using a VPN.
- zarzavat 6mo ago
  I got it and I'm on a residential internet connection with a normal web browser. I have no idea what signal they're using but it's not working.
  - rglullis 6mo ago
    Brave browser user?
userbinator 6mo ago
To those who automatically assume humans with "weird" setups are "AI scrapers" (also a bit of a boogeyman these days): FUCK YOU. I'm a human, not a stupid mindless sheeple.
- dalmo3 6mo ago
  I assumed the author was one of those HN haters and was filtering by referrer. Now I'm curious.
  - rglullis 6mo ago
    No, I tried going to the page on a fresh incognito session. Same thing.
- rglullis 6mo ago
  Yeah, that's the part of the Fediverse that I really don't like... All the talk about "putting the human first" and "embracing diversity" goes out of the window the moment you are not diverse in the way they want.
- goblin89 6mo ago
  Some people feel like we are being turned into content producers for large corporations to monetize and they’re not entirely wrong. I don’t mind when people take a stance, even if their methods aren’t perfect and it may inconvenience myself personally. If you’re in rush to airport or hospital and you are delayed by protesters for a cause you don’t understand, it’s one thing, I could understand a bit of cursing. However, this is someone’s web resource, they are free to do with it whatever they want, and they owe you and me nothing.
  - rglullis 6mo ago
    > I don’t mind when people take a stance, even if their methods aren’t perfect and it may inconvenience myself personally. The problem is that not only their methods "aren't perfect", but completely ineffective. If you are posting on a public social network, your data will be available to the public, one way or another. The whole protest becomes a "performance art" kind of thing: it might be useful for creating awareness, but in most cases the people who will be seeing it are the ones who are aware of it in the first place.
DomB 6mo ago
Unbelievable, yes.

Oh, that's kinda fun. I got the same that I get for every Mastodon (and Anubis-protected) link: a page telling me that it won't work without JavaScript. I guess since AI scrapers these days do run some amount of JS, that is some second layer of defense? At least for Twitter there are proxies that work without JS. For Mastodon, none that I'm aware of. I usually just audibly sigh and remark that they shall "keep their secrets then", and move on.

rglullis 6mo ago
We are not talking about the same thing, it seems. I can understand a web page that doesn't work without javascript. What I do not understand is someone going through all this work of putting an AI-scraper tarpit on Mastodon, a system that fundamentally needs to have its data distributed to other servers. It's just signalling and posturing, because that content is available on any server that has someone following the account. (Tip to AI scrapers: if you want to slurp all the data from the fediverse, just create an account on mastodon.social and pull the data from the "Federated timeline" stream.)
- NitpickLawyer 6mo ago
  > What I do not understand It's not you. It's the people that were somehow convinced that serving crap is gonna "hurt" the models. These are people who have 0 clue on how models are trained and how they work, but have been riled up by others who similarly don't understand the technical details, but have strong biases against them. This is ignorance signalling at its finest. And, as expected, it's hurting their (regular) users more than they'll hurt the model trainers. Oh well..

stingraycharles 6mo ago

I got it as well, using a VPN.

zarzavat 6mo ago
I got it and I'm on a residential internet connection with a normal web browser. I have no idea what signal they're using but it's not working.
- rglullis 6mo ago
  Brave browser user?

userbinator 6mo ago

To those who automatically assume humans with "weird" setups are "AI scrapers" (also a bit of a boogeyman these days): FUCK YOU. I'm a human, not a stupid mindless sheeple.

dalmo3 6mo ago
I assumed the author was one of those HN haters and was filtering by referrer. Now I'm curious.
- rglullis 6mo ago
  No, I tried going to the page on a fresh incognito session. Same thing.
rglullis 6mo ago
Yeah, that's the part of the Fediverse that I really don't like... All the talk about "putting the human first" and "embracing diversity" goes out of the window the moment you are not diverse in the way they want.
goblin89 6mo ago
Some people feel like we are being turned into content producers for large corporations to monetize and they’re not entirely wrong. I don’t mind when people take a stance, even if their methods aren’t perfect and it may inconvenience myself personally. If you’re in rush to airport or hospital and you are delayed by protesters for a cause you don’t understand, it’s one thing, I could understand a bit of cursing. However, this is someone’s web resource, they are free to do with it whatever they want, and they owe you and me nothing.
- rglullis 6mo ago
  > I don’t mind when people take a stance, even if their methods aren’t perfect and it may inconvenience myself personally. The problem is that not only their methods "aren't perfect", but completely ineffective. If you are posting on a public social network, your data will be available to the public, one way or another. The whole protest becomes a "performance art" kind of thing: it might be useful for creating awareness, but in most cases the people who will be seeing it are the ones who are aware of it in the first place.

DomB 6mo ago

Unbelievable, yes.