4 ms·

Hi, as a non-webdev I want to know if rate limiting wouldn't make this a non concern?

by bobajeff 1y ago

mrweasel 1y ago
I still don't want you to create 1000 non-sense accounts, even if you can only create 100 per hour.
- overfeed 1y ago
  Then you need to level up & have defense in depth instead of relying on security through obscurity. On the public internet, web clients are user agents, and not all users are benign. This is an arms race: asking the other side to unilaterally disarm is unlikely to work, so you change what you can control.
  - joepie91_ 1y ago
    This is a defeatist argument. That it's technically possible to abuse things doesn't mean the responsibility needs to fall on the defending party, especially not when that is brought up in response to asking someone to reflect on possibilities for abuse - by that point it starts looking a lot more like a "well you'll just have to deal with it" argument that socially defends the abusers, and a lot less like genuine advice.
    - overfeed 1y ago
      > This is a defeatist argument. No side is getting defeated any time soon. I've been involved in skirmishes on both sides of scraping, and as I said, it's an arms race with no clear winner. To be clear, not all scraping is abuse. The number of people who'll start scraping because a new tool exists is a negligible (i.e. <0.001 of scraping). Scraping itself is not hard at all, a noob who can copy-paste code from the web or vibe-code a client that can scrape 80-90% of the web. A motivated junior can raise that to maybe 98/99% of the Internet using nothing but libraries that existed before this tool. > especially not when that is brought up in response to asking someone to reflect on possibilities for abuse Sir/ma'am - this is hacker news, granted, it's aspirational, but still, hiding information is not the way. As someone who's familiar with the arts, there is nothing new or groundbreaking in this engine. Further, is no inherent moral high ground for the "defenders" either: many anti-scraping methods rely on client fingerprinting and other privacy-destroying techniques, so it's not the existence of the tool or technique, but how one uses it. >... "well you'll just have to deal with it" argument that socially defends the abusers The abuse predate the tool, so wishing the tool away is unlikely to help. Scraping is a numbers game on both sides, the best one vam hope for is to defeat the vast majority of the average adversaries, but a few fall through the cracks, the point is to outrun your fellow hiker, not the bear. However, should you encounter an adversery who has specifically chosen you as a target, then victory is far from assured. The usual result is a drawn-out stalemate. Most well-behaved scrapers are left alone.

Hi, as a non-webdev I want to know if rate limiting wouldn't make this a non concern?

mrweasel 1y ago

I still don't want you to create 1000 non-sense accounts, even if you can only create 100 per hour.

overfeed 1y ago
Then you need to level up & have defense in depth instead of relying on security through obscurity. On the public internet, web clients are user agents, and not all users are benign. This is an arms race: asking the other side to unilaterally disarm is unlikely to work, so you change what you can control.
- joepie91_ 1y ago
  This is a defeatist argument. That it's technically possible to abuse things doesn't mean the responsibility needs to fall on the defending party, especially not when that is brought up in response to asking someone to reflect on possibilities for abuse - by that point it starts looking a lot more like a "well you'll just have to deal with it" argument that socially defends the abusers, and a lot less like genuine advice.
  - overfeed 1y ago
    > This is a defeatist argument. No side is getting defeated any time soon. I've been involved in skirmishes on both sides of scraping, and as I said, it's an arms race with no clear winner. To be clear, not all scraping is abuse. The number of people who'll start scraping because a new tool exists is a negligible (i.e. <0.001 of scraping). Scraping itself is not hard at all, a noob who can copy-paste code from the web or vibe-code a client that can scrape 80-90% of the web. A motivated junior can raise that to maybe 98/99% of the Internet using nothing but libraries that existed before this tool. > especially not when that is brought up in response to asking someone to reflect on possibilities for abuse Sir/ma'am - this is hacker news, granted, it's aspirational, but still, hiding information is not the way. As someone who's familiar with the arts, there is nothing new or groundbreaking in this engine. Further, is no inherent moral high ground for the "defenders" either: many anti-scraping methods rely on client fingerprinting and other privacy-destroying techniques, so it's not the existence of the tool or technique, but how one uses it. >... "well you'll just have to deal with it" argument that socially defends the abusers The abuse predate the tool, so wishing the tool away is unlikely to help. Scraping is a numbers game on both sides, the best one vam hope for is to defeat the vast majority of the average adversaries, but a few fall through the cracks, the point is to outrun your fellow hiker, not the bear. However, should you encounter an adversery who has specifically chosen you as a target, then victory is far from assured. The usual result is a drawn-out stalemate. Most well-behaved scrapers are left alone.