4 ms·
That's a shame, really. You are risking that none of your stuff will show up on search engines, not even Marginalia. You are risking that none of your stuff wil
by immibis 2y ago
That's a shame, really. You are risking that none of your stuff will show up on search engines, not even Marginalia. You are risking that none of your stuff will be saved in the Wayback Machine. Maybe you want that, in which case you should block all the clouds and data centers, just to be sure. You might even be blocking your site from some small ISPs based on where they run their CGNAT gateway (I doubt this, but it's possible).
As far as I noticed, ping with a spoofed source address is the only actual abuse mentioned in the article. It should go without saying that you can't tell if a spoofed ping packet came from AWS, because the source address is the address the spoofer wants you to send a reply to, not the spoofer's address. And a much less invasive mitigation would be rate-limiting pings to, say, 10 per second.
While the Internet is becoming balkanized this is mostly because of social media siloing itself to generate advertising and data revenue and to extract profit from AI training data (e.g. the Reddit/Google exclusivity deal) rather than because of providers blocking IP ranges.
I certainly don't understand the rational mindset behind blocking certain providers over some pings and then complaining about IP connectivity becoming balkanized. The balkanization is caused by the ones doing the blocking.
- wmf 2y agoI see it more as a lament that they have been "forced" to take actions that may cause balkanization as a side effect. However the overall tone is that of weaving a grand historical narrative out of a mosquito bite.
- Laforet 2y agoI had to read the article twice to be sure that it was a utilitarian move (however questionable it might be) rather than a grand ideological stand that the article seems to spend much time portraying. FWIW, data center IP addresses are already being treated as second class citizens by major content/service providers, and this has become an escalating barrier to self hosting. I am honestly not sure what the author is trying to accomplish.
- chrisweekly 2y ago> "data center IP addresses are already being treated as second class citizens by major content/service providers, and this has become an escalating barrier to self hosting" Could you please expand on this a bit?
- thrdbndndn 2y agoDC IPs are often: 1. totally blocked by some services (especially those related to copyright, like almost all the streaming services), 2. treated as suspicious by lots of CDNs (so you would get captchas more frequently; have stricter rate control, etc.)
- chrisweekly 2y agoThanks. Still interested in OP's response too. Also, what qualifies as a data center?
- Laforet 2y agoHi, OP here. I did not respond since another poster had beaten me to it but here we go. The reply above yours is mostly correct though I have to admit that “data center IP” could be a bit of a misnomer when it comes to IP reputation. There are essentially 4 categories: - Residential landline connections are the most mundane but are also least restricted because this is where your average users are found. The odds of bad actors on the same network is fairly low, and most ISPs will overlook minor transgressions to not incur additional customer support costs. - Mobile data connections are often behind CG-NAT. Blocking entire IP range tends to generate a lot of false positives so it doesn’t happen very often. - Institutional IP ranges (such as 17.0.0.0/8 or any org that maintains their own ASN) tends to get a pass as well because they tends to have their own IT and networking department to take collective responsibility if something untoward was to happen. - This leaves public could and hosting services on the lowest tier because these networks have very low barrier of entry for bad actors . Connections from these IP addresses are also far more likely to be bots and scrapers than a human user so most TDS systems are all too happy to block them.
- altairprime 2y agoYou frame ‘not showing up in search engines’ as a drawback, a downside. What underlies that assumption for you? Could you help me understand it? I don’t share the viewpoint, but I’m still working to understand it.
- rkeene2 2y agoThe logic seems clear to me: if you want people to be able to make use of/benefit from your published knowledge/artifacts you would want people to be able to find it when they are searching for that kind of published knowledge/artifact. People use search engines to perform the search for published knowledge/artifacts. Thus, publishing knowledge/artifacts but restricting them such that they cannot be found is a set of simultaneous contradictory actions.
- altairprime 2y agoNoted, thank you; I’d still like to hear OP’s viewpoint, though.
- noirbot 2y agoI'd be curious about your opinion. I have no real love for search engines, but if I'm going to the trouble of putting things on the internet in public vs. just sending them to people directly, presumably it would be because I want people outside of my circle of friends to see it. If you're not out personally marketing/reposting your blog/site places, search engines are pretty much the only other way people would find you. And even if I saw your site on here, say, and liked it, if I don't bookmark it immediately, I'd still go to a search engine to try to re-find it in the future if I wanted to go back. Not to say you need to be doing paid ads or trying to raise your SEO or whatever, but I've had times where I remember some unique phrase from an article I read years ago and Google can use that to find the original source.
- m3047 2y ago> putting things on the internet in public vs. just sending them to people directly A large proportion of the resources I host on-prem are just that. Stuff that's too large for email, or isn't static, or may get updated. Unless people host their own email server (like I do), it's going to e.g. Gmrgle anyway if you email it to them. Maybe it gets blocked, maybe it gets fubared. Uploading it to an on-prem server and sending the link in an email is no more trouble and any issues are easier to debug.
- SoftTalker 2y agoNothing shows up on search engines anymore except huge ad-laden content farms and very large sites like wikipedia, stackoverflow, and big news/media company sites, and of course YouTube. I have not seen individual blogs or small enthusiast sites in search results for quite some time. If I want wikipedia or stackoverflow I can just search those sites directly. I'd like an option to exclude all the "usual suspects" and see some more long tail stuff.
- immibis 2y agoTry Marginalia Search
- chx 2y agoHave you tried Kagi?
- Brian_K_White 2y agokagi, & even more kagi small web
- bcye 2y agoSmall web literally gives 0 results a lot of times, blocking or deranking websites that don't have a "reject all" button works like a charm though
- arcza 2y agoThe author has said for a long time they have had a "no crawl" policy anyways, I don't think the author cares and with the enshittification of Google they are probably in the right anyways.
- Tijdreiziger 2y ago> You might even be blocking your site from some small ISPs based on where they run their CGNAT gateway (I doubt this, but it's possible). Why wouldn’t (even small) ISPs run their CGNAT gateway in their own IP space? Running a CGNAT gateway in the cloud would lead to a lot of problems. I think your subscribers wouldn’t be able to watch Netflix and the like, since they wouldn’t be on a residential IP. It would probably also lead to more anti-bot CAPTCHAs from Cloudflare and Google. Are there actually any known examples of this?