8 ms·

I'm blocking connections from AWS to my on-prem services

174 points by m3047 2y ago

immibis 2y ago
That's a shame, really. You are risking that none of your stuff will show up on search engines, not even Marginalia. You are risking that none of your stuff will be saved in the Wayback Machine. Maybe you want that, in which case you should block all the clouds and data centers, just to be sure. You might even be blocking your site from some small ISPs based on where they run their CGNAT gateway (I doubt this, but it's possible). As far as I noticed, ping with a spoofed source address is the only actual abuse mentioned in the article. It should go without saying that you can't tell if a spoofed ping packet came from AWS, because the source address is the address the spoofer wants you to send a reply to, not the spoofer's address. And a much less invasive mitigation would be rate-limiting pings to, say, 10 per second. While the Internet is becoming balkanized this is mostly because of social media siloing itself to generate advertising and data revenue and to extract profit from AI training data (e.g. the Reddit/Google exclusivity deal) rather than because of providers blocking IP ranges. I certainly don't understand the rational mindset behind blocking certain providers over some pings and then complaining about IP connectivity becoming balkanized. The balkanization is caused by the ones doing the blocking.
- wmf 2y ago
  I see it more as a lament that they have been "forced" to take actions that may cause balkanization as a side effect. However the overall tone is that of weaving a grand historical narrative out of a mosquito bite.
  - Laforet 2y ago
    I had to read the article twice to be sure that it was a utilitarian move (however questionable it might be) rather than a grand ideological stand that the article seems to spend much time portraying. FWIW, data center IP addresses are already being treated as second class citizens by major content/service providers, and this has become an escalating barrier to self hosting. I am honestly not sure what the author is trying to accomplish.
    - chrisweekly 2y ago
      > "data center IP addresses are already being treated as second class citizens by major content/service providers, and this has become an escalating barrier to self hosting" Could you please expand on this a bit?
      - thrdbndndn 2y ago
        DC IPs are often: 1. totally blocked by some services (especially those related to copyright, like almost all the streaming services), 2. treated as suspicious by lots of CDNs (so you would get captchas more frequently; have stricter rate control, etc.)
        chrisweekly 2y ago
        Thanks. Still interested in OP's response too. Also, what qualifies as a data center?
        Laforet 2y ago
        Hi, OP here. I did not respond since another poster had beaten me to it but here we go. The reply above yours is mostly correct though I have to admit that “data center IP” could be a bit of a misnomer when it comes to IP reputation. There are essentially 4 categories: - Residential landline connections are the most mundane but are also least restricted because this is where your average users are found. The odds of bad actors on the same network is fairly low, and most ISPs will overlook minor transgressions to not incur additional customer support costs. - Mobile data connections are often behind CG-NAT. Blocking entire IP range tends to generate a lot of false positives so it doesn’t happen very often. - Institutional IP ranges (such as 17.0.0.0/8 or any org that maintains their own ASN) tends to get a pass as well because they tends to have their own IT and networking department to take collective responsibility if something untoward was to happen. - This leaves public could and hosting services on the lowest tier because these networks have very low barrier of entry for bad actors . Connections from these IP addresses are also far more likely to be bots and scrapers than a human user so most TDS systems are all too happy to block them.
- altairprime 2y ago
  You frame ‘not showing up in search engines’ as a drawback, a downside. What underlies that assumption for you? Could you help me understand it? I don’t share the viewpoint, but I’m still working to understand it.
  - rkeene2 2y ago
    The logic seems clear to me: if you want people to be able to make use of/benefit from your published knowledge/artifacts you would want people to be able to find it when they are searching for that kind of published knowledge/artifact. People use search engines to perform the search for published knowledge/artifacts. Thus, publishing knowledge/artifacts but restricting them such that they cannot be found is a set of simultaneous contradictory actions.
    - altairprime 2y ago
      Noted, thank you; I’d still like to hear OP’s viewpoint, though.
  - noirbot 2y ago
    I'd be curious about your opinion. I have no real love for search engines, but if I'm going to the trouble of putting things on the internet in public vs. just sending them to people directly, presumably it would be because I want people outside of my circle of friends to see it. If you're not out personally marketing/reposting your blog/site places, search engines are pretty much the only other way people would find you. And even if I saw your site on here, say, and liked it, if I don't bookmark it immediately, I'd still go to a search engine to try to re-find it in the future if I wanted to go back. Not to say you need to be doing paid ads or trying to raise your SEO or whatever, but I've had times where I remember some unique phrase from an article I read years ago and Google can use that to find the original source.
    - m3047 2y ago
      > putting things on the internet in public vs. just sending them to people directly A large proportion of the resources I host on-prem are just that. Stuff that's too large for email, or isn't static, or may get updated. Unless people host their own email server (like I do), it's going to e.g. Gmrgle anyway if you email it to them. Maybe it gets blocked, maybe it gets fubared. Uploading it to an on-prem server and sending the link in an email is no more trouble and any issues are easier to debug.
- SoftTalker 2y ago
  Nothing shows up on search engines anymore except huge ad-laden content farms and very large sites like wikipedia, stackoverflow, and big news/media company sites, and of course YouTube. I have not seen individual blogs or small enthusiast sites in search results for quite some time. If I want wikipedia or stackoverflow I can just search those sites directly. I'd like an option to exclude all the "usual suspects" and see some more long tail stuff.
  - immibis 2y ago
    Try Marginalia Search
  - chx 2y ago
    Have you tried Kagi?
  - Brian_K_White 2y ago
    kagi, & even more kagi small web
    - bcye 2y ago
      Small web literally gives 0 results a lot of times, blocking or deranking websites that don't have a "reject all" button works like a charm though
- arcza 2y ago
  The author has said for a long time they have had a "no crawl" policy anyways, I don't think the author cares and with the enshittification of Google they are probably in the right anyways.
- Tijdreiziger 2y ago
  > You might even be blocking your site from some small ISPs based on where they run their CGNAT gateway (I doubt this, but it's possible). Why wouldn’t (even small) ISPs run their CGNAT gateway in their own IP space? Running a CGNAT gateway in the cloud would lead to a lot of problems. I think your subscribers wouldn’t be able to watch Netflix and the like, since they wouldn’t be on a residential IP. It would probably also lead to more anti-bot CAPTCHAs from Cloudflare and Google. Are there actually any known examples of this?
xyzzy123 2y ago
All the major cloud providers publish machine-readable lists of their ranges, e.g. https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-rang... https://www.microsoft.com/en-us/download/details.aspx?id=56519 https://www.microsoft.com/en-us/download/details.aspx?id=565... https://support.google.com/a/answer/10026322?product_name=UnuFlow&hl=en&visit_id=638605762510076874-1416434010&rd=1&src=supportwidget0&hl=en https://support.google.com/a/answer/10026322?product_name=Un... etc etc...
- m3047 2y ago
  I haven't checked today, but e.g. Amazon's JSON file belies the fact that they own 3.0.0.0/8 in practice if not in fact. I'm not going to play "let's block Amazon's huge JSON list". I've got 53 rules. I'm willing to step on some fingers / toes. I can get to sites in AWS just fine.
  - oneplane 2y ago
    Amazon does have 3.0.0.0/8. Granted, it is noted as two separate /9 blocks, but it's all Amazon. Perhaps a better use of your time (because de-balkanization of the internet would be more of an academic exercise than a mass market reality; consumers have already self-selected into social media and crappy LLM outputs which is a far bigger matter and driver than ephemeral resources or reverse DNS) would be making your TLS work in a way that doesn't allow for easy abuse. So automatic redirects, HSTS, and a certificate that is standards-compliant.
    - huggingmouth 2y ago
      > consumers have already self-selected into ... crappy LLM outputs Tangent, but last I checked, it was all the big tech giants pushing it down our throat. Unless it's your full time job to find loopholes and workarounds, there is no reasonable way for consumers to opt-out.
  - zamadatix 2y ago
    > I haven't checked today, but e.g. Amazon's JSON file belies the fact that they own 3.0.0.0/8 in practice if not in fact. While it may seem more useful to aggregate the ranges in some points of view it'd be significantly less useful from other points of view. E.g. those who want to whitelist any IP ranges matching a specific DC, service, availability zone, or country. You can always aggregate the detailed list but you can't do the inverse on your own.
UI_at_80x24 2y ago
I understand the appeal. If I wanted to wall-off large swaths of the internet from what I create I would do the same thing. It's not much different then blocking entire countries. The desire to limit the noise and only allow a "small circle of friends" is also appealing. But I do that for specific services, not my domains in general. Mumble server: only open to the 3-4 countries that my friends are in, and none of the 'cloud providers'. Tech blog: world+dog can see it. I am firmly in the 'We all benefit from shared knowledge' camp. So if my notes on modem init strings for my 300-baud C64 modem can help one other person; they won't go through the same pain I went through, and the world will be a better place. I get the desire, for many reasons. That's cool. You do you.
- m3047 2y ago
  I wrote the "Zen InterSLIP Dialing Script" and it literally went around the world. Amazon is too large to ignore. I understand that a lot of ICMP and SYN traffic is garbage. I'd be happy to help out and block it (and I do have mitigations in place); in fact I do, by default. That's part of why Amazon is a PITA because they trigger my mitigations "at scale". Amazon doesn't help sort the wheat from the chaff: "send a PCAP (for a ping issue)". I don't learn anything by sending Amazon stuff and hearing nothing. I don't need their good traffic any more than I need their bad traffic... or the traffic which is spoofed which is attacking them. If they can't see fit to help me help them, I don't need any of it. I'm just keeping my life simple.
  - Dylan16807 2y ago
    Is the traffic causing you problems though? Or do you just dislike that they don't want to fix this?
    - m3047 2y ago
      Problems? What kind of problems? Discovering word embeddings was sort of a happy accident of LLMs: back propagation "improved" them from a similarly random starting point as the NNs. It's not a technical problem as in overwhelming any resource. It overwhelms me, for starters. Secondly my existing mitigations suggested it, I resisted the move for some of the reasons people here are saying it's a bad idea; I finally concluded the benefits outweighed the costs and I'd try it and see what happens. So far, so good. Once the fire burns out it should be great.
bdcravens 2y ago
In considering this, my first thought was it would block bona fide desktops running in AWS, especially for service offerings like Amazon Workspaces. However, it looks like the IP space for such services are publicly documented if the need arises to specifically whitelist those IPs. All that said, it's trivial to use proxies or VPNs to bypass any blocks.
- Sammi 2y ago
  > All that said, it's trivial to use proxies or VPNs to bypass any blocks. Maybe for you sure. The large drove of people flocking to serverless these days suggests that even most technical people don't want anything to do with their own networking or infrastructure.
  - immibis 2y ago
    Proxy selling is a successful business (access on the order of 10 million residential IP addresses for on the order of $100 per month), and they don't say how they get access to those. Probably, the more people block access to non-residential IP addresses, the more money botnet vendors make.
    - bdcravens 2y ago
      At least among the "ethical" ones, my understanding is that it's freeware that has been packaged with proxy access (presumably disclosed to the end user, but that's a matter of interpretation)
xcdzvyn 2y ago
I'm slightly ashamed to admit I don't know what RPZ, PTR, or SPF are, nor do I understand the asides about AI or reverse DNS. What precisely is his problem with Amazon?
- arcza 2y ago
  Meaningless low-value and non-human traffic wasting resources and/or training models. Their blog is for real humans per the author's writing, which is a valid stance to take.
theideaofcoffee 2y ago
Yawn. Old man yells at cloud (literally). So he's taking his little netblock ball and going home because of some failed purity tests: bad or nonexistent PTRs, excessive ICMP, oh my! The gentleman's agreements that held together the early internet and web and the unwritten practices like that are long gone, get with the times, there ain't any going back to how it was. Otherwise, feel free to disconnect entirely if you don't want to deal with the new reality. I'm going to going out on a limb and guess that all of this traffic that isn't related directly to AWS, but its customers. You can set PTRs for your allocated elastic IPs with a request to support. But then again nobody is going to do it because... it doesn't matter. It may have mattered when you were hosting with a block that you actually truly owned, before the ICANN times, but no more. No one cares. Everything is ephemeral, so why should the reverse matter when things get cycled through addresses multiple times per day? If you're seeing excessive anything, then it's probably time to reach out to the abuse contact published in the whois. Let me help you with that: OrgAbuseHandle: AEA8-ARIN OrgAbuseName: Amazon EC2 Abuse OrgAbuseEmail: trustandsafety@support.aws.com OrgAbuseRef: https://rdap.arin.net/registry/entity/AEA8-ARIN Comment: All abuse reports MUST include: Comment: * src IP Comment: * dest IP (your IP) Comment: * dest port Comment: * Accurate date/timestamp and timezone of activity Comment: * Intensity/frequency (short log extracts) Comment: * Your contact details (phone and email) Without these we will be unable to identify the correct owner of the IP address at that point in time. Use modern features built in to modern versions of common packages and products: rate limiting, redirects, filters, and on and on. If you're just blocking to block to make some sort of statement into the void, you're just hastening that balkanization.
- cdchn 2y ago
  AWS doesn't mess around with abuse reports either. If you send a report and it checks out, they're opening a case with the customer in question telling them to explain themselves.
  - hughesjj 2y ago
    But but that requires an async unidirectional communication with another human (the horror!)
issafram 2y ago
[flagged]
- jaktet 2y ago
  https://12ft.io/http://consulting.m3047.net/dubai-letters/balkanized-internet.html https://12ft.io/http://consulting.m3047.net/dubai-letters/ba...
  - NetOpWibby 2y ago
    Bless
- enjoyyourlife 2y ago
  You can use Reader mode on Firefox
  - dredmorbius 2y ago
    Not only does Reader Mode work beautifully, but, colour choice notwithstanding, the article's CSS styling is sufficiently vanilla that my own enhanced Reader Mode stylesheet (drop caps and a few other fiddly bits) works beautifully. That's often ... not ... the case.
- huimang 2y ago
  It's high contrast, better than a lot of websites. You'll be fine.
- tomcam 2y ago
  Reader mode is the way but tbh after the first 1500 words or so I didn’t know where it was going. I’m old and life is short,
  - runxel 2y ago
    I agree. Horrible writing. For a while I thought it's just some LLM output. Not worth reading anyway, so you've lost nothing, OP.
User23 2y ago
This makes me miss the Internet. It’s really hard to explain how wonderful it was pre-commercialization. The sour pleasure of having been exactly right about how that would turn out isn’t nearly satisfactory compensation for the loss.
- Yeul 2y ago
  Ah yes nostalgic bullshit. It was always about the money.
  - RGamma 2y ago
    Not with the fierceness and at the scale of today. Boiling frog and all.
tonetegeatinst 2y ago
Firefox on mobile, using dark mode. That font actually hurts my eye, and I straigup closed the site. Many I'll write a script to scrape the text into a terminal output so I can read it without feeling like I'm having a headache. Probably just output the text via a terminal windows.
- Lucasoato 2y ago
  From iOS Safari I see it pink on dark blue… nothing wrong there with these colors, actually they’re pleasant to look at. The font as well seems nice, is there anything wrong with it?
- baq 2y ago
  You don't need to do this, Firefox on mobile has a very good reader mode.
ThePhysicist 2y ago
It's crazy how fast people start attacking your infrastructure on the Internet these days. I recently started announcing one of my /23 subnets (512 addresses) over BGP for an anycast setup, once the route was announced and traffic flowing to the router tcpdump blew up with port scan activity on all IPs of the range. Of course that doesn't have anything to do with the route, it's just that tons of people seem to indiscriminately and continuously scan for open ports on all IP ranges (my range has been unannounced for many years before so it wasn't on someones list of active servers). I find it a shocking that people still expose internal web services (e.g. Gitlab) openly to the Internet, in my opinion you should at least have one additional layer of protection through a VPN or similar mechanism so that your services aren't discoverable from the public Internet. I only expose SSH from a single bastion host, which is the only host that's publicly reachable, something that I'd like to get rid off in the future as well by adding a VPN layer on top.
- jcalvinowens 2y ago
  > something that I'd like to get rid off in the future as well by adding a VPN layer on top. What VPN software would you use? Personally I've never found anything I consider as trustworthy as OpenSSH.
  - throwaway63467 2y ago
    I use OpenVPN for historical reasons but today I’d go for Wireguard, much simpler, faster and integrated in the kernel, connectionless so much less friction when e.g. rebooting or changing networks.
  - com 2y ago
    Wireguard is quite good too, and if you’re up for some complication in your life you can do full mesh quite easily with it if your online infra is a bit distributed.
    - RGamma 2y ago
      There's also helpers for wireguard meshes. Or become dependent on yet another service like tailscale (at least there's headscale) or zerotier.
rkagerer 2y ago
I did this a long time ago after dozens of Amazon servers started slowing down our on-prem servers. I think it might have been some kind of attempted SSO stuff but never did entirely track it down. Just wrote a script to periodically download a list of their IP ranges and block 'em all, and moved on.
perching_aix 2y ago
https is broken for the site
- m3047 2y ago
  Accept my signing certificate and try again. /s
  - perching_aix 2y ago
    im good, thanks
opentokix 2y ago
[flagged]
aorth 2y ago
It has really gotten terrible. Between DDoS and bots from massive tech companies, it seems like I have several events a year where thousands or tens of thousands of IPs from a single datacenter (Singapore!) are making requests to some of my infrastructure concurrently. What can we do? I opted for CIDR aggregation and rate limiting of data center ISPs in nginx for one of my frontends. There are reasonable limits for normal IPs too. Not all of us have the capacity or desire to scale.
- m3047 2y ago
  This is the chauvinism I maybe failed to surface effectively enough. People in the cloud "just scale"; and big cuddly cloud says "ooooh, we block DoS and we won't charge you for the ingress traffic" when they whine. Whereas it really should be alleged DoS, it's not like I won't implement mitigations. I publish telemetry outing the worst offenders. What is Amazon doing for me, as a non-customer, to support our presumably shared goal? So maybe it's not really a shared goal. Here's an analogy which is very visceral for me: I have a chronic condition, which has never been "root caused" in my case by the school of western allopathic medicine. They put me on (several) drugs, I've taken them for a decade. Due to the ongoing enshittification of medical care over the past several years, I ended up having to see a naturopathic physician (with prescribing privileges, and not covered by my insurance) to get the prescriptions renewed because the western docs simply had better things to do than schedule and keep an appointment. Dude does the things a western doc should do, looks at the labs, and says with all seriousness: have you considered maybe you're allergic to drug "Y"? Holy fuck well that causes all kinds of problems identifying a replacement therapy, but that's not the point of the story here. Point is they've never done root cause, blamed the increasingly worse side effects on the condition they never root caused, put me on more drugs for the side effects, and told me to just live with it because that's what happens when you get older: this is what happens when you get trapped in an ecosystem. So I have identified a replacement therapy, it maybe doesn't work so well for the symptom drug Y was supposed to mitigate but it works. On the other hand, here are the side effects: constellation of symptoms of the actual condition (drug Y was intended to mitigate only one of them) has virtually disappeared, I sleep better, I have more energy, and I've lost nearly 20 pounds in the last six months. In summary, it's not that hard to run my own server (I have the skills, knowledge and specialized bespoke tools to support doing it my way). It keeps my professional skills sharp and gives me real practical intelligence about the fight in the streets.
jeroenhd 2y ago
I should probably do this for most of my stuff. I run some servers that require cloud-to-cloud networking, but the only inbound stuff I see coming from cloud services is bots, scanners, and scrapers. I've had to block off China's largest ISP because some broken scraper kept re-downloading the same image assets for no reason, and kept popping up on other subnets. I don't think anyone will miss my stuff if they're part of the small minority of people accessing the internet through a VPN hosted in large data centres. The biggest challenge for implementing this will probably be figuring out how to block inbound connections but keep outbound connections working. I'm sure there's a good nftables rule I can come up with eventually.
- sulandor 2y ago
  > The biggest challenge for implementing this will probably be figuring out how to block inbound connections but keep outbound connections working. nah, there's connection tracking for that. ctstate new srcip <amazon-set> drop
- m3047 2y ago
  For TCP you're just looking for packets having SYN without ACK. client server -------- SYN ---------> <-------- SYN+ACK -----
hello_computer 2y ago
AWS is a boy scout compared to places like DigitalOcean, OVHcloud, ColoCrossing, Scaleway, Tencent, or even Google. I think DigitalOcean, in particular, has made a terrible mistake marketing to the “cybersecurity” community.
- m3047 2y ago
  In fact (if you read the post) I mention blocking "some small hosting providers as a proof of concept": that would be DigitalOcean, for the reason you state.
arcza 2y ago
I've been blocking Hetzner, Digital Ocean, Linode, OVH and Contabo for a while. You can do this with pfBlocker NG by blocking ASNs, or UFW rules (https://blog.abctaylor.com/ufw-and-firewalld-rules-to-block-vps-bots-and-scrapers/ https://blog.abctaylor.com/ufw-and-firewalld-rules-to-block-...)
- blueflow 2y ago
  oof. Why Hetzner?
  - arcza 2y ago
    Due to firewall logs showing DNS amplification attack attempts
    - Dylan16807 2y ago
      Why go beyond blocking direct DNS access? (Ideally you'd make then switch to TCP by truncating UDP responses to specific clients but that sounds like a hassle to set up so it's understandable to skip that.)
    - immibis 2y ago
      Everyone is attempting all attacks all the time from everywhere. Why not secure yourself so the attempts fail?
      - oneplane 2y ago
        At that point secure would be 'offline'... It's not like botnets, "unlocker" farms and P2P doesn't originate from residential netblocks all day long. The idea of "I just want the legitimate traffic" is a simple one, but the implementation of the idea has very little to do with "I will just block the big bad cloud!".
        immibis 2y ago
        Securing yourself means not being vulnerable to the attacks. Who cares if you are exposed to an internet radiation banana equivalent? Why worry? You'll hurt yourself more from the worry than from the radiation. Blocking huge IP ranges is knocking yourself half offline, and it doesn't even stop you being "attacked". I'd start blocking if and only if there is some actual problem for your server (e.g. excessive CPU or bandwidth usage), not just because big bad scary cloud.
ruthmarx 2y ago
The cynic in me thinks this won't accomplish much. They can/do just buy data from other companies that scrape or some subsidiary. This isn't a technical problem, it's a legal/social problem.
nullc 2y ago
I've found that blocking china is also a pretty good improvement on abusive traffic relative to disruption.
cmeacham98 2y ago
Bias disclaimer: AWS is my current employer. Maybe I'm missing something obvious, but if the author believes the ping traffic is being spoofed, how could they know AWS is the source?
- mrweasel 2y ago
  AWS has a terrible reputation as being the source of absolute massive amounts of abuse and poorly written scrappers and crawler (hard to tell the difference between a bad crawler and an active attack). From experience I've seen AWS be the source of overwhelming traffic so many times that we in some cases resorted to the same solution, blocking AWS completely. I don't know if AWS doesn't care or is just slow to react. Maybe reporting is to difficult, I don't know. So the obvious you're missing is: AWS IS a huge source of "bad" traffic and getting a misbehaving customer shutdown is too hard, while renting insane amounts of capacity is too easy for bad actors. Edit: I've almost never seen GCP or Azure being the source of the same amount of crazy traffic.
knallfrosch 2y ago
Where's the summary? I didn't quite get what the problem is that you're trying to solve. Data scraping? DDOS attacks? Bandwidth trouble? Security?
Ellardo654 2y ago
[flagged]

174 points by m3047 2y ago

That's a shame, really. You are risking that none of your stuff will show up on search engines, not even Marginalia. You are risking that none of your stuff will be saved in the Wayback Machine. Maybe you want that, in which case you should block all the clouds and data centers, just to be sure. You might even be blocking your site from some small ISPs based on where they run their CGNAT gateway (I doubt this, but it's possible). As far as I noticed, ping with a spoofed source address is the only actual abuse mentioned in the article. It should go without saying that you can't tell if a spoofed ping packet came from AWS, because the source address is the address the spoofer wants you to send a reply to, not the spoofer's address. And a much less invasive mitigation would be rate-limiting pings to, say, 10 per second. While the Internet is becoming balkanized this is mostly because of social media siloing itself to generate advertising and data revenue and to extract profit from AI training data (e.g. the Reddit/Google exclusivity deal) rather than because of providers blocking IP ranges. I certainly don't understand the rational mindset behind blocking certain providers over some pings and then complaining about IP connectivity becoming balkanized. The balkanization is caused by the ones doing the blocking.

wmf 2y ago
I see it more as a lament that they have been "forced" to take actions that may cause balkanization as a side effect. However the overall tone is that of weaving a grand historical narrative out of a mosquito bite.
- Laforet 2y ago
  I had to read the article twice to be sure that it was a utilitarian move (however questionable it might be) rather than a grand ideological stand that the article seems to spend much time portraying. FWIW, data center IP addresses are already being treated as second class citizens by major content/service providers, and this has become an escalating barrier to self hosting. I am honestly not sure what the author is trying to accomplish.
  - chrisweekly 2y ago
    > "data center IP addresses are already being treated as second class citizens by major content/service providers, and this has become an escalating barrier to self hosting" Could you please expand on this a bit?
    - thrdbndndn 2y ago
      DC IPs are often: 1. totally blocked by some services (especially those related to copyright, like almost all the streaming services), 2. treated as suspicious by lots of CDNs (so you would get captchas more frequently; have stricter rate control, etc.)
      - chrisweekly 2y ago
        Thanks. Still interested in OP's response too. Also, what qualifies as a data center?
        Laforet 2y ago
        Hi, OP here. I did not respond since another poster had beaten me to it but here we go. The reply above yours is mostly correct though I have to admit that “data center IP” could be a bit of a misnomer when it comes to IP reputation. There are essentially 4 categories: - Residential landline connections are the most mundane but are also least restricted because this is where your average users are found. The odds of bad actors on the same network is fairly low, and most ISPs will overlook minor transgressions to not incur additional customer support costs. - Mobile data connections are often behind CG-NAT. Blocking entire IP range tends to generate a lot of false positives so it doesn’t happen very often. - Institutional IP ranges (such as 17.0.0.0/8 or any org that maintains their own ASN) tends to get a pass as well because they tends to have their own IT and networking department to take collective responsibility if something untoward was to happen. - This leaves public could and hosting services on the lowest tier because these networks have very low barrier of entry for bad actors . Connections from these IP addresses are also far more likely to be bots and scrapers than a human user so most TDS systems are all too happy to block them.
altairprime 2y ago
You frame ‘not showing up in search engines’ as a drawback, a downside. What underlies that assumption for you? Could you help me understand it? I don’t share the viewpoint, but I’m still working to understand it.
- rkeene2 2y ago
  The logic seems clear to me: if you want people to be able to make use of/benefit from your published knowledge/artifacts you would want people to be able to find it when they are searching for that kind of published knowledge/artifact. People use search engines to perform the search for published knowledge/artifacts. Thus, publishing knowledge/artifacts but restricting them such that they cannot be found is a set of simultaneous contradictory actions.
  - altairprime 2y ago
    Noted, thank you; I’d still like to hear OP’s viewpoint, though.
- noirbot 2y ago
  I'd be curious about your opinion. I have no real love for search engines, but if I'm going to the trouble of putting things on the internet in public vs. just sending them to people directly, presumably it would be because I want people outside of my circle of friends to see it. If you're not out personally marketing/reposting your blog/site places, search engines are pretty much the only other way people would find you. And even if I saw your site on here, say, and liked it, if I don't bookmark it immediately, I'd still go to a search engine to try to re-find it in the future if I wanted to go back. Not to say you need to be doing paid ads or trying to raise your SEO or whatever, but I've had times where I remember some unique phrase from an article I read years ago and Google can use that to find the original source.
  - m3047 2y ago
    > putting things on the internet in public vs. just sending them to people directly A large proportion of the resources I host on-prem are just that. Stuff that's too large for email, or isn't static, or may get updated. Unless people host their own email server (like I do), it's going to e.g. Gmrgle anyway if you email it to them. Maybe it gets blocked, maybe it gets fubared. Uploading it to an on-prem server and sending the link in an email is no more trouble and any issues are easier to debug.
SoftTalker 2y ago
Nothing shows up on search engines anymore except huge ad-laden content farms and very large sites like wikipedia, stackoverflow, and big news/media company sites, and of course YouTube. I have not seen individual blogs or small enthusiast sites in search results for quite some time. If I want wikipedia or stackoverflow I can just search those sites directly. I'd like an option to exclude all the "usual suspects" and see some more long tail stuff.
- immibis 2y ago
  Try Marginalia Search
- chx 2y ago
  Have you tried Kagi?
- Brian_K_White 2y ago
  kagi, & even more kagi small web
  - bcye 2y ago
    Small web literally gives 0 results a lot of times, blocking or deranking websites that don't have a "reject all" button works like a charm though
arcza 2y ago
The author has said for a long time they have had a "no crawl" policy anyways, I don't think the author cares and with the enshittification of Google they are probably in the right anyways.
Tijdreiziger 2y ago
> You might even be blocking your site from some small ISPs based on where they run their CGNAT gateway (I doubt this, but it's possible). Why wouldn’t (even small) ISPs run their CGNAT gateway in their own IP space? Running a CGNAT gateway in the cloud would lead to a lot of problems. I think your subscribers wouldn’t be able to watch Netflix and the like, since they wouldn’t be on a residential IP. It would probably also lead to more anti-bot CAPTCHAs from Cloudflare and Google. Are there actually any known examples of this?

xyzzy123 2y ago

All the major cloud providers publish machine-readable lists of their ranges, e.g. https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-rang... https://www.microsoft.com/en-us/download/details.aspx?id=56519 https://www.microsoft.com/en-us/download/details.aspx?id=565... https://support.google.com/a/answer/10026322?product_name=UnuFlow&hl=en&visit_id=638605762510076874-1416434010&rd=1&src=supportwidget0&hl=en https://support.google.com/a/answer/10026322?product_name=Un... etc etc...

m3047 2y ago
I haven't checked today, but e.g. Amazon's JSON file belies the fact that they own 3.0.0.0/8 in practice if not in fact. I'm not going to play "let's block Amazon's huge JSON list". I've got 53 rules. I'm willing to step on some fingers / toes. I can get to sites in AWS just fine.
- oneplane 2y ago
  Amazon does have 3.0.0.0/8. Granted, it is noted as two separate /9 blocks, but it's all Amazon. Perhaps a better use of your time (because de-balkanization of the internet would be more of an academic exercise than a mass market reality; consumers have already self-selected into social media and crappy LLM outputs which is a far bigger matter and driver than ephemeral resources or reverse DNS) would be making your TLS work in a way that doesn't allow for easy abuse. So automatic redirects, HSTS, and a certificate that is standards-compliant.
  - huggingmouth 2y ago
    > consumers have already self-selected into ... crappy LLM outputs Tangent, but last I checked, it was all the big tech giants pushing it down our throat. Unless it's your full time job to find loopholes and workarounds, there is no reasonable way for consumers to opt-out.
- zamadatix 2y ago
  > I haven't checked today, but e.g. Amazon's JSON file belies the fact that they own 3.0.0.0/8 in practice if not in fact. While it may seem more useful to aggregate the ranges in some points of view it'd be significantly less useful from other points of view. E.g. those who want to whitelist any IP ranges matching a specific DC, service, availability zone, or country. You can always aggregate the detailed list but you can't do the inverse on your own.

UI_at_80x24 2y ago

I understand the appeal. If I wanted to wall-off large swaths of the internet from what I create I would do the same thing. It's not much different then blocking entire countries. The desire to limit the noise and only allow a "small circle of friends" is also appealing. But I do that for specific services, not my domains in general. Mumble server: only open to the 3-4 countries that my friends are in, and none of the 'cloud providers'. Tech blog: world+dog can see it. I am firmly in the 'We all benefit from shared knowledge' camp. So if my notes on modem init strings for my 300-baud C64 modem can help one other person; they won't go through the same pain I went through, and the world will be a better place. I get the desire, for many reasons. That's cool. You do you.

m3047 2y ago
I wrote the "Zen InterSLIP Dialing Script" and it literally went around the world. Amazon is too large to ignore. I understand that a lot of ICMP and SYN traffic is garbage. I'd be happy to help out and block it (and I do have mitigations in place); in fact I do, by default. That's part of why Amazon is a PITA because they trigger my mitigations "at scale". Amazon doesn't help sort the wheat from the chaff: "send a PCAP (for a ping issue)". I don't learn anything by sending Amazon stuff and hearing nothing. I don't need their good traffic any more than I need their bad traffic... or the traffic which is spoofed which is attacking them. If they can't see fit to help me help them, I don't need any of it. I'm just keeping my life simple.
- Dylan16807 2y ago
  Is the traffic causing you problems though? Or do you just dislike that they don't want to fix this?
  - m3047 2y ago
    Problems? What kind of problems? Discovering word embeddings was sort of a happy accident of LLMs: back propagation "improved" them from a similarly random starting point as the NNs. It's not a technical problem as in overwhelming any resource. It overwhelms me, for starters. Secondly my existing mitigations suggested it, I resisted the move for some of the reasons people here are saying it's a bad idea; I finally concluded the benefits outweighed the costs and I'd try it and see what happens. So far, so good. Once the fire burns out it should be great.

bdcravens 2y ago

In considering this, my first thought was it would block bona fide desktops running in AWS, especially for service offerings like Amazon Workspaces. However, it looks like the IP space for such services are publicly documented if the need arises to specifically whitelist those IPs. All that said, it's trivial to use proxies or VPNs to bypass any blocks.

Sammi 2y ago
> All that said, it's trivial to use proxies or VPNs to bypass any blocks. Maybe for you sure. The large drove of people flocking to serverless these days suggests that even most technical people don't want anything to do with their own networking or infrastructure.
- immibis 2y ago
  Proxy selling is a successful business (access on the order of 10 million residential IP addresses for on the order of $100 per month), and they don't say how they get access to those. Probably, the more people block access to non-residential IP addresses, the more money botnet vendors make.
  - bdcravens 2y ago
    At least among the "ethical" ones, my understanding is that it's freeware that has been packaged with proxy access (presumably disclosed to the end user, but that's a matter of interpretation)

xcdzvyn 2y ago

I'm slightly ashamed to admit I don't know what RPZ, PTR, or SPF are, nor do I understand the asides about AI or reverse DNS. What precisely is his problem with Amazon?

arcza 2y ago
Meaningless low-value and non-human traffic wasting resources and/or training models. Their blog is for real humans per the author's writing, which is a valid stance to take.

theideaofcoffee 2y ago

Yawn. Old man yells at cloud (literally). So he's taking his little netblock ball and going home because of some failed purity tests: bad or nonexistent PTRs, excessive ICMP, oh my! The gentleman's agreements that held together the early internet and web and the unwritten practices like that are long gone, get with the times, there ain't any going back to how it was. Otherwise, feel free to disconnect entirely if you don't want to deal with the new reality. I'm going to going out on a limb and guess that all of this traffic that isn't related directly to AWS, but its customers. You can set PTRs for your allocated elastic IPs with a request to support. But then again nobody is going to do it because... it doesn't matter. It may have mattered when you were hosting with a block that you actually truly owned, before the ICANN times, but no more. No one cares. Everything is ephemeral, so why should the reverse matter when things get cycled through addresses multiple times per day? If you're seeing excessive anything, then it's probably time to reach out to the abuse contact published in the whois. Let me help you with that: OrgAbuseHandle: AEA8-ARIN OrgAbuseName: Amazon EC2 Abuse OrgAbuseEmail: trustandsafety@support.aws.com OrgAbuseRef: https://rdap.arin.net/registry/entity/AEA8-ARIN Comment: All abuse reports MUST include: Comment: * src IP Comment: * dest IP (your IP) Comment: * dest port Comment: * Accurate date/timestamp and timezone of activity Comment: * Intensity/frequency (short log extracts) Comment: * Your contact details (phone and email) Without these we will be unable to identify the correct owner of the IP address at that point in time. Use modern features built in to modern versions of common packages and products: rate limiting, redirects, filters, and on and on. If you're just blocking to block to make some sort of statement into the void, you're just hastening that balkanization.

cdchn 2y ago
AWS doesn't mess around with abuse reports either. If you send a report and it checks out, they're opening a case with the customer in question telling them to explain themselves.
- hughesjj 2y ago
  But but that requires an async unidirectional communication with another human (the horror!)

issafram 2y ago

[flagged]

jaktet 2y ago
https://12ft.io/http://consulting.m3047.net/dubai-letters/balkanized-internet.html https://12ft.io/http://consulting.m3047.net/dubai-letters/ba...
- NetOpWibby 2y ago
  Bless
enjoyyourlife 2y ago
You can use Reader mode on Firefox
- dredmorbius 2y ago
  Not only does Reader Mode work beautifully, but, colour choice notwithstanding, the article's CSS styling is sufficiently vanilla that my own enhanced Reader Mode stylesheet (drop caps and a few other fiddly bits) works beautifully. That's often ... not ... the case.
huimang 2y ago
It's high contrast, better than a lot of websites. You'll be fine.
tomcam 2y ago
Reader mode is the way but tbh after the first 1500 words or so I didn’t know where it was going. I’m old and life is short,
- runxel 2y ago
  I agree. Horrible writing. For a while I thought it's just some LLM output. Not worth reading anyway, so you've lost nothing, OP.

User23 2y ago

This makes me miss the Internet. It’s really hard to explain how wonderful it was pre-commercialization. The sour pleasure of having been exactly right about how that would turn out isn’t nearly satisfactory compensation for the loss.

Yeul 2y ago
Ah yes nostalgic bullshit. It was always about the money.
- RGamma 2y ago
  Not with the fierceness and at the scale of today. Boiling frog and all.

tonetegeatinst 2y ago

Firefox on mobile, using dark mode. That font actually hurts my eye, and I straigup closed the site. Many I'll write a script to scrape the text into a terminal output so I can read it without feeling like I'm having a headache. Probably just output the text via a terminal windows.

Lucasoato 2y ago
From iOS Safari I see it pink on dark blue… nothing wrong there with these colors, actually they’re pleasant to look at. The font as well seems nice, is there anything wrong with it?
baq 2y ago
You don't need to do this, Firefox on mobile has a very good reader mode.

ThePhysicist 2y ago

It's crazy how fast people start attacking your infrastructure on the Internet these days. I recently started announcing one of my /23 subnets (512 addresses) over BGP for an anycast setup, once the route was announced and traffic flowing to the router tcpdump blew up with port scan activity on all IPs of the range. Of course that doesn't have anything to do with the route, it's just that tons of people seem to indiscriminately and continuously scan for open ports on all IP ranges (my range has been unannounced for many years before so it wasn't on someones list of active servers). I find it a shocking that people still expose internal web services (e.g. Gitlab) openly to the Internet, in my opinion you should at least have one additional layer of protection through a VPN or similar mechanism so that your services aren't discoverable from the public Internet. I only expose SSH from a single bastion host, which is the only host that's publicly reachable, something that I'd like to get rid off in the future as well by adding a VPN layer on top.

jcalvinowens 2y ago
> something that I'd like to get rid off in the future as well by adding a VPN layer on top. What VPN software would you use? Personally I've never found anything I consider as trustworthy as OpenSSH.
- throwaway63467 2y ago
  I use OpenVPN for historical reasons but today I’d go for Wireguard, much simpler, faster and integrated in the kernel, connectionless so much less friction when e.g. rebooting or changing networks.
- com 2y ago
  Wireguard is quite good too, and if you’re up for some complication in your life you can do full mesh quite easily with it if your online infra is a bit distributed.
  - RGamma 2y ago
    There's also helpers for wireguard meshes. Or become dependent on yet another service like tailscale (at least there's headscale) or zerotier.

rkagerer 2y ago

I did this a long time ago after dozens of Amazon servers started slowing down our on-prem servers. I think it might have been some kind of attempted SSO stuff but never did entirely track it down. Just wrote a script to periodically download a list of their IP ranges and block 'em all, and moved on.

perching_aix 2y ago

https is broken for the site

m3047 2y ago
Accept my signing certificate and try again. /s
- perching_aix 2y ago
  im good, thanks

opentokix 2y ago

[flagged]

aorth 2y ago

It has really gotten terrible. Between DDoS and bots from massive tech companies, it seems like I have several events a year where thousands or tens of thousands of IPs from a single datacenter (Singapore!) are making requests to some of my infrastructure concurrently. What can we do? I opted for CIDR aggregation and rate limiting of data center ISPs in nginx for one of my frontends. There are reasonable limits for normal IPs too. Not all of us have the capacity or desire to scale.

m3047 2y ago
This is the chauvinism I maybe failed to surface effectively enough. People in the cloud "just scale"; and big cuddly cloud says "ooooh, we block DoS and we won't charge you for the ingress traffic" when they whine. Whereas it really should be alleged DoS, it's not like I won't implement mitigations. I publish telemetry outing the worst offenders. What is Amazon doing for me, as a non-customer, to support our presumably shared goal? So maybe it's not really a shared goal. Here's an analogy which is very visceral for me: I have a chronic condition, which has never been "root caused" in my case by the school of western allopathic medicine. They put me on (several) drugs, I've taken them for a decade. Due to the ongoing enshittification of medical care over the past several years, I ended up having to see a naturopathic physician (with prescribing privileges, and not covered by my insurance) to get the prescriptions renewed because the western docs simply had better things to do than schedule and keep an appointment. Dude does the things a western doc should do, looks at the labs, and says with all seriousness: have you considered maybe you're allergic to drug "Y"? Holy fuck well that causes all kinds of problems identifying a replacement therapy, but that's not the point of the story here. Point is they've never done root cause, blamed the increasingly worse side effects on the condition they never root caused, put me on more drugs for the side effects, and told me to just live with it because that's what happens when you get older: this is what happens when you get trapped in an ecosystem. So I have identified a replacement therapy, it maybe doesn't work so well for the symptom drug Y was supposed to mitigate but it works. On the other hand, here are the side effects: constellation of symptoms of the actual condition (drug Y was intended to mitigate only one of them) has virtually disappeared, I sleep better, I have more energy, and I've lost nearly 20 pounds in the last six months. In summary, it's not that hard to run my own server (I have the skills, knowledge and specialized bespoke tools to support doing it my way). It keeps my professional skills sharp and gives me real practical intelligence about the fight in the streets.

jeroenhd 2y ago

I should probably do this for most of my stuff. I run some servers that require cloud-to-cloud networking, but the only inbound stuff I see coming from cloud services is bots, scanners, and scrapers. I've had to block off China's largest ISP because some broken scraper kept re-downloading the same image assets for no reason, and kept popping up on other subnets. I don't think anyone will miss my stuff if they're part of the small minority of people accessing the internet through a VPN hosted in large data centres. The biggest challenge for implementing this will probably be figuring out how to block inbound connections but keep outbound connections working. I'm sure there's a good nftables rule I can come up with eventually.

sulandor 2y ago
> The biggest challenge for implementing this will probably be figuring out how to block inbound connections but keep outbound connections working. nah, there's connection tracking for that. ctstate new srcip <amazon-set> drop
m3047 2y ago
For TCP you're just looking for packets having SYN without ACK. client server -------- SYN ---------> <-------- SYN+ACK -----

hello_computer 2y ago

AWS is a boy scout compared to places like DigitalOcean, OVHcloud, ColoCrossing, Scaleway, Tencent, or even Google. I think DigitalOcean, in particular, has made a terrible mistake marketing to the “cybersecurity” community.

m3047 2y ago
In fact (if you read the post) I mention blocking "some small hosting providers as a proof of concept": that would be DigitalOcean, for the reason you state.

arcza 2y ago

I've been blocking Hetzner, Digital Ocean, Linode, OVH and Contabo for a while. You can do this with pfBlocker NG by blocking ASNs, or UFW rules (https://blog.abctaylor.com/ufw-and-firewalld-rules-to-block-vps-bots-and-scrapers/ https://blog.abctaylor.com/ufw-and-firewalld-rules-to-block-...)

blueflow 2y ago
oof. Why Hetzner?
- arcza 2y ago
  Due to firewall logs showing DNS amplification attack attempts
  - Dylan16807 2y ago
    Why go beyond blocking direct DNS access? (Ideally you'd make then switch to TCP by truncating UDP responses to specific clients but that sounds like a hassle to set up so it's understandable to skip that.)
  - immibis 2y ago
    Everyone is attempting all attacks all the time from everywhere. Why not secure yourself so the attempts fail?
    - oneplane 2y ago
      At that point secure would be 'offline'... It's not like botnets, "unlocker" farms and P2P doesn't originate from residential netblocks all day long. The idea of "I just want the legitimate traffic" is a simple one, but the implementation of the idea has very little to do with "I will just block the big bad cloud!".
      - immibis 2y ago
        Securing yourself means not being vulnerable to the attacks. Who cares if you are exposed to an internet radiation banana equivalent? Why worry? You'll hurt yourself more from the worry than from the radiation. Blocking huge IP ranges is knocking yourself half offline, and it doesn't even stop you being "attacked". I'd start blocking if and only if there is some actual problem for your server (e.g. excessive CPU or bandwidth usage), not just because big bad scary cloud.

ruthmarx 2y ago

The cynic in me thinks this won't accomplish much. They can/do just buy data from other companies that scrape or some subsidiary. This isn't a technical problem, it's a legal/social problem.

nullc 2y ago

I've found that blocking china is also a pretty good improvement on abusive traffic relative to disruption.

cmeacham98 2y ago

Bias disclaimer: AWS is my current employer. Maybe I'm missing something obvious, but if the author believes the ping traffic is being spoofed, how could they know AWS is the source?

mrweasel 2y ago
AWS has a terrible reputation as being the source of absolute massive amounts of abuse and poorly written scrappers and crawler (hard to tell the difference between a bad crawler and an active attack). From experience I've seen AWS be the source of overwhelming traffic so many times that we in some cases resorted to the same solution, blocking AWS completely. I don't know if AWS doesn't care or is just slow to react. Maybe reporting is to difficult, I don't know. So the obvious you're missing is: AWS IS a huge source of "bad" traffic and getting a misbehaving customer shutdown is too hard, while renting insane amounts of capacity is too easy for bad actors. Edit: I've almost never seen GCP or Azure being the source of the same amount of crazy traffic.

knallfrosch 2y ago

Where's the summary? I didn't quite get what the problem is that you're trying to solve. Data scraping? DDOS attacks? Bandwidth trouble? Security?

Ellardo654 2y ago

[flagged]