8 ms·

I have done something a tad different. I am running my custom made proxy (dns blocking like pihole is a joke, you can circumvent it as simply as https://289990

by stiray 4y ago

I have done something a tad different. I am running my custom made proxy (dns blocking like pihole is a joke, you can circumvent it as simply as https://2899908462 https://2899908462 ) with interesting features like ASN ( https://en.wikipedia.org/wiki/Autonomous_system_(Internet) https://en.wikipedia.org/wiki/Autonomous_system_(Internet) ) blocking and for fun I have blocked all google ASNs. Half of the internet stopped working. Then I have expanded this to google, microsoft, amazon, facebook ASNs. The whole internet stopped working. From all search engines I am aware of, only yandex.ru was still operational. What you are seeing with beeping is just a tip of iceberg. The google is getting far more data from its cloud. I don't believe that most of advanced users are aware of, how deep the rabbit hole of internet centralization has gone. And until people figure out how important self hosting actually is, it is only going for worse (yeah, I understand how convenient, blah, blah... the cloud is).

deleted 4y ago
[deleted]
wombat-man 4y ago
Did the internet maybe stop working because you blocked all the major cloud providers? Nextdns (lazy man's pihole) does a decent job of adblock and seems to stop stuff like Braze tracking from working.
- stiray 4y ago
  Sure it did. But those are the data that google is getting, it is network transfer trough its servers. They are far beyond "tracking the clicks". And btw, no. The DNS blocking is stopping only domains. The primary addressing level is ip and you cant block it by preventing access to domains. But you can do it using proxy (or firewall, but typically firewall is clueless about the data layer), that is why I made it and while it can read the pihole blacklist and use them, it can do much more (like screwing js browser fingerprinting on network level by injecting js that randomizes the image creation etc.).
  - pedrocr 4y ago
    Google is not getting any data when you connect to a GCP VM through TLS. Or did I miss another huge PRISM-level scandal? Same for Azure and AWS. But if you block those no wonder a huge part of the internet is gone.
    - sp332 4y ago
      They know who owns the VM, and now they know that you connected, sent & received a certain amount of data at a certain time. [Edit: apparently this is more encrypted than I was thinking, so the next bit is probably wrong.] They could potentially look inside the VM to look at the specific data on the other side of TLS.
      - carlivar 4y ago
        Can anyone provide proof that Google records this (for Google, not as network telemetry for cloud customers)?
        metadat 4y ago
        Google Cloud would be negligent if they didn't collect information about ingress and egress traffic mapped to each of their tenants. Since they own the servers and network, it's on them to be able to investigate and track abuse.
        carlivar 4y ago
        Perhaps I phrased it poorly. The inference seems to be Google using this data for their gain (beyond operational needs). Is there any proof of that? Or is my inference incorrect?
      - cortesoft 4y ago
        Sure, and your ISP knows every single IP you have connected to, too. That is just how the internet works.
- Schroedingersat 4y ago
  > Did the internet maybe stop working because you blocked all the major cloud providers? You're saying this like it's not the point. Six or seven companies control the internet and have root and logs of all incoming traffic on ~all servers.
  - wombat-man 4y ago
    Yeah I mean I guess that's sort of a problem? No reason why people can't setup their own hardware other than cost/sloth. Cloud providers made themselves the best solution for a lot of services.
  - pedrocr 4y ago
    There's a huge difference between Google/Microsoft/Amazon having the possibility of accessing "root and logs" just as a side effect of running the service and the OPs assertion about them actively gathering information from that source. It would be a huge scandal if any of those cloud providers were caught peering inside their customer's VMs. Have I missed that scandal?
    - Schroedingersat 4y ago
      It's been a consistently effective strategy to assume any amount of surveillance or control they can implement without leaving evidence is already something they're doing.
    - Schroedingersat 4y ago
      Microsoft openly read every email and teams message and frequently leak the contents to bing and nobody cares. Windows laptops from OEMs frequently come with backdoored https implementations and nobody cared after the first one. Dell business laptops come with malware that sends videos and photos to ukraine and israel and nobody cares. It won't even be a blip when they are caught doing it.
    - nonrandomstring 4y ago
      > It would be a huge scandal if any of those cloud providers were caught peering inside their customer's VMs. No it wouldn't. At this point it could be revealed tomorrow by mountain of incontrovertible evidence and most people would shrug, move on and ask "what next?". Snowden. Shrug. Cisco backdoors. Shrug. Pegasus. Shrug. Solar winds. Shrug... The past decade can be described by the pattern "It would be a terrible scandal if X happened", and then precisely X happens. Then we normalise to it. > Have I missed that scandal? You may say it semi-sarcastically, but of course the irony is that actually you very well could have misse it. You only need to take a vacation for one week, a major shitstorm hits the front pages and fades from the news cycle. Now it's the "new normal". The important point is, you might never know. Without homomorphic encryption you simply have to trust entities that have the means, the motive, the opportunity and the track record for screwing you over.
decasteve 4y ago
Add Akamai to that list of ASNs and you cover everything. I did that experiment a couple of years ago and my Apple devices went unresponsive unless I disconnected them from the Internet entirely. The rabbit hole goes as far as locking yourself out of many government services. If you’re a Canadian, it means not being able to travel without submitting to corporate privacy policies and ToS (ArriveCan apps/website).
- sidewndr46 4y ago
  Unresponsive? As in the user interface stopped working?
  - foobiekr 4y ago
    Yeah, there are a lot of synchronous network calls in many applications. I ran out mobile bandwidth recently and have spent the last week in AT&T's penalty box and it's amazing what broke. For example, Audible's app goes to complete hell in this state when you launch it; you can bypass the hang by clicking library but it would be content to mostly just sit there.
  - Rebelgecko 4y ago
    I found a fun issue with the Apple Music app- if you're connected to wifi but NOT connected to the internet, it takes 60 seconds for the app to respond when you try to play something or change tracks.
    - Andrex 4y ago
      DNS issues are the worst. Everything in your stack reports having a "connection" but everything takes 60 seconds to not even load.
shadowgovt 4y ago
How important is self-hosting? Because as it stands, six or seven corporations control the majority of internet traffic and it seems to be fine. There have never been more users doing more things at once than right now. Self hosting is a huge undertaking and one really has to justify the value proposition, especially for people who want a presence on the internet but don't want their full-time job to be internet administration.
- beebeepka 4y ago
  Why is self hosting a huge undertaking compared to using the cloud? What are the major differences from a layperson view? Is spinning VMs on your hardware that much harder than doing it on other people's computers? It's an honest question because I haven't done both
  - shadowgovt 4y ago
    Yes. Cloud hosting puts out of the operator's space of concern actual maintenance of the hardware, which for most companies that want something like 24/7 uptime is a big deal. I worked at a small company where we planned for 24/7 uptime before clouds were ubiquitous. We planned out which of the three engineers in the team would hold the pager and the cost of gas reimbursements for them to drive one state over to deal with the machine in the secured rack facility if it physically went down. We didn't have nearly enough bandwidth capacity to our building itself to support the traffic we anticipated for our service. Smaller projects that don't require 24/7 uptime can be self-hosted, but you still want to be aware of fabric-layer security... If you're hosting on a machine in your building and somebody roots it, what will physical access to your intranet let them get away with? Can they see source code from there? Financials? Employee database? All of this is less a concern if you're using AWS with separated instances that are no more connected to each other than Netflix is to Disney+, even if they're in the same building. Cloud hosting lets you focus on the software and credentials and pay someone else to focus on hardware and application of credentials.
  - BigRedDog1669 4y ago
    Yes, it's not just clicking a "create VM" button or setting up a box in the closet. It's setting up the networking and security so that only the right people can access it and the packages are updated etc. And being the support person if anyone besides yourself is using it.
_int3_ 4y ago
Maybe next step would be to make a tool which says if given website is clean or not? So we can start making whitelist.
- ranger_danger 4y ago
  please consider a more inclusive term such as allowlist.
suction 4y ago
Eduard 4y ago
Huh, what is https://2899908462 https://2899908462 ?
- TrinaryWorksToo 4y ago
  A different format for writing ip addresses
  - Eduard 4y ago
    Ah okay, I got it: it's the decimal representation of the IP address. https://www.lookip.net/ip/172.217.23.110 https://www.lookip.net/ip/172.217.23.110 Funny I wasn't aware of this despite 25 years of being an Internet user
    - blooalien 4y ago
      > "Funny I wasn't aware of this despite 25 years of being an Internet user" You're not alone… I been using the Internet longer'n that and I didn't ever think of converting an IP address to a decimal number either. It makes perfect sense now that it's been pointed out, but for some reason it just never crossed my mind to even try it.
      - simonh 4y ago
        To be fair, there are many different bases. I wonder if there’s anything special about base 10 that made them support it.
        mlyle 4y ago
        IPv4 addresses are written in base 10 for human users. The "special" case is supporting network addresses to be written as 10.2932832 ("convenient" for class A's), 172.16.61031 (ditto for B's), or just one big address like 39282329, when we're used to 4 octets separated by dots. Not every bit of host software supports these cases anymore, as basically their sole remaining use case is as a curiosity or to circumvent bad security controls.
deleted 4y ago
[deleted]
soheil 4y ago
> only yandex.ru was still operational. Maybe having Russia not being part of the same monopoly is actually a good thing for the internet.
jrockway 4y ago
What's interesting to me is how split into node type the Internet is. All the servers are Google/AWS/Azure, but there are almost no legitimate clients on those ASNs. I've also done AS blocking (preventing certain IPs from getting a free compute trial without human intervention; this was back in the crypto days), and indeed, blocking the networks that you did is great for getting rid of bots and harms 0 legitimate users. (I think the big culprit is "free for open source" CI systems; they tend to be hosted on the major cloud providers and I found those doing a lot of command-and-control. I'm surprised those are viable to keep around for free, though.)
freeflight 4y ago
> Then I have expanded this to google, microsoft, amazon, facebook ASNs. > The whole internet stopped working. Not that surprising, considering these companies already had direct influence over 70%+ of internet traffic back in 2014 [0] By now that number is probably in the 80-90% range as it's a problem the vast majority of people are either completely unaware of, or sometimes deny it to be even a problem in the first place. [0] https://staltz.com/the-web-began-dying-in-2014-heres-how.html https://staltz.com/the-web-began-dying-in-2014-heres-how.htm...
thomassmith65 4y ago
Then I have expanded this to google, microsoft, amazon, facebook ASNs. The whole internet stopped working. Neither am I delighted to have my machine talk to Cloudflare or Akamai (per https://news.ycombinator.com/item?id=32618098 https://news.ycombinator.com/item?id=32618098). Add to that a few font, framework services and ad delivery networks and why bother having any independent servers at all? For decent people there is no privacy, but, for malicious people, it is more than enough privacy to avoid accountability. That's no surprise today, but who would have looked forward to a net like this back in the 1990's? Just scrap everything and start again. What we have now is a failed experiment.
- nikau 4y ago
  > Neither am I delighted to have my machine talk to Cloudflare or Akamai what's your concern around CDNs? Its not like its viable for every company to put in 1000s of edge nodes to ensure every area of every country has a good website/app experience.
  - thomassmith65 4y ago
    Its not like its viable [...] I agree. That's why I ended by writing "Just scrap everything and start again. What we have now is a failed experiment"
ranger_danger 4y ago
>you can circumvent it as simply as except no company actually does this

by stiray 4y ago

deleted 4y ago

[deleted]

wombat-man 4y ago

Did the internet maybe stop working because you blocked all the major cloud providers? Nextdns (lazy man's pihole) does a decent job of adblock and seems to stop stuff like Braze tracking from working.

stiray 4y ago
Sure it did. But those are the data that google is getting, it is network transfer trough its servers. They are far beyond "tracking the clicks". And btw, no. The DNS blocking is stopping only domains. The primary addressing level is ip and you cant block it by preventing access to domains. But you can do it using proxy (or firewall, but typically firewall is clueless about the data layer), that is why I made it and while it can read the pihole blacklist and use them, it can do much more (like screwing js browser fingerprinting on network level by injecting js that randomizes the image creation etc.).
- pedrocr 4y ago
  Google is not getting any data when you connect to a GCP VM through TLS. Or did I miss another huge PRISM-level scandal? Same for Azure and AWS. But if you block those no wonder a huge part of the internet is gone.
  - sp332 4y ago
    They know who owns the VM, and now they know that you connected, sent & received a certain amount of data at a certain time. [Edit: apparently this is more encrypted than I was thinking, so the next bit is probably wrong.] They could potentially look inside the VM to look at the specific data on the other side of TLS.
    - carlivar 4y ago
      Can anyone provide proof that Google records this (for Google, not as network telemetry for cloud customers)?
      - metadat 4y ago
        Google Cloud would be negligent if they didn't collect information about ingress and egress traffic mapped to each of their tenants. Since they own the servers and network, it's on them to be able to investigate and track abuse.
        carlivar 4y ago
        Perhaps I phrased it poorly. The inference seems to be Google using this data for their gain (beyond operational needs). Is there any proof of that? Or is my inference incorrect?
    - cortesoft 4y ago
      Sure, and your ISP knows every single IP you have connected to, too. That is just how the internet works.
Schroedingersat 4y ago
> Did the internet maybe stop working because you blocked all the major cloud providers? You're saying this like it's not the point. Six or seven companies control the internet and have root and logs of all incoming traffic on ~all servers.
- wombat-man 4y ago
  Yeah I mean I guess that's sort of a problem? No reason why people can't setup their own hardware other than cost/sloth. Cloud providers made themselves the best solution for a lot of services.
- pedrocr 4y ago
  There's a huge difference between Google/Microsoft/Amazon having the possibility of accessing "root and logs" just as a side effect of running the service and the OPs assertion about them actively gathering information from that source. It would be a huge scandal if any of those cloud providers were caught peering inside their customer's VMs. Have I missed that scandal?
  - Schroedingersat 4y ago
    It's been a consistently effective strategy to assume any amount of surveillance or control they can implement without leaving evidence is already something they're doing.
  - Schroedingersat 4y ago
    Microsoft openly read every email and teams message and frequently leak the contents to bing and nobody cares. Windows laptops from OEMs frequently come with backdoored https implementations and nobody cared after the first one. Dell business laptops come with malware that sends videos and photos to ukraine and israel and nobody cares. It won't even be a blip when they are caught doing it.
  - nonrandomstring 4y ago
    > It would be a huge scandal if any of those cloud providers were caught peering inside their customer's VMs. No it wouldn't. At this point it could be revealed tomorrow by mountain of incontrovertible evidence and most people would shrug, move on and ask "what next?". Snowden. Shrug. Cisco backdoors. Shrug. Pegasus. Shrug. Solar winds. Shrug... The past decade can be described by the pattern "It would be a terrible scandal if X happened", and then precisely X happens. Then we normalise to it. > Have I missed that scandal? You may say it semi-sarcastically, but of course the irony is that actually you very well could have misse it. You only need to take a vacation for one week, a major shitstorm hits the front pages and fades from the news cycle. Now it's the "new normal". The important point is, you might never know. Without homomorphic encryption you simply have to trust entities that have the means, the motive, the opportunity and the track record for screwing you over.

decasteve 4y ago

Add Akamai to that list of ASNs and you cover everything. I did that experiment a couple of years ago and my Apple devices went unresponsive unless I disconnected them from the Internet entirely. The rabbit hole goes as far as locking yourself out of many government services. If you’re a Canadian, it means not being able to travel without submitting to corporate privacy policies and ToS (ArriveCan apps/website).

sidewndr46 4y ago
Unresponsive? As in the user interface stopped working?
- foobiekr 4y ago
  Yeah, there are a lot of synchronous network calls in many applications. I ran out mobile bandwidth recently and have spent the last week in AT&T's penalty box and it's amazing what broke. For example, Audible's app goes to complete hell in this state when you launch it; you can bypass the hang by clicking library but it would be content to mostly just sit there.
- Rebelgecko 4y ago
  I found a fun issue with the Apple Music app- if you're connected to wifi but NOT connected to the internet, it takes 60 seconds for the app to respond when you try to play something or change tracks.
  - Andrex 4y ago
    DNS issues are the worst. Everything in your stack reports having a "connection" but everything takes 60 seconds to not even load.

shadowgovt 4y ago

How important is self-hosting? Because as it stands, six or seven corporations control the majority of internet traffic and it seems to be fine. There have never been more users doing more things at once than right now. Self hosting is a huge undertaking and one really has to justify the value proposition, especially for people who want a presence on the internet but don't want their full-time job to be internet administration.

beebeepka 4y ago
Why is self hosting a huge undertaking compared to using the cloud? What are the major differences from a layperson view? Is spinning VMs on your hardware that much harder than doing it on other people's computers? It's an honest question because I haven't done both
- shadowgovt 4y ago
  Yes. Cloud hosting puts out of the operator's space of concern actual maintenance of the hardware, which for most companies that want something like 24/7 uptime is a big deal. I worked at a small company where we planned for 24/7 uptime before clouds were ubiquitous. We planned out which of the three engineers in the team would hold the pager and the cost of gas reimbursements for them to drive one state over to deal with the machine in the secured rack facility if it physically went down. We didn't have nearly enough bandwidth capacity to our building itself to support the traffic we anticipated for our service. Smaller projects that don't require 24/7 uptime can be self-hosted, but you still want to be aware of fabric-layer security... If you're hosting on a machine in your building and somebody roots it, what will physical access to your intranet let them get away with? Can they see source code from there? Financials? Employee database? All of this is less a concern if you're using AWS with separated instances that are no more connected to each other than Netflix is to Disney+, even if they're in the same building. Cloud hosting lets you focus on the software and credentials and pay someone else to focus on hardware and application of credentials.
- BigRedDog1669 4y ago
  Yes, it's not just clicking a "create VM" button or setting up a box in the closet. It's setting up the networking and security so that only the right people can access it and the packages are updated etc. And being the support person if anyone besides yourself is using it.

_int3_ 4y ago

Maybe next step would be to make a tool which says if given website is clean or not? So we can start making whitelist.

ranger_danger 4y ago
please consider a more inclusive term such as allowlist.

suction 4y ago

Eduard 4y ago

Huh, what is https://2899908462 https://2899908462 ?

TrinaryWorksToo 4y ago
A different format for writing ip addresses
- Eduard 4y ago
  Ah okay, I got it: it's the decimal representation of the IP address. https://www.lookip.net/ip/172.217.23.110 https://www.lookip.net/ip/172.217.23.110 Funny I wasn't aware of this despite 25 years of being an Internet user
  - blooalien 4y ago
    > "Funny I wasn't aware of this despite 25 years of being an Internet user" You're not alone… I been using the Internet longer'n that and I didn't ever think of converting an IP address to a decimal number either. It makes perfect sense now that it's been pointed out, but for some reason it just never crossed my mind to even try it.
    - simonh 4y ago
      To be fair, there are many different bases. I wonder if there’s anything special about base 10 that made them support it.
      - mlyle 4y ago
        IPv4 addresses are written in base 10 for human users. The "special" case is supporting network addresses to be written as 10.2932832 ("convenient" for class A's), 172.16.61031 (ditto for B's), or just one big address like 39282329, when we're used to 4 octets separated by dots. Not every bit of host software supports these cases anymore, as basically their sole remaining use case is as a curiosity or to circumvent bad security controls.

deleted 4y ago

[deleted]

soheil 4y ago

> only yandex.ru was still operational. Maybe having Russia not being part of the same monopoly is actually a good thing for the internet.

jrockway 4y ago

What's interesting to me is how split into node type the Internet is. All the servers are Google/AWS/Azure, but there are almost no legitimate clients on those ASNs. I've also done AS blocking (preventing certain IPs from getting a free compute trial without human intervention; this was back in the crypto days), and indeed, blocking the networks that you did is great for getting rid of bots and harms 0 legitimate users. (I think the big culprit is "free for open source" CI systems; they tend to be hosted on the major cloud providers and I found those doing a lot of command-and-control. I'm surprised those are viable to keep around for free, though.)

freeflight 4y ago

> Then I have expanded this to google, microsoft, amazon, facebook ASNs. > The whole internet stopped working. Not that surprising, considering these companies already had direct influence over 70%+ of internet traffic back in 2014 [0] By now that number is probably in the 80-90% range as it's a problem the vast majority of people are either completely unaware of, or sometimes deny it to be even a problem in the first place. [0] https://staltz.com/the-web-began-dying-in-2014-heres-how.html https://staltz.com/the-web-began-dying-in-2014-heres-how.htm...

thomassmith65 4y ago

Then I have expanded this to google, microsoft, amazon, facebook ASNs. The whole internet stopped working. Neither am I delighted to have my machine talk to Cloudflare or Akamai (per https://news.ycombinator.com/item?id=32618098 https://news.ycombinator.com/item?id=32618098). Add to that a few font, framework services and ad delivery networks and why bother having any independent servers at all? For decent people there is no privacy, but, for malicious people, it is more than enough privacy to avoid accountability. That's no surprise today, but who would have looked forward to a net like this back in the 1990's? Just scrap everything and start again. What we have now is a failed experiment.

nikau 4y ago
> Neither am I delighted to have my machine talk to Cloudflare or Akamai what's your concern around CDNs? Its not like its viable for every company to put in 1000s of edge nodes to ensure every area of every country has a good website/app experience.
- thomassmith65 4y ago
  Its not like its viable [...] I agree. That's why I ended by writing "Just scrap everything and start again. What we have now is a failed experiment"

ranger_danger 4y ago

>you can circumvent it as simply as except no company actually does this