5 ms·

Creating an Intranet Like the Internet

76 points by marcodiego 4y ago

_int3_ 4y ago
> Some people self-host music. I'm trying to self-host my own Internet. Me too . Haven't got far. Yet
- EGreg 4y ago
  Consider qbix.com/platform
captn3m0 4y ago
You can use the X.509 Name Constraints extension to limit the power of your CA to just a single TLD. I personally just run on a public TLD with a wildcard cert (so my domains don't show up in cert transparency), but there is an alternative. - https://www.sysadmins.lv/blog-en/x509-name-constraints-certificate-extension-all-you-should-know.aspx https://www.sysadmins.lv/blog-en/x509-name-constraints-certi... - https://www.rfc-editor.org/rfc/rfc5280#section-4.2.1.10 https://www.rfc-editor.org/rfc/rfc5280#section-4.2.1.10 Apparently (I haven't tested this), Chrome only checks this on intermediate certs not root certs but you can mitigate around that by keeping your root cert offline. Will only work on "modern" browsers and operating systems. https://bugs.chromium.org/p/chromium/issues/detail?id=1072083 https://bugs.chromium.org/p/chromium/issues/detail?id=107208...
- bo0tzz 4y ago
  > you can mitigate around that by keeping your root cert offline. This is something you should probably do anyways.
- xoa 4y ago
  I use that extensively on a bunch of private CAs, know though that it can sometimes be a touch quirky. For example, certain applications/libraries will apparently look for anything that "looks like it has a domain in it" in the cert and check it against the constraint, so for example if you have a root CA with a constraint to only abc.com but then have the admin contact email set as admin@xyz.com it'll complain even though browsers are fine. This is clearly a bug but it's been floating around for a long time (though I haven't checked in a while granted). And I always do [root ca] > [intermediate online ca] > [more specific constrained cas if required] with root kept fully offline and the intermediate one used for online signing convenience. Overall I think doing one's own CA is unnecessarily irritating to get into, plenty of very strange behavior in openssl and how everything interacts, irritating to learn and keep up with best practices, and of course it's a lot harder to get things trusted for a domain that you own then it should be. I wish the whole ecosystem there was a lot better because it's one of the key bits of gatekeeping/centralization/privacy. Let's Encrypt helps but is specifically for public websites, not other usage. Even so it's still very useful to have that power and flexibility for nothing, and there are capabilities in extensions to help enforce better practices.
- ithkuil 4y ago
  And MacOS supports them finally since High Sierra
philipwhiuk 4y ago
This sounds like a new dark web, so I assume they will eventually have the same problems that does.
- dsr_ 4y ago
  To me, this sounds like a cooperative. As such, it will thrive on the efforts of its members and be poisoned by personal abuse. It could survive for decades or be abandoned after the second or third crisis. But having an infrastructure plan and documentation for a cooperative would be a valuable contribution to the whole world.
  - lotw_dot_site 4y ago
    Yes, a cooperative, or even a tech-savvy intentional community. That is the kind of thing I'm really going for with "Linux on the Web". Something like a new hippie age must start arising as the post-peak-oil era continues to take its toll on the global economy*. It's now up to the younger Millenials and older Zoomers to start leading the way with innovative, sustainable, future-forward ways of living. *https://www.theguardian.com/environment/2021/jul/25/gaya-herrington-mit-study-the-limits-to-growth https://www.theguardian.com/environment/2021/jul/25/gaya-her...
- lotw_dot_site 4y ago
  If it's truly an intranet then it is physically disconnected from the public internet, and would not be merely a "dark web", but more like a "black hole web". But if they are trying to run public services, then your assumption is correct. All in all, though, it sounds more like a bunch of college friends who are just trying to find their bearings in the wild, huge, crazy world of tech.
  - thetruedoormat 4y ago
    Hey, author of that (terrible) article here. The only "public" service I run is a Shynet instance for some friends. We're just a bunch of secondary or uni students wanting to still be able to do things we like if Cloudflare decides to turn turtle.
evol262 4y ago
This is a great idea, but the author is making a mistake (well, "all eggs in the k8s/k3s basket is also a mistake") in banking on CoreDNS. BIND is THE reference DNS server. It is the RFC, and the k8s external-dns sig to dynamically create record for services is the right way to go here, with delegation of some subdomain to k8s if you really want to. BIND's ocnfiguration is a relatively tiny text file and some RNDC keys. You could practically run it on on an Arduino, and adding secondaries is mindblowingly simple. BIND supports every single part of the DNS spec. BIND will work with k8s/k3s plus whatever infrastructure is added later, out of the box, with no changes, in plain text files. Drive failure? Just grab stuff from git. It is 100% the right solution. K8s/k3s are fine, but the authors are dramatically overestimating how far "we admin some websites" is going to take them during major version changes changes which adjust the arguments to kubelet/kube-admin, tryint to get some legacy/future software working, etc. Either use Kubevirt or, if I were starting this in 2022, I'd use a system like Nomad as the base and delegate things which belong in k8s to k8s so your core infrastructure doesn't depend on the hip tech. Core infrastructure should be boring, stable, and "just work". PSQL is great. BIND and isc-dhcpd on whatever embedded boards you feel like will run forever, the config will probably never have breaking compatibility changes, and integrate with everything. Layer other things on top of that. Ceph/Rook are great. Until you run out of storage. Shared nothing for DNS/dhcpd (let them handle transferring themselves). If you need "real" shared storage, pick whichever one you think you can recover when it catastrophically fails, because it WILL catastrophically fail. Ceph will run out of free inodes or the ratio will get too high and nodes will fail to start until you add capacity or recover manually with ceph-osd/ceph-bluestore-tool/etc. Like k8s, don't select tools which are built to be monitored/managed by a team of dedicated people for your more or less hobby project in your free time. Pick something boring and stable. If you want to build it like the internet, run it like the internet. The SSL concerns can easily be mitigated. So can whatever weird TLD issues this is talking about with appending "/". Host a private DNS namespace. This has been done for decades. Your DNS servers (BIND) will be authoritative. Users of your network will need to disable DNS-over-http. Or make BIND do it. As mentioned, use Name Constraints for the CA.
- _fp3j 4y ago
  Author of that (terrible) article here. The blunt of this article written at (probably) 1 in the morning is something I think either I failed to mention or should have made more clear: for most of this, we just want to side-skirt the major outages of people like Cloudflare. For the few like myself, this is a breeding ground to experiment. Right now, I don't have the energy to convert services I maintain that are public to other platforms. It would mean significant downtime just to learn. However, Farer lets me play around and see what works, K8s/K3s being a prime example. > making a mistake ... in banking on CoreDNS Well, I'll start by quoting the article: "Sanely hosting services 101 ... The first step is to not be us ... we created a lot of complexity that was generally [un]necesssary." You are 100% right in saying that BIND is more than likely a better solution than CoreDNS. And although some of it is "well, I found this that should work," it sometimes boils deeper, namely in the case of CoreDNS. It's something included in every pod of K3s and makes sense to get familiar with; which loops back to the start of wanting to learn how things work. tl;dr : I appreciate the insights! Things are done for: a reason, for fun/experiment, or no reason. The Internet has some expected uptimes (not to say we don't for our intranet), but considering the small circle of trust, we aren't as constrained to the angry fist-waving when it goes down, rather we're all trying to figure out what happened, how to fix it, and how to prevent it from being an issue again: a fun "practicum," per se.
thetruedoormat 4y ago
Author of the (terrible) article here. I woke up to messages from people in Farer saying "Congrats, you're on the front page of HN" which was an interesting way to start my day considering how I am not awake still. I wanted to thank everyone for the advice you've been putting out; all of us have been looking into basically all of it and seeing what we want to go moving forward based on it. A few people have asked if documentation and an infra plan would ever be public. Right now, we're in such an early state of spinning around and seeing what works best not only for us, but for future us in 5-10yrs. Why should we waste time implementing service X if it's going to be a PITA to maintain; service Y is plenty decent and fits the same criteria? It's a matter of the seemingly easy decision between service X and Y. Sometimes we go with Y because that makes sense, other times we go with X because it (somehow) makes more sense. All I can say is keep adding your two-cents! It's helping us think even more about what needs to be done, and we really appreciate all the help so far. I (never) check this website, but I'll keep looking here throughout the day to see if there's something more. My email (should) be at the bottom of that article as well if you want to make sure I see what you have to say. Cheers!
- dicknuckle 4y ago
  What is Farer?
  - dsr_ 4y ago
    It's the subject organization of the article.

76 points by marcodiego 4y ago

_int3_ 4y ago

> Some people self-host music. I'm trying to self-host my own Internet. Me too . Haven't got far. Yet

EGreg 4y ago
Consider qbix.com/platform

captn3m0 4y ago

You can use the X.509 Name Constraints extension to limit the power of your CA to just a single TLD. I personally just run on a public TLD with a wildcard cert (so my domains don't show up in cert transparency), but there is an alternative. - https://www.sysadmins.lv/blog-en/x509-name-constraints-certificate-extension-all-you-should-know.aspx https://www.sysadmins.lv/blog-en/x509-name-constraints-certi... - https://www.rfc-editor.org/rfc/rfc5280#section-4.2.1.10 https://www.rfc-editor.org/rfc/rfc5280#section-4.2.1.10 Apparently (I haven't tested this), Chrome only checks this on intermediate certs not root certs but you can mitigate around that by keeping your root cert offline. Will only work on "modern" browsers and operating systems. https://bugs.chromium.org/p/chromium/issues/detail?id=1072083 https://bugs.chromium.org/p/chromium/issues/detail?id=107208...

bo0tzz 4y ago
> you can mitigate around that by keeping your root cert offline. This is something you should probably do anyways.
xoa 4y ago
I use that extensively on a bunch of private CAs, know though that it can sometimes be a touch quirky. For example, certain applications/libraries will apparently look for anything that "looks like it has a domain in it" in the cert and check it against the constraint, so for example if you have a root CA with a constraint to only abc.com but then have the admin contact email set as admin@xyz.com it'll complain even though browsers are fine. This is clearly a bug but it's been floating around for a long time (though I haven't checked in a while granted). And I always do [root ca] > [intermediate online ca] > [more specific constrained cas if required] with root kept fully offline and the intermediate one used for online signing convenience. Overall I think doing one's own CA is unnecessarily irritating to get into, plenty of very strange behavior in openssl and how everything interacts, irritating to learn and keep up with best practices, and of course it's a lot harder to get things trusted for a domain that you own then it should be. I wish the whole ecosystem there was a lot better because it's one of the key bits of gatekeeping/centralization/privacy. Let's Encrypt helps but is specifically for public websites, not other usage. Even so it's still very useful to have that power and flexibility for nothing, and there are capabilities in extensions to help enforce better practices.
ithkuil 4y ago
And MacOS supports them finally since High Sierra

philipwhiuk 4y ago

This sounds like a new dark web, so I assume they will eventually have the same problems that does.

dsr_ 4y ago
To me, this sounds like a cooperative. As such, it will thrive on the efforts of its members and be poisoned by personal abuse. It could survive for decades or be abandoned after the second or third crisis. But having an infrastructure plan and documentation for a cooperative would be a valuable contribution to the whole world.
- lotw_dot_site 4y ago
  Yes, a cooperative, or even a tech-savvy intentional community. That is the kind of thing I'm really going for with "Linux on the Web". Something like a new hippie age must start arising as the post-peak-oil era continues to take its toll on the global economy*. It's now up to the younger Millenials and older Zoomers to start leading the way with innovative, sustainable, future-forward ways of living. *https://www.theguardian.com/environment/2021/jul/25/gaya-herrington-mit-study-the-limits-to-growth https://www.theguardian.com/environment/2021/jul/25/gaya-her...
lotw_dot_site 4y ago
If it's truly an intranet then it is physically disconnected from the public internet, and would not be merely a "dark web", but more like a "black hole web". But if they are trying to run public services, then your assumption is correct. All in all, though, it sounds more like a bunch of college friends who are just trying to find their bearings in the wild, huge, crazy world of tech.
- thetruedoormat 4y ago
  Hey, author of that (terrible) article here. The only "public" service I run is a Shynet instance for some friends. We're just a bunch of secondary or uni students wanting to still be able to do things we like if Cloudflare decides to turn turtle.

evol262 4y ago

This is a great idea, but the author is making a mistake (well, "all eggs in the k8s/k3s basket is also a mistake") in banking on CoreDNS. BIND is THE reference DNS server. It is the RFC, and the k8s external-dns sig to dynamically create record for services is the right way to go here, with delegation of some subdomain to k8s if you really want to. BIND's ocnfiguration is a relatively tiny text file and some RNDC keys. You could practically run it on on an Arduino, and adding secondaries is mindblowingly simple. BIND supports every single part of the DNS spec. BIND will work with k8s/k3s plus whatever infrastructure is added later, out of the box, with no changes, in plain text files. Drive failure? Just grab stuff from git. It is 100% the right solution. K8s/k3s are fine, but the authors are dramatically overestimating how far "we admin some websites" is going to take them during major version changes changes which adjust the arguments to kubelet/kube-admin, tryint to get some legacy/future software working, etc. Either use Kubevirt or, if I were starting this in 2022, I'd use a system like Nomad as the base and delegate things which belong in k8s to k8s so your core infrastructure doesn't depend on the hip tech. Core infrastructure should be boring, stable, and "just work". PSQL is great. BIND and isc-dhcpd on whatever embedded boards you feel like will run forever, the config will probably never have breaking compatibility changes, and integrate with everything. Layer other things on top of that. Ceph/Rook are great. Until you run out of storage. Shared nothing for DNS/dhcpd (let them handle transferring themselves). If you need "real" shared storage, pick whichever one you think you can recover when it catastrophically fails, because it WILL catastrophically fail. Ceph will run out of free inodes or the ratio will get too high and nodes will fail to start until you add capacity or recover manually with ceph-osd/ceph-bluestore-tool/etc. Like k8s, don't select tools which are built to be monitored/managed by a team of dedicated people for your more or less hobby project in your free time. Pick something boring and stable. If you want to build it like the internet, run it like the internet. The SSL concerns can easily be mitigated. So can whatever weird TLD issues this is talking about with appending "/". Host a private DNS namespace. This has been done for decades. Your DNS servers (BIND) will be authoritative. Users of your network will need to disable DNS-over-http. Or make BIND do it. As mentioned, use Name Constraints for the CA.

_fp3j 4y ago
Author of that (terrible) article here. The blunt of this article written at (probably) 1 in the morning is something I think either I failed to mention or should have made more clear: for most of this, we just want to side-skirt the major outages of people like Cloudflare. For the few like myself, this is a breeding ground to experiment. Right now, I don't have the energy to convert services I maintain that are public to other platforms. It would mean significant downtime just to learn. However, Farer lets me play around and see what works, K8s/K3s being a prime example. > making a mistake ... in banking on CoreDNS Well, I'll start by quoting the article: "Sanely hosting services 101 ... The first step is to not be us ... we created a lot of complexity that was generally [un]necesssary." You are 100% right in saying that BIND is more than likely a better solution than CoreDNS. And although some of it is "well, I found this that should work," it sometimes boils deeper, namely in the case of CoreDNS. It's something included in every pod of K3s and makes sense to get familiar with; which loops back to the start of wanting to learn how things work. tl;dr : I appreciate the insights! Things are done for: a reason, for fun/experiment, or no reason. The Internet has some expected uptimes (not to say we don't for our intranet), but considering the small circle of trust, we aren't as constrained to the angry fist-waving when it goes down, rather we're all trying to figure out what happened, how to fix it, and how to prevent it from being an issue again: a fun "practicum," per se.

thetruedoormat 4y ago

Author of the (terrible) article here. I woke up to messages from people in Farer saying "Congrats, you're on the front page of HN" which was an interesting way to start my day considering how I am not awake still. I wanted to thank everyone for the advice you've been putting out; all of us have been looking into basically all of it and seeing what we want to go moving forward based on it. A few people have asked if documentation and an infra plan would ever be public. Right now, we're in such an early state of spinning around and seeing what works best not only for us, but for future us in 5-10yrs. Why should we waste time implementing service X if it's going to be a PITA to maintain; service Y is plenty decent and fits the same criteria? It's a matter of the seemingly easy decision between service X and Y. Sometimes we go with Y because that makes sense, other times we go with X because it (somehow) makes more sense. All I can say is keep adding your two-cents! It's helping us think even more about what needs to be done, and we really appreciate all the help so far. I (never) check this website, but I'll keep looking here throughout the day to see if there's something more. My email (should) be at the bottom of that article as well if you want to make sure I see what you have to say. Cheers!

dicknuckle 4y ago
What is Farer?
- dsr_ 4y ago
  It's the subject organization of the article.