14 ms·

Self-Host and Tech Independence: The Joy of Building Your Own

499 points by articsputnik 1y ago

larodi 1y ago
Can definitely become a trend given so many devs out there and so much that AI can produce at home which can be of arbitrary code quality…
ryandrake 1y ago
> The premise is that by learning some of the fundamentals, in this case Linux, you can host most things yourself. Not because you need to, but because you want to, and the feeling of using your own services just gives you pleasure. And you learn from it. Not only that, but it helps to eliminate the very real risk that you get kicked off of a platform that you depend on without recourse. Imagine if you lost your Gmail account. I'd bet that most normies would be in deep shit, since that's basically their identity online, and they need it to reset passwords and maybe even to log into things. I bet there are a non-zero number of HN commenters who would be fucked if they so much as lost their Gmail account. You've got to at least own your own E-mail identity! Rinse and repeat for every other online service you depend on. What if your web host suddenly deleted you? Or AWS? Or Spotify or Netflix? Or some other cloud service? What's your backup? If your answer is "a new cloud host" you're just trading identical problems.
- ozim 1y ago
  Self hosting at home - what is higher risk? Your HDD dying or losing Gmail account? Oh now you don’t only self host, now you have to have space to keep gear, plan backups, install updates, oh would be good to test updates so some bug doesn’t mess your system. Oh you know installing updates or while backups are running it would be bad if you have power outage- now you need a UPS. Oh you know what - my UPS turned out to be faulty and it f-up my HDD in my NAS. No I don’t have time to deal with any of it anymore I have other things to do with my life ;)
  - layoric 1y ago
    Different strokes for different folks. Motivation for me has been a combination of independence and mistrust. Every single one of the larger tech companies have shown their priority to growth above making good products and services, and not being directly user hostile. Google search is worse now than it was 10 years ago. Netflix has ads with a paid subscription, so does YouTube. Windows is absolute joke, more and more we see user hostile software. Incentives aren’t aligned at all. As people who work in software, I get not wanting to do this stuff at home as well. But honestly I’m hoping for a future where a lot of these services can legit be self hosted by technical people for their local communities. Mastodon is doing this really well IMO. Self hosted software is also getting a lot easier to manage, so I’m quite optimistic that things will keep heading this way. Note, I’ve got all the things you mentioned down to the UPSes setup in my garage, as well as multiple levels of backups. It’s not perfect, but works for me without much time input vs utility it provides. Each to their own.
    - deadbabe 1y ago
      If your trust is violated, typically the worst that happens is you are fed a couple more relevant ads or your data is used for some commercial purpose that has little to no effect on your life. Is it really worth going through so much effort to mitigate that risk?
      - layoric 1y ago
        Again, it's a value judgement, so the answer is largely personal. For me, yes. The social license we give these larger companies after all the violated trust doesn't make sense. If your local shop owner/operator that you talked to everyday had the same attitude towards your when you went shopping and exchanged pleasantries with most weeks, people would confront them about their actions, and that shop wouldn't last long. We have created the disconnect for convenience, and tried to ignore the level of control these companies have on our day to day lives if they are so inclined or instructed to change their systems. Cloud is just someone else's computer. These systems aren't special. Yes they are impressively engineered to deal with the scale they deal with, but when systems are smaller, they can get a lot simpler. I think as an industry we have conflated distributed systems with really hard engineering problems, when it really matter at what level of abstraction the distribution happens when it comes to down stream complexity.
- weitendorf 1y ago
  It introduces some pretty important risks of its own though. If you accidentally delete/forget a local private key or lose your primary email domain there is no recourse. It's significantly easier to set up 2FA and account recovery on a third party service Note that I'm not saying you shouldn't self-host email or anything else. But it's probably more risky for 99% of people compared to just making sure they can recover their accounts.
  - elashri 1y ago
    I have seen much more stories about people losing access to their Gmail because of a comment flagged somewhere else (i.e YouTube) than people losing access to their domains (it is hard to miss all these reminders about renewal and you shouldn't wait until then anyway so that's something under you control). And good luck getting anyone from Google to solve your problem assuming you get to a human.
    - jeffbee 1y ago
      > losing access to their Gmail because Google will never comment on the reasons they disable an account, so all you've read are the unilateral claims of people who may or may not be admitting what they actually did to lose their accounts.
- JoshTriplett 1y ago
  Own your own domain, point it to the email hosting provider of your choice, and if something went horribly wrong, switch providers. Domains are cheap; never use an email address that's email-provider-specific. That's orthogonal to whether you host your own email or use a professional service to do it for you.
  - doubled112 1y ago
    This is my plan. I will lose some email history, but at least I don’t lose my email future. However, you can’t own a domain, you are just borrowing it. There is still a risk that gets shut down too, but I don’t think it is super common.
    - JoshTriplett 1y ago
      > I will lose some email history, but at least I don’t lose my email future. I back up all my email every day, independent of my hosting provider. I have an automatic nightly sync to my laptop, which happens right before my nightly laptop backups.
    - danillonunes 1y ago
      As for the domain risks, my suggestions is to stick with the .com/.net/.org or something common in your country and avoid novelty ones such as .app, .dev, etc, even if you can't get the shortest and simpler name. And if you have some money to spare, just renew it to 10 years.
      - data-ottawa 1y ago
        Even if you renew for 10 years, set a calendar reminder annually to check in and make sure your renewal info is still good.
        spencerflem 1y ago
        You can also top it up every year as well. Two for one :)
- whartung 1y ago
  My singular issue with self hosting specifically with email is not setting it up. Lots of documentation on setting up an email server. But running it is different issue. Notably, I have no idea, and have not seen a resource talking about troubleshooting and problem solving for a self hosted service. Particularly in regards with interoperability with other providers. As a contrived example, if Google blackballs your server, who do you talk to about it? How do you know? Do that have email addresses, or procedures for resolution in the error messages you get talking with them? Or these other global, IP ban sites. I’d like to see a troubleshooting guide for email. Not so much for the protocols like DKIM, or setting DNS up properly, but in dealing with these other actors that can impact your service even if it’s, technically, according to Hoyle, set up and configured properly.
  - boplicity 1y ago
    Most services, including email providers, spam databases, and "ip-ban sites" have clear documentation, in terms of how to get on their good side, if needed, and it is often surprisingly straightforward to do so. Often it's as simple as filling out a relatively form.
    - bluGill 1y ago
      They claim that, but everyone small I know who self hosted email has discovered that forms don't do anything. I switched to fastmail 15 years ago and my email got a lot better because they are big enough that nobody dares ignore them. (maybe the forms work better today than 15 years ago, but enough people keep complaining about this issue that I doubt it)
    - dantodor 1y ago
      Have you ever tried to use it? Because I fought for about 2 months with both Google and Microsoft, trying to self-host my mail server, to no success. The only answer was amongst the lines 'your server has not enough reputation'. Even though perfectly configured, DKIM, DMARC, etc. Now imagine a business not being able to send a message to anyone hosted on Gmail or Outlook, probably 80-90 percents of the companies out there.
- II2II 1y ago
  The risk may be real, but is it likely to happen to many people? The reason why I bring this up is because many early adopters of Gmail switched to it or grew to rely upon it because the alternatives were much worse. The account through your ISP, gone as soon as you switched to another ISP. That switch may have been a necessary switch if you moved to a place the ISP did not service. University email address, gone soon after graduation. Employer's email address, gone as soon as you switched employers (and risky to use for personal use anyhow). Through another dedicated provider, I suspect most of those dedicated providers are now gone. Yeap, self-hosting can sort of resolve the problem. The key word being sort of. Controlling your identity doesn't mean terribly much if you don't have the knowledge to setup and maintain a secure email server. If you know how to do it, and noone is targetting you in particular, you'll probably be fine. Otherwise, all bets are off. Any you don't have total control anyhow. You still have the domain name to deal with after all. You should be okay if you do your homework and stay on top of renewals, almost certainly better off than you would be with Google, but again it is only as reliable as you are. There are reasons why people go with Gmail, and a handful of other providers. In the end, virtually all of those people will be better off in both the short to mid-term.
holoduke 1y ago
I spend quite some years with linux systems, but i am using llms for configurating systems a lot these days. Last week i setup a server for a group of interns. They needed a docker kubernetes setup with some other tooling. I would have spend at least a day or two to set it up normally. Now it took maybe an hour. All the configurations, commands and some issues were solved with help of chatgpt. You still need to know your stuff, but its like having a super tool at hand. Nice.
- haiku2077 1y ago
  Similarly, I was reconfiguring my home server and having Claude generate systemd units and timers was very handy. As you said you do need to know the material to fix the few mistakes and know what to ask for. But it can do the busywork of turning "I need this backup job to run once a week" into the .service and .timer file syntax for you to tweak instead of writing it from scratch.
  - SoftTalker 1y ago
    Isn't depending on Claude to administer your systems rather divergent from the theme of "Self-Host and Tech Independence?"
    - iforgotpassword 1y ago
      I think it's just a turbo mode for figuring things out. Like posting to a forum and getting an answer immediately, without all those idiots asking you why you even want to do this, how software X is better than what you are using etc. Obviously you should have enough technical knowledge to do a rough sanity check on the reply, as there's still a chance you get stupid shit out of it, but mostly it's really efficient for getting started with some tooling or programming language you're not familiar with. You can perfectly do without, it just takes longer. Plus You're not dependent on it to keep your stuff running once it's set up.
    - layoric 1y ago
      Claude and others are still in the adoption phase so the services are good, and not user hostile as they will be in the extraction phase. Hopefully by then some agreement on how to setup RAG systems for actual human constructed documentation for these systems will be way more accessible, and have good results with much smaller self hosted models. IMO, this is where I think/hope the LLMs value to the average person will land long term. Search, but better at understanding the query. Sadly, they will also been used for a lot of user hostile nonsense as well.
johnea 1y ago
Nice article! It's heartening in the new millennium to see some younger people show awareness of the crippling dependency on big tech. Way back in the stone ages, before instagram and tic toc, when the internet was new, anyone having a presence on the net was rolling their own. It's actually only gotten easier, but the corporate candy has gotten exponentially more candyfied, and most people think it's the most straightforward solution to getting a little corner on the net. Like the fluffy fluffy "cloud", it's just another shrink-wrap of vendor lockin. Hook 'em and gouge 'em, as we used to say. There are many ways to stake your own little piece of virtual ground. Email is another whole category. It's linked to in the article, but still uses an external service to access port 25. I've found it not too expensive to have a "business" ISP account, that allows connections on port 25 (and others). Email is much more critical than having a place to blag on, and port 25 access is only the beginning of the "journey". The modern email "reputation" system is a big tech blockade between people and the net, but it can, and should, be overcome by all individuals with the interest in doing so.
- johnea 1y ago
  Just for reference, take a look at this email system using FreeBSD: https://www.purplehat.org/?page_id=1450 https://www.purplehat.org/?page_id=1450 p.s. That was another place the article could mention a broader scope, there is always the BSDs, not just linux...
sunshine-o 1y ago
I self-host most of what I need but I recently faced the ultimate test when my Internet went down intermittently. It raised some interesting questions: - How long can I be productive without the Internet? - What am I missing? The answer for me was I should archive more documentation and NixOS is unusable offline if you do not host a cache (so that is pretty bad). Ultimately I also found out self-hosting most of what I need and being offline really improve my productivity.
- elashri 1y ago
  I find that self hosting "devdocs" [1] and having zeal (on linux) [2] solves a lot of these problems with the offline docs. [1] https://github.com/freeCodeCamp/devdocs https://github.com/freeCodeCamp/devdocs [2] https://zealdocs.org/ https://zealdocs.org/
  - teddyh 1y ago
    For offline documentation, I use these in order of preference: • Info¹ documentation, which I read directly in Emacs. (If you have ever used the terminal-based standalone “info” program, please try to forget all about it. Use Emacs to read Info documentation, and preferably use a graphical Emacs instead of a terminal-based one; Info documentation occasionally has images.) • Gnome Devhelp². • Zeal³ • RFC archive⁴ dumps provided by the Debian “doc-rfc“ package⁵. 1. https://www.gnu.org/software/emacs/manual/html_node/info/ https://www.gnu.org/software/emacs/manual/html_node/info/ 2. https://wiki.gnome.org/Apps/Devhelp https://wiki.gnome.org/Apps/Devhelp 3. https://zealdocs.org/ https://zealdocs.org/ 4. https://www.rfc-editor.org/ https://www.rfc-editor.org/ 5. https://tracker.debian.org/pkg/doc-rfc https://tracker.debian.org/pkg/doc-rfc
- AstroBen 1y ago
  I've taken this as far as I can. I love being disconnected from the internet for extended periods - they're my most productive times I have a bash alias to use wget to recursively save full websites yt-dlp will download videos you want to watch Kiwix will give you a full offline copy of Wikipedia My email is saved locally. I can queue up drafts offline SingleFile extension will allow you to save single pages really effectively Zeal is a great open source documentation browser
aucisson_masque 1y ago
I get why you want to self host, although I also get why you don’t want. Selfhosting is a pain in the ass, it needs updating docker, things break sometimes, sometimes it’s only you and not anyone else so you’re left alone searching the solution, and even when it works it’s often a bit clunky. I have a extremely limited list of self hosted tool that just work and are saving me time (first one on that list would be firefly) but god knows i wasted quite a bit of my time setting up stuffs that eventually broke and that i just abandoned. Today I’m very happy with paying for stuff if the company is respecting privacy and has descent pricing.
- Larrikin 1y ago
  What project did you run into issues with? I've found any project that has gotten to the point of offering a Docker Compose seems to just work. Plus I've found nearly every company will betray your trust in them at some point so why even give them the chance? I self host Home Assistant, but they seem to be the only company that actively enacts legal barriers for themselves so if Paulus gets hit by a bus tomorrow the project can't suddenly start going against the users.
- zdw 1y ago
  > docker There's your problem. Docker adds indirection on storage, networking, etc., and also makes upgrades difficult as you have to either rebuild the container, or rely on others to do so to get security and other updates. If you stick to things that can be deployed as an upstream OS vendor package, or as a single binary (go-based projects frequently do this), you'll likely have a better time in the long run.
  - bluGill 1y ago
    Maybe. There are pros and cons. Docker means you can run two+ different things on the same machine and update them separately. This is sometimes important when one project releases a feature you really want, while a different one just did a major update that broke something you care about. Running on the OS often means you have to update both. Single binary sometimes works, but means you need more memory and disk space. (granted much less a concern today than it was back in 1996 when I first started self hosting, but it still can be an issue)
Havoc 1y ago
Ever since arch got an installer I’m not sure I’d consider it hard anymore. Still dumps you into a command line sure but it’s a long way away from the days of trying to figure out arcane partition block math
- MarcelOlsz 1y ago
  RIP "I use arch btw"
  - bombcar 1y ago
    Hello, I'm "I use gentoo btw"
klabb3 1y ago
I propose a slightly different boundary: not ”to self-host” but ”ability to self-host”. It simply means that you can if you want to, but you can let someone else host it. This is a lot more inclusive, both to those who are less technical and those who are willing to pay for it. People who don’t care, ”I’ll just pay”, are especially affected, and the ones who should care the most. Why? Because today, businesses are more predatory, preying on future technical dependence of their victims. Even if you don’t care about FOSS, it’s incredibly important to be able to migrate providers. If you are locked in they will exploit that. Some do it so systematically they are not interested in any other kind of business.
- crabmusket 1y ago
  This sounds like the "credible exit" idea Bluesky talk about. Also shout-out to Zulip for being open source, self hostable, with a cloud hosted service and transfer between these setups.
- sylens 1y ago
  This is why I like what Ente Photos offers. Pay for them today but you can self host using their software so it is a really nice exit path if you need one.
nodesocket 1y ago
I run a Kubernetes 4x pi cluster and an Intel N150 mini PC both managed with Portainer in my homelab. The following open source ops tools have been a game changer. All tools below run in containers. - kubetail: Kubernetes log viewer for the entire cluster. Deployments, pods, statefulsets. Installed via Helm chart. Really awesome. - Dozzle: Docker container log viewing for the N150 mini pc which just runs docker not Kubernetes. Portainer manual install. - UptimeKuma: Monitor and alerting for all servers, http/https endpoints, and even PostgreSQL. Portainer manual install. - Beszel: Monitoring of server cpu, memory, disk, network and docker containers. Can be installed into Kubernetes via helm chart. Also installed manually via Portainer on the N150 mini pc. - Semaphore UI: UI for running ansible playbooks. Support for scheduling as well. Portainer manual install.
sdf4j 1y ago
> I always say to buy a domain first. You can only rent a domain. The landlord is merciless if you miss a payment, you are out. There are risks everywhere, and it depresses me how fragile is our online identity.
- XorNot 1y ago
  It's something of a technical limitation though: there's no reason all my devices - the consumers of my domain name - couldn't just accept that anything signed with some key is actually XorNot.com or whatever...but good luck keeping that configuration together. You very reasonably could replace the whole system with just "lists of trusted keys to names" if the concept has enough popular technical support.
- 1vuio0pswjnm7 1y ago
  "You can only rent a domain." If ICANN-approved root.zone and ICANN-approved registries are the only options. As an experiment I created own registry, not shared with anyone. For many years I have run own root server, i.e., I serve own custom root.zone to all computers I own. I have a search experiment that uses a custom TLD that embeds a well-known classification system. The TLD portion of the domainname can catgorise any product or service on Earth. ICANN TLDs are vague, ambiguous, sometimes even deceptive.
  - iampims 1y ago
    You should write something about this…
  - coldfoundry 1y ago
    This sounds like a wonderful project, do you have any documentation of the process you wouldn't mind sharing? Would love to play around with something similar to what you did, almost like a mini-internet.
  - eximius 1y ago
    Is there any difference here from running a normal DNS server? Any of your special domains will be ones your server claims as authoritative, so I don't understand why you need a root server?
kassner 1y ago
Warning: shameless plug ahead Self-hosting doesn’t mean you have to buy hardware. After a few years, low-end machines are borderline unusable with Windows, but they are still plenty strong for a Linux server. It’s quite likely you or a friend has an old laptop laying around, which can be repurposed. I’ve done this with an i3 from 2011 [1] for two users, and in 2025 I have no signs that I need an upgrade. Laptops are also quite power efficient at idle, so in the long run they make more sense than a desktop. If you are just starting, they are a great first server. (And no, laptops don’t have an inbuilt UPS. I recommend everyone to remove the battery before using it plugged 24x7) 1: https://www.kassner.com.br/en/2023/05/16/reusing-old-hardware/ https://www.kassner.com.br/en/2023/05/16/reusing-old-hardwar...
- anotherpaul 1y ago
  Glad I am not alone in this. Old laptops are much better than Raspberry pies and often free and power efficient.
  - Onavo 1y ago
    > free and power efficient Free yes. Power efficient no. Unless you switch your laptops every two years, it's unlikely to be more efficient.
    - kassner 1y ago
      My laptop from 2011 idles at 8W, with two SATA SSDs. I have an Intel 10th-gen mini PC that idles at 5W with one SSD. 3W is not groundbreaking, but for a computer you paid $0, it would take many years to offset the $180 paid on a mini PC.
      - motorest 1y ago
        > My laptop from 2011 idles at 8W, with two SATA SSDs. some benchmarks show the Raspberry Pi 4 idling below 3W and consuming a tad over 6W under sustained high load. Power consumption is not an argument that's in favor of old laptops.
Onavo 1y ago
No love for Pangolin? https://www.reddit.com/r/selfhosted/comments/1kqrwev/im_addicted_to_pangolin/ https://www.reddit.com/r/selfhosted/comments/1kqrwev/im_addi...
- PeterStuer 1y ago
  I'm going with Pangolin, small hosted VPS on Hetzner, to front my Homelab. Takes away much of the complications of serving securely directly from the home LAN.
arjie 1y ago
Tooling for self-hosting is quite powerful nowadays. You can start with hosted components and swap various things in for a self-hosted bit. For instance, my blog is self-hosted on a home-server. It has Cloudflare Tunnel in front of it, but I previously have used nginx+letsencrypt+public_ip. It stores data on Cloudflare R2 but I've stored on S3 or I could store on a local NAS (since I access R2 through FUSE it wouldn't matter that much). You have to rent: * your domain name - and it is right that this is not a permanent purchase * your internet access But almost all other things now have tools that you can optionally use. If you turn them off the experience gets worse but everything still works. It's a much easier time than ever before. Back in the '90s and early 2000s, there was nothing like this. It is a glorious time. The one big difference is that email anti-spam is much stricter but I've handled mail myself as recently as 8 years ago without any trouble (though I now use G Suite).
davidcalloway 1y ago
While I like the article and agree with the sentiment, I do feel it would have been nice to at least mention the GNU project and not leave the impression that we have free software only thanks to Linus Torvalds.
briHass 1y ago
I highly recommend anyone going this route to use Proxmox as your base install on the (old) hardware, and then use individual LXCs/VMs for the services you run. Maybe it's just me, but I find LXCs to be much easier to manage and reason about than Docker containers, and the excellent collection of scripts maintained by the community: https://community-scripts.github.io/ProxmoxVE/scripts https://community-scripts.github.io/ProxmoxVE/scripts makes it just as easy as a Docker container registry link. I try to use LXCs whenever the software runs directly on Debian (Proxmox's underlying OS), but it's nice to be able to use a VM for stuff that wants more control like Home Assistant's HAOS. Proxmox makes it fairly straightforward to share things like disks between LXCs, and automated backups are built in.
- leosanchez 1y ago
  I use lxd to manage lxc containers. Am I missing out on anything?
  - nullwarp 1y ago
    A handy mostly straightforward UI with built in backup/restore and other useful tools. It's hardly a requirement but if someone is just starting to learn, proxmox has lots of documentation on how to do things and the UI keeps you from footgunning yourself copy/pasting config code off websites/LLM too much.
  - briHass 1y ago
    Personally, I didn't want to manage my management/virtualization layer. I wanted something that was an all-in iso that wouldn't tempt me to configure at all. I wanted to be able to restore just my container backups to a new PM install without worrying about anything missing at the host (to the extent possible). I also like that Proxmox can be fully managed from the web UI. I'm sure most of this is possible with LCD on some distro, but Proxmox was the standard at the time I set it up (LXD wasn't as polished then)
- esskay 1y ago
  I think it depends on the usecase (and budget) as to what's 'best' really. I've got Proxmox running on a couple of servers and it's great. But I also run Unraid on the main NAS server purely for its ZFS drive setup. Being able to throw in a bunch of drives of various sizes and brands on a home machine is pretty valuable and saves a huge amount of money.
Yeul 1y ago
As someone who recently had to install Windows on a new PC I am convinced Microsoft wants to turn computers into terminals. Which is not exactly what you want from a gaming PC.
budududuroiu 1y ago
I’m almost done with my switch away from a fully Apple ecosystem and I feel great about my Framework laptop, GrapheneOS Pixel and cluster of servers in my closet. I can’t help but wonder if mainstream adoption of open source and self hosting will cause a regulatory backlash in favour of big corpo again (thinking of Bill Gates’ letter against hobbyists)
buildItN0w_ 1y ago
self hosting my own things helped me to gain so much knowledge! Great read!
9283409232 1y ago
I hope we can make hosting open source on VPS much more accessible to the average person. Something like Sandstorm[0] or Umbrel[1]. [0] https://sandstorm.org https://sandstorm.org [1] https://umbrel.com/ https://umbrel.com/
- carlosjobim 1y ago
  Hosting on VPS has recently become much better for the average person with the introduction of Fastpanel. I know that people here are going to hate it because it's not open source, but it is free, user friendly, and very easy to use while still being powerful. It's a total win for me.
kldg 1y ago
SBCs are great for public webservers and suited to save you quite a bit in energy costs. I've used a Raspbery Pi4B for about 5 years with around 10k human visitors (~5k bots) per year just fine. I'd like to try a RISC-V SBC as server, but maybe I have a few more years to wait. I don't run into resource issues on the Pi4B, but resource paranoia (like range anxiety in EVs) keeps me on my toes about bandwidth use and encoding anyway. I did actually repurpose my former workstation and put it in a rackmount case a couple weeks ago to take over duties and take on some new ones, but it consumes so much electricity that it embarrasses me and I turned it off. Not sure what to do with it now; it is comically over-spec'd for a web server. Most helpful thing to have is a good router; networking is a pain in the butt, and there's a lot to do when you host your own when you start serving flask servers or whatever. Mikrotik has made more things doable for me.
- ravetcofx 1y ago
  how are you tracking visitors and differentiating them with bots?
  - kldg 1y ago
    crudely. apache2 logs are parsed every 5 minutes. if the IP address exists already in post-processed database, ignore the entry; if they didn't exist in database, a script parses user agent strings and checks against a list of known "consumer" browsers; a whitelist. If they match, we assume they're human. we then delete the detailed apache2 logs and put just the IP address, when we first saw them (date, not datetime), and whether they were deemed human or bot into database. faking user agent strings or using something like playwright would confuse the script; but the browser list will also inherently not have all entries of existing "consumer browsers". every day, a script checks all IP addresses in the post-processed database to see if there are "clusters" on the same subnet. I think it's if we see 3 visitors on the same subnet, we consider it a likely bot and retroactively switch those entries to being a bot in the database. Without taking in millions of visitors, I think this is reasonable, but it can introduce errors, too.
NicoSchwandner 1y ago
Nice post, very inspiring! It's definitely addictive to self-host your services! And with modern LLMs, this gets much easier!
igtztorrero 1y ago
PostalServer also a great open source software to send massive transactional emails. https://github.com/postalserver/install/ https://github.com/postalserver/install/
czhu12 1y ago
I was able to replicate some of this by building my own hosting platform (https://canine.sh https://canine.sh) that can deploy a Github repo to anywhere -- from a Kubernetes cluster to a home raspberry pi server. I've built tons of stuff in my career, but building the thing that can host all of it for myself has been hugely rewarding (instead of relying on hosting providers that inevitably start charging you) I now have almost 15 apps hosted across 3 clusters: https://imgur.com/a/RYg0wzh https://imgur.com/a/RYg0wzh One of the most cherised things I've built, and I find myself constantly coming back and improving / updating out of love.
owl_vision 1y ago
for the past 20 odd years, old hardware with tweaked, custom compiled and build FreeBSD, NetBSD served me and my few customers quiet well. There is lot of joy in it. Recently, I started modifying open source software to be self hostable. Some does not work well enough when the internet is not accessible. for example FarmOS.
whitefang 1y ago
With self hosting my biggest pain point is disaster and recovery management. I would like to know how do you handle it.
- snowstormsun 1y ago
  Redundant Backups
- mxuribe 1y ago
  I'm not an expert on disaster recovery, but have to think of it for $day_job on occasion. The way i think about it is something along the lines of when (not if), disaster strikes, what needs to be saved vs what can wait a while to have it recovered...plus what level of effort are the different sets of activities worth. For example, if i host my public blog website on my machine at my home, and power goes out (thus making my blog unavailable during the power outage), am i ok with this, or do i need to ensure that it can be brought back up? If i feel that it needs to be brought back up during such a scenario, how much effort (which really means time plus money) do i wish to plan for? Obviously this is a vast oversimplification...but that kind of thinking has helped me bring up topics and questions in advance in order to plan for extreme scenarios...and you know what? Sometimes the things people think they "really need to be up/available" are not that important. Other times, sure, but then you commit, plan, test, and prepare for such truly essential activities. (And, of course, other times, whether the thing that needs to be recovered is important or not, you might be compelled by laws, regulations, contracts, etc...but that's a whole separate topic.) So, again, start at the end - when disaster strikes - and consider what you think is worth keeping/bringing back up, and then plan backwards from there to make it so.
deleted 1y ago
[deleted]

499 points by articsputnik 1y ago

Can definitely become a trend given so many devs out there and so much that AI can produce at home which can be of arbitrary code quality…

ryandrake 1y ago

> The premise is that by learning some of the fundamentals, in this case Linux, you can host most things yourself. Not because you need to, but because you want to, and the feeling of using your own services just gives you pleasure. And you learn from it. Not only that, but it helps to eliminate the very real risk that you get kicked off of a platform that you depend on without recourse. Imagine if you lost your Gmail account. I'd bet that most normies would be in deep shit, since that's basically their identity online, and they need it to reset passwords and maybe even to log into things. I bet there are a non-zero number of HN commenters who would be fucked if they so much as lost their Gmail account. You've got to at least own your own E-mail identity! Rinse and repeat for every other online service you depend on. What if your web host suddenly deleted you? Or AWS? Or Spotify or Netflix? Or some other cloud service? What's your backup? If your answer is "a new cloud host" you're just trading identical problems.

ozim 1y ago
Self hosting at home - what is higher risk? Your HDD dying or losing Gmail account? Oh now you don’t only self host, now you have to have space to keep gear, plan backups, install updates, oh would be good to test updates so some bug doesn’t mess your system. Oh you know installing updates or while backups are running it would be bad if you have power outage- now you need a UPS. Oh you know what - my UPS turned out to be faulty and it f-up my HDD in my NAS. No I don’t have time to deal with any of it anymore I have other things to do with my life ;)
- layoric 1y ago
  Different strokes for different folks. Motivation for me has been a combination of independence and mistrust. Every single one of the larger tech companies have shown their priority to growth above making good products and services, and not being directly user hostile. Google search is worse now than it was 10 years ago. Netflix has ads with a paid subscription, so does YouTube. Windows is absolute joke, more and more we see user hostile software. Incentives aren’t aligned at all. As people who work in software, I get not wanting to do this stuff at home as well. But honestly I’m hoping for a future where a lot of these services can legit be self hosted by technical people for their local communities. Mastodon is doing this really well IMO. Self hosted software is also getting a lot easier to manage, so I’m quite optimistic that things will keep heading this way. Note, I’ve got all the things you mentioned down to the UPSes setup in my garage, as well as multiple levels of backups. It’s not perfect, but works for me without much time input vs utility it provides. Each to their own.
  - deadbabe 1y ago
    If your trust is violated, typically the worst that happens is you are fed a couple more relevant ads or your data is used for some commercial purpose that has little to no effect on your life. Is it really worth going through so much effort to mitigate that risk?
    - layoric 1y ago
      Again, it's a value judgement, so the answer is largely personal. For me, yes. The social license we give these larger companies after all the violated trust doesn't make sense. If your local shop owner/operator that you talked to everyday had the same attitude towards your when you went shopping and exchanged pleasantries with most weeks, people would confront them about their actions, and that shop wouldn't last long. We have created the disconnect for convenience, and tried to ignore the level of control these companies have on our day to day lives if they are so inclined or instructed to change their systems. Cloud is just someone else's computer. These systems aren't special. Yes they are impressively engineered to deal with the scale they deal with, but when systems are smaller, they can get a lot simpler. I think as an industry we have conflated distributed systems with really hard engineering problems, when it really matter at what level of abstraction the distribution happens when it comes to down stream complexity.
weitendorf 1y ago
It introduces some pretty important risks of its own though. If you accidentally delete/forget a local private key or lose your primary email domain there is no recourse. It's significantly easier to set up 2FA and account recovery on a third party service Note that I'm not saying you shouldn't self-host email or anything else. But it's probably more risky for 99% of people compared to just making sure they can recover their accounts.
- elashri 1y ago
  I have seen much more stories about people losing access to their Gmail because of a comment flagged somewhere else (i.e YouTube) than people losing access to their domains (it is hard to miss all these reminders about renewal and you shouldn't wait until then anyway so that's something under you control). And good luck getting anyone from Google to solve your problem assuming you get to a human.
  - jeffbee 1y ago
    > losing access to their Gmail because Google will never comment on the reasons they disable an account, so all you've read are the unilateral claims of people who may or may not be admitting what they actually did to lose their accounts.
JoshTriplett 1y ago
Own your own domain, point it to the email hosting provider of your choice, and if something went horribly wrong, switch providers. Domains are cheap; never use an email address that's email-provider-specific. That's orthogonal to whether you host your own email or use a professional service to do it for you.
- doubled112 1y ago
  This is my plan. I will lose some email history, but at least I don’t lose my email future. However, you can’t own a domain, you are just borrowing it. There is still a risk that gets shut down too, but I don’t think it is super common.
  - JoshTriplett 1y ago
    > I will lose some email history, but at least I don’t lose my email future. I back up all my email every day, independent of my hosting provider. I have an automatic nightly sync to my laptop, which happens right before my nightly laptop backups.
  - danillonunes 1y ago
    As for the domain risks, my suggestions is to stick with the .com/.net/.org or something common in your country and avoid novelty ones such as .app, .dev, etc, even if you can't get the shortest and simpler name. And if you have some money to spare, just renew it to 10 years.
    - data-ottawa 1y ago
      Even if you renew for 10 years, set a calendar reminder annually to check in and make sure your renewal info is still good.
      - spencerflem 1y ago
        You can also top it up every year as well. Two for one :)
whartung 1y ago
My singular issue with self hosting specifically with email is not setting it up. Lots of documentation on setting up an email server. But running it is different issue. Notably, I have no idea, and have not seen a resource talking about troubleshooting and problem solving for a self hosted service. Particularly in regards with interoperability with other providers. As a contrived example, if Google blackballs your server, who do you talk to about it? How do you know? Do that have email addresses, or procedures for resolution in the error messages you get talking with them? Or these other global, IP ban sites. I’d like to see a troubleshooting guide for email. Not so much for the protocols like DKIM, or setting DNS up properly, but in dealing with these other actors that can impact your service even if it’s, technically, according to Hoyle, set up and configured properly.
- boplicity 1y ago
  Most services, including email providers, spam databases, and "ip-ban sites" have clear documentation, in terms of how to get on their good side, if needed, and it is often surprisingly straightforward to do so. Often it's as simple as filling out a relatively form.
  - bluGill 1y ago
    They claim that, but everyone small I know who self hosted email has discovered that forms don't do anything. I switched to fastmail 15 years ago and my email got a lot better because they are big enough that nobody dares ignore them. (maybe the forms work better today than 15 years ago, but enough people keep complaining about this issue that I doubt it)
  - dantodor 1y ago
    Have you ever tried to use it? Because I fought for about 2 months with both Google and Microsoft, trying to self-host my mail server, to no success. The only answer was amongst the lines 'your server has not enough reputation'. Even though perfectly configured, DKIM, DMARC, etc. Now imagine a business not being able to send a message to anyone hosted on Gmail or Outlook, probably 80-90 percents of the companies out there.
II2II 1y ago
The risk may be real, but is it likely to happen to many people? The reason why I bring this up is because many early adopters of Gmail switched to it or grew to rely upon it because the alternatives were much worse. The account through your ISP, gone as soon as you switched to another ISP. That switch may have been a necessary switch if you moved to a place the ISP did not service. University email address, gone soon after graduation. Employer's email address, gone as soon as you switched employers (and risky to use for personal use anyhow). Through another dedicated provider, I suspect most of those dedicated providers are now gone. Yeap, self-hosting can sort of resolve the problem. The key word being sort of. Controlling your identity doesn't mean terribly much if you don't have the knowledge to setup and maintain a secure email server. If you know how to do it, and noone is targetting you in particular, you'll probably be fine. Otherwise, all bets are off. Any you don't have total control anyhow. You still have the domain name to deal with after all. You should be okay if you do your homework and stay on top of renewals, almost certainly better off than you would be with Google, but again it is only as reliable as you are. There are reasons why people go with Gmail, and a handful of other providers. In the end, virtually all of those people will be better off in both the short to mid-term.

holoduke 1y ago

I spend quite some years with linux systems, but i am using llms for configurating systems a lot these days. Last week i setup a server for a group of interns. They needed a docker kubernetes setup with some other tooling. I would have spend at least a day or two to set it up normally. Now it took maybe an hour. All the configurations, commands and some issues were solved with help of chatgpt. You still need to know your stuff, but its like having a super tool at hand. Nice.

haiku2077 1y ago
Similarly, I was reconfiguring my home server and having Claude generate systemd units and timers was very handy. As you said you do need to know the material to fix the few mistakes and know what to ask for. But it can do the busywork of turning "I need this backup job to run once a week" into the .service and .timer file syntax for you to tweak instead of writing it from scratch.
- SoftTalker 1y ago
  Isn't depending on Claude to administer your systems rather divergent from the theme of "Self-Host and Tech Independence?"
  - iforgotpassword 1y ago
    I think it's just a turbo mode for figuring things out. Like posting to a forum and getting an answer immediately, without all those idiots asking you why you even want to do this, how software X is better than what you are using etc. Obviously you should have enough technical knowledge to do a rough sanity check on the reply, as there's still a chance you get stupid shit out of it, but mostly it's really efficient for getting started with some tooling or programming language you're not familiar with. You can perfectly do without, it just takes longer. Plus You're not dependent on it to keep your stuff running once it's set up.
  - layoric 1y ago
    Claude and others are still in the adoption phase so the services are good, and not user hostile as they will be in the extraction phase. Hopefully by then some agreement on how to setup RAG systems for actual human constructed documentation for these systems will be way more accessible, and have good results with much smaller self hosted models. IMO, this is where I think/hope the LLMs value to the average person will land long term. Search, but better at understanding the query. Sadly, they will also been used for a lot of user hostile nonsense as well.

johnea 1y ago

Nice article! It's heartening in the new millennium to see some younger people show awareness of the crippling dependency on big tech. Way back in the stone ages, before instagram and tic toc, when the internet was new, anyone having a presence on the net was rolling their own. It's actually only gotten easier, but the corporate candy has gotten exponentially more candyfied, and most people think it's the most straightforward solution to getting a little corner on the net. Like the fluffy fluffy "cloud", it's just another shrink-wrap of vendor lockin. Hook 'em and gouge 'em, as we used to say. There are many ways to stake your own little piece of virtual ground. Email is another whole category. It's linked to in the article, but still uses an external service to access port 25. I've found it not too expensive to have a "business" ISP account, that allows connections on port 25 (and others). Email is much more critical than having a place to blag on, and port 25 access is only the beginning of the "journey". The modern email "reputation" system is a big tech blockade between people and the net, but it can, and should, be overcome by all individuals with the interest in doing so.

johnea 1y ago
Just for reference, take a look at this email system using FreeBSD: https://www.purplehat.org/?page_id=1450 https://www.purplehat.org/?page_id=1450 p.s. That was another place the article could mention a broader scope, there is always the BSDs, not just linux...

sunshine-o 1y ago

I self-host most of what I need but I recently faced the ultimate test when my Internet went down intermittently. It raised some interesting questions: - How long can I be productive without the Internet? - What am I missing? The answer for me was I should archive more documentation and NixOS is unusable offline if you do not host a cache (so that is pretty bad). Ultimately I also found out self-hosting most of what I need and being offline really improve my productivity.

elashri 1y ago
I find that self hosting "devdocs" [1] and having zeal (on linux) [2] solves a lot of these problems with the offline docs. [1] https://github.com/freeCodeCamp/devdocs https://github.com/freeCodeCamp/devdocs [2] https://zealdocs.org/ https://zealdocs.org/
- teddyh 1y ago
  For offline documentation, I use these in order of preference: • Info¹ documentation, which I read directly in Emacs. (If you have ever used the terminal-based standalone “info” program, please try to forget all about it. Use Emacs to read Info documentation, and preferably use a graphical Emacs instead of a terminal-based one; Info documentation occasionally has images.) • Gnome Devhelp². • Zeal³ • RFC archive⁴ dumps provided by the Debian “doc-rfc“ package⁵. 1. https://www.gnu.org/software/emacs/manual/html_node/info/ https://www.gnu.org/software/emacs/manual/html_node/info/ 2. https://wiki.gnome.org/Apps/Devhelp https://wiki.gnome.org/Apps/Devhelp 3. https://zealdocs.org/ https://zealdocs.org/ 4. https://www.rfc-editor.org/ https://www.rfc-editor.org/ 5. https://tracker.debian.org/pkg/doc-rfc https://tracker.debian.org/pkg/doc-rfc
AstroBen 1y ago
I've taken this as far as I can. I love being disconnected from the internet for extended periods - they're my most productive times I have a bash alias to use wget to recursively save full websites yt-dlp will download videos you want to watch Kiwix will give you a full offline copy of Wikipedia My email is saved locally. I can queue up drafts offline SingleFile extension will allow you to save single pages really effectively Zeal is a great open source documentation browser

aucisson_masque 1y ago

I get why you want to self host, although I also get why you don’t want. Selfhosting is a pain in the ass, it needs updating docker, things break sometimes, sometimes it’s only you and not anyone else so you’re left alone searching the solution, and even when it works it’s often a bit clunky. I have a extremely limited list of self hosted tool that just work and are saving me time (first one on that list would be firefly) but god knows i wasted quite a bit of my time setting up stuffs that eventually broke and that i just abandoned. Today I’m very happy with paying for stuff if the company is respecting privacy and has descent pricing.

Larrikin 1y ago
What project did you run into issues with? I've found any project that has gotten to the point of offering a Docker Compose seems to just work. Plus I've found nearly every company will betray your trust in them at some point so why even give them the chance? I self host Home Assistant, but they seem to be the only company that actively enacts legal barriers for themselves so if Paulus gets hit by a bus tomorrow the project can't suddenly start going against the users.
zdw 1y ago
> docker There's your problem. Docker adds indirection on storage, networking, etc., and also makes upgrades difficult as you have to either rebuild the container, or rely on others to do so to get security and other updates. If you stick to things that can be deployed as an upstream OS vendor package, or as a single binary (go-based projects frequently do this), you'll likely have a better time in the long run.
- bluGill 1y ago
  Maybe. There are pros and cons. Docker means you can run two+ different things on the same machine and update them separately. This is sometimes important when one project releases a feature you really want, while a different one just did a major update that broke something you care about. Running on the OS often means you have to update both. Single binary sometimes works, but means you need more memory and disk space. (granted much less a concern today than it was back in 1996 when I first started self hosting, but it still can be an issue)

Havoc 1y ago

Ever since arch got an installer I’m not sure I’d consider it hard anymore. Still dumps you into a command line sure but it’s a long way away from the days of trying to figure out arcane partition block math

MarcelOlsz 1y ago
RIP "I use arch btw"
- bombcar 1y ago
  Hello, I'm "I use gentoo btw"

klabb3 1y ago

I propose a slightly different boundary: not ”to self-host” but ”ability to self-host”. It simply means that you can if you want to, but you can let someone else host it. This is a lot more inclusive, both to those who are less technical and those who are willing to pay for it. People who don’t care, ”I’ll just pay”, are especially affected, and the ones who should care the most. Why? Because today, businesses are more predatory, preying on future technical dependence of their victims. Even if you don’t care about FOSS, it’s incredibly important to be able to migrate providers. If you are locked in they will exploit that. Some do it so systematically they are not interested in any other kind of business.

crabmusket 1y ago
This sounds like the "credible exit" idea Bluesky talk about. Also shout-out to Zulip for being open source, self hostable, with a cloud hosted service and transfer between these setups.
sylens 1y ago
This is why I like what Ente Photos offers. Pay for them today but you can self host using their software so it is a really nice exit path if you need one.

nodesocket 1y ago

I run a Kubernetes 4x pi cluster and an Intel N150 mini PC both managed with Portainer in my homelab. The following open source ops tools have been a game changer. All tools below run in containers. - kubetail: Kubernetes log viewer for the entire cluster. Deployments, pods, statefulsets. Installed via Helm chart. Really awesome. - Dozzle: Docker container log viewing for the N150 mini pc which just runs docker not Kubernetes. Portainer manual install. - UptimeKuma: Monitor and alerting for all servers, http/https endpoints, and even PostgreSQL. Portainer manual install. - Beszel: Monitoring of server cpu, memory, disk, network and docker containers. Can be installed into Kubernetes via helm chart. Also installed manually via Portainer on the N150 mini pc. - Semaphore UI: UI for running ansible playbooks. Support for scheduling as well. Portainer manual install.

sdf4j 1y ago

> I always say to buy a domain first. You can only rent a domain. The landlord is merciless if you miss a payment, you are out. There are risks everywhere, and it depresses me how fragile is our online identity.

XorNot 1y ago
It's something of a technical limitation though: there's no reason all my devices - the consumers of my domain name - couldn't just accept that anything signed with some key is actually XorNot.com or whatever...but good luck keeping that configuration together. You very reasonably could replace the whole system with just "lists of trusted keys to names" if the concept has enough popular technical support.
1vuio0pswjnm7 1y ago
"You can only rent a domain." If ICANN-approved root.zone and ICANN-approved registries are the only options. As an experiment I created own registry, not shared with anyone. For many years I have run own root server, i.e., I serve own custom root.zone to all computers I own. I have a search experiment that uses a custom TLD that embeds a well-known classification system. The TLD portion of the domainname can catgorise any product or service on Earth. ICANN TLDs are vague, ambiguous, sometimes even deceptive.
- iampims 1y ago
  You should write something about this…
- coldfoundry 1y ago
  This sounds like a wonderful project, do you have any documentation of the process you wouldn't mind sharing? Would love to play around with something similar to what you did, almost like a mini-internet.
- eximius 1y ago
  Is there any difference here from running a normal DNS server? Any of your special domains will be ones your server claims as authoritative, so I don't understand why you need a root server?

kassner 1y ago

Warning: shameless plug ahead Self-hosting doesn’t mean you have to buy hardware. After a few years, low-end machines are borderline unusable with Windows, but they are still plenty strong for a Linux server. It’s quite likely you or a friend has an old laptop laying around, which can be repurposed. I’ve done this with an i3 from 2011 [1] for two users, and in 2025 I have no signs that I need an upgrade. Laptops are also quite power efficient at idle, so in the long run they make more sense than a desktop. If you are just starting, they are a great first server. (And no, laptops don’t have an inbuilt UPS. I recommend everyone to remove the battery before using it plugged 24x7) 1: https://www.kassner.com.br/en/2023/05/16/reusing-old-hardware/ https://www.kassner.com.br/en/2023/05/16/reusing-old-hardwar...

anotherpaul 1y ago
Glad I am not alone in this. Old laptops are much better than Raspberry pies and often free and power efficient.
- Onavo 1y ago
  > free and power efficient Free yes. Power efficient no. Unless you switch your laptops every two years, it's unlikely to be more efficient.
  - kassner 1y ago
    My laptop from 2011 idles at 8W, with two SATA SSDs. I have an Intel 10th-gen mini PC that idles at 5W with one SSD. 3W is not groundbreaking, but for a computer you paid $0, it would take many years to offset the $180 paid on a mini PC.
    - motorest 1y ago
      > My laptop from 2011 idles at 8W, with two SATA SSDs. some benchmarks show the Raspberry Pi 4 idling below 3W and consuming a tad over 6W under sustained high load. Power consumption is not an argument that's in favor of old laptops.

Onavo 1y ago

No love for Pangolin? https://www.reddit.com/r/selfhosted/comments/1kqrwev/im_addicted_to_pangolin/ https://www.reddit.com/r/selfhosted/comments/1kqrwev/im_addi...

PeterStuer 1y ago
I'm going with Pangolin, small hosted VPS on Hetzner, to front my Homelab. Takes away much of the complications of serving securely directly from the home LAN.

arjie 1y ago

Tooling for self-hosting is quite powerful nowadays. You can start with hosted components and swap various things in for a self-hosted bit. For instance, my blog is self-hosted on a home-server. It has Cloudflare Tunnel in front of it, but I previously have used nginx+letsencrypt+public_ip. It stores data on Cloudflare R2 but I've stored on S3 or I could store on a local NAS (since I access R2 through FUSE it wouldn't matter that much). You have to rent: * your domain name - and it is right that this is not a permanent purchase * your internet access But almost all other things now have tools that you can optionally use. If you turn them off the experience gets worse but everything still works. It's a much easier time than ever before. Back in the '90s and early 2000s, there was nothing like this. It is a glorious time. The one big difference is that email anti-spam is much stricter but I've handled mail myself as recently as 8 years ago without any trouble (though I now use G Suite).

davidcalloway 1y ago

While I like the article and agree with the sentiment, I do feel it would have been nice to at least mention the GNU project and not leave the impression that we have free software only thanks to Linus Torvalds.

briHass 1y ago

I highly recommend anyone going this route to use Proxmox as your base install on the (old) hardware, and then use individual LXCs/VMs for the services you run. Maybe it's just me, but I find LXCs to be much easier to manage and reason about than Docker containers, and the excellent collection of scripts maintained by the community: https://community-scripts.github.io/ProxmoxVE/scripts https://community-scripts.github.io/ProxmoxVE/scripts makes it just as easy as a Docker container registry link. I try to use LXCs whenever the software runs directly on Debian (Proxmox's underlying OS), but it's nice to be able to use a VM for stuff that wants more control like Home Assistant's HAOS. Proxmox makes it fairly straightforward to share things like disks between LXCs, and automated backups are built in.

leosanchez 1y ago
I use lxd to manage lxc containers. Am I missing out on anything?
- nullwarp 1y ago
  A handy mostly straightforward UI with built in backup/restore and other useful tools. It's hardly a requirement but if someone is just starting to learn, proxmox has lots of documentation on how to do things and the UI keeps you from footgunning yourself copy/pasting config code off websites/LLM too much.
- briHass 1y ago
  Personally, I didn't want to manage my management/virtualization layer. I wanted something that was an all-in iso that wouldn't tempt me to configure at all. I wanted to be able to restore just my container backups to a new PM install without worrying about anything missing at the host (to the extent possible). I also like that Proxmox can be fully managed from the web UI. I'm sure most of this is possible with LCD on some distro, but Proxmox was the standard at the time I set it up (LXD wasn't as polished then)
esskay 1y ago
I think it depends on the usecase (and budget) as to what's 'best' really. I've got Proxmox running on a couple of servers and it's great. But I also run Unraid on the main NAS server purely for its ZFS drive setup. Being able to throw in a bunch of drives of various sizes and brands on a home machine is pretty valuable and saves a huge amount of money.

Yeul 1y ago

As someone who recently had to install Windows on a new PC I am convinced Microsoft wants to turn computers into terminals. Which is not exactly what you want from a gaming PC.

budududuroiu 1y ago

I’m almost done with my switch away from a fully Apple ecosystem and I feel great about my Framework laptop, GrapheneOS Pixel and cluster of servers in my closet. I can’t help but wonder if mainstream adoption of open source and self hosting will cause a regulatory backlash in favour of big corpo again (thinking of Bill Gates’ letter against hobbyists)

buildItN0w_ 1y ago

self hosting my own things helped me to gain so much knowledge! Great read!

9283409232 1y ago

I hope we can make hosting open source on VPS much more accessible to the average person. Something like Sandstorm[0] or Umbrel[1]. [0] https://sandstorm.org https://sandstorm.org [1] https://umbrel.com/ https://umbrel.com/

carlosjobim 1y ago
Hosting on VPS has recently become much better for the average person with the introduction of Fastpanel. I know that people here are going to hate it because it's not open source, but it is free, user friendly, and very easy to use while still being powerful. It's a total win for me.

kldg 1y ago

SBCs are great for public webservers and suited to save you quite a bit in energy costs. I've used a Raspbery Pi4B for about 5 years with around 10k human visitors (~5k bots) per year just fine. I'd like to try a RISC-V SBC as server, but maybe I have a few more years to wait. I don't run into resource issues on the Pi4B, but resource paranoia (like range anxiety in EVs) keeps me on my toes about bandwidth use and encoding anyway. I did actually repurpose my former workstation and put it in a rackmount case a couple weeks ago to take over duties and take on some new ones, but it consumes so much electricity that it embarrasses me and I turned it off. Not sure what to do with it now; it is comically over-spec'd for a web server. Most helpful thing to have is a good router; networking is a pain in the butt, and there's a lot to do when you host your own when you start serving flask servers or whatever. Mikrotik has made more things doable for me.

ravetcofx 1y ago
how are you tracking visitors and differentiating them with bots?
- kldg 1y ago
  crudely. apache2 logs are parsed every 5 minutes. if the IP address exists already in post-processed database, ignore the entry; if they didn't exist in database, a script parses user agent strings and checks against a list of known "consumer" browsers; a whitelist. If they match, we assume they're human. we then delete the detailed apache2 logs and put just the IP address, when we first saw them (date, not datetime), and whether they were deemed human or bot into database. faking user agent strings or using something like playwright would confuse the script; but the browser list will also inherently not have all entries of existing "consumer browsers". every day, a script checks all IP addresses in the post-processed database to see if there are "clusters" on the same subnet. I think it's if we see 3 visitors on the same subnet, we consider it a likely bot and retroactively switch those entries to being a bot in the database. Without taking in millions of visitors, I think this is reasonable, but it can introduce errors, too.

NicoSchwandner 1y ago

Nice post, very inspiring! It's definitely addictive to self-host your services! And with modern LLMs, this gets much easier!

igtztorrero 1y ago

PostalServer also a great open source software to send massive transactional emails. https://github.com/postalserver/install/ https://github.com/postalserver/install/

czhu12 1y ago

I was able to replicate some of this by building my own hosting platform (https://canine.sh https://canine.sh) that can deploy a Github repo to anywhere -- from a Kubernetes cluster to a home raspberry pi server. I've built tons of stuff in my career, but building the thing that can host all of it for myself has been hugely rewarding (instead of relying on hosting providers that inevitably start charging you) I now have almost 15 apps hosted across 3 clusters: https://imgur.com/a/RYg0wzh https://imgur.com/a/RYg0wzh One of the most cherised things I've built, and I find myself constantly coming back and improving / updating out of love.

owl_vision 1y ago

for the past 20 odd years, old hardware with tweaked, custom compiled and build FreeBSD, NetBSD served me and my few customers quiet well. There is lot of joy in it. Recently, I started modifying open source software to be self hostable. Some does not work well enough when the internet is not accessible. for example FarmOS.

whitefang 1y ago

With self hosting my biggest pain point is disaster and recovery management. I would like to know how do you handle it.

snowstormsun 1y ago
Redundant Backups
mxuribe 1y ago
I'm not an expert on disaster recovery, but have to think of it for $day_job on occasion. The way i think about it is something along the lines of when (not if), disaster strikes, what needs to be saved vs what can wait a while to have it recovered...plus what level of effort are the different sets of activities worth. For example, if i host my public blog website on my machine at my home, and power goes out (thus making my blog unavailable during the power outage), am i ok with this, or do i need to ensure that it can be brought back up? If i feel that it needs to be brought back up during such a scenario, how much effort (which really means time plus money) do i wish to plan for? Obviously this is a vast oversimplification...but that kind of thinking has helped me bring up topics and questions in advance in order to plan for extreme scenarios...and you know what? Sometimes the things people think they "really need to be up/available" are not that important. Other times, sure, but then you commit, plan, test, and prepare for such truly essential activities. (And, of course, other times, whether the thing that needs to be recovered is important or not, you might be compelled by laws, regulations, contracts, etc...but that's a whole separate topic.) So, again, start at the end - when disaster strikes - and consider what you think is worth keeping/bringing back up, and then plan backwards from there to make it so.

deleted 1y ago

[deleted]