55 ms·

Self hosting is important

566 points by hucste 5y ago

dadior 5y ago
Or something is a mix of both: semi-self-hosting, like this app: midinote.me, all your data is on your own computer, but you still can connect to the server, to backup your data, in case you break your computer.
- saurik 5y ago
  offline != self-hosting (which this product doesn't seem to support? I might have just failed to find it, though)
woliveirajr 5y ago
> Self hosting is better when it's done in community, (...) are a good way to create a resilient Internet while not giving away your rights to capitalist companies. If you take away this political (?) tone you'll remember that being dependent of government have the same issues. Or being dependent of a group of friends. Or your family. Anytime you rely on somebody to hold your data, you have to trust that it won't disappear on purpose or by accident. And both things can happen. And, yes, it's hard to deal with everything by yourself. Chose wisely whether you're being cautious and paranoic.
- helsinkiandrew 5y ago
  The weakest point in any system is the incompetent person with admin privileges.
- foobar33333 5y ago
  >Anytime you rely on somebody to hold your data, you have to trust that it won't disappear on purpose or by accident. And both things can happen. This is why I use the feature on Google takeout which emails me a link to a dump of all my data every 2 months. That way if something happens I just sign up to the competitor service and upload my data.
- goodpoint 5y ago
  > being dependent of government have the same issues In some societies there is a [more-or-less] democratic process that is completely absent in private companies. > being dependent of a group ...that you trust. Unlike a company conglomerate that profits from selling your data.
  - woliveirajr 5y ago
    I'd say "that trust you". When a group do something that breaks you, it happens when you trusted them but they excluded you somehow (even not intentionally). If you don't trust, you'd have "protected" yourself.
Santosh83 5y ago
We're well on the path of universal, always connected, high-speed devices with cheap data. Software is also surging in sophistication and complexity. I really don't see why most applications cannot be truly self-hosted (which means your own device or hardware, not a VPS or colocated) these days, except for video. I can only speculate that the abysmal state of self-hosted software for the general public is because there is not enough money to be made in terms of recurring subscriptions or constant inflow of data.
- FooBarWidget 5y ago
  The appeal of hosting is that somebody else takes care of infra, OS, application management for you. Until this can be meaningfully solved for self-hosted situations, self-hosting will always be at a disadvantage.
  - Santosh83 5y ago
    Why can it not be meaningfully solved right now, technically? Do we need AGI to solve easy to use self-hosted apps? I don't think so. I think the blockers are more economic than technical. Which is why in fact the field is heading the other way, towards the universal cloud and thin clients.
    - FooBarWidget 5y ago
      Self-hosting means that the person who does the administering, has no control over the infrastructure, configuration, etc. There are millions of ways in which something could be configured. The user could have installed a kernel extension that panics every 2 days, for all he knows. This means that the administrator must be sufficiently skilled to be able to handle all anything that might come at him. Which means that he's expensive. This makes it economically hard to compete against hosting, where they administrators can be cheaper due to having more controlled environments. One solution is for administrators to insist on that environment conforms to some sort of standard. But no meaningful standardization currently exist for this context. Making the device/software resilient enough, is also very hard, and suffers the same problems as with human administrators. If you install a device in a network with a faulty router, then what is that device supposed to do? How does it even know the router is the culprit?
chmike 5y ago
I wouldn't call it self hosting. It is more associative hosting. Self hosting is when you do everything yourself and that can be really cheap. It's more work and require more competence but you have the minimum dependency. I had a bad experience in using non self hosting. I used weebly for my blog because it was free and convenient. Without warning they disallowed free access. I can't modify my data and can't export it. That gives me an unpleasant feeling about weebly and such type of free service. I now do true self hosting as far as I can. I wouldn't even trust an association.
- juandazapata 5y ago
  > Self hosting is when you do everything yourself and that can be really cheap “Cheap”, only if you don’t value your own time.
  - gsich 5y ago
    Learning is valuable time.
    - WJW 5y ago
      It can also just be enjoyable and therefore not wasted time. That said, the learned skills are only actually valuable if you can use what you learned later on in life. I've done my fair share of fiddling around with raspberry pis and kernel compiling when I was younger, but can't think of a single time in the last few years where I had to use that knowledge in my day job now that everything is containers+k8s+<some cloud hoster>. Maybe we can argue that it gave me a slight speedup when trying to grok the container execution model or something like that, but I could have gained that knowledge much more efficiently in other ways.
    - dmitriid 5y ago
      There are infinite things to learn. Why should I prioritize learning all the broken things that will allow we to self-host, and not, say, carpentry. Or knitting. Or the history and evolution of a non-y language. Or...
bullen 5y ago
Here is my roadmap to the "metaverse" or the final medium if you like: The clients will be X86/Win and slowly migrate to ARM/Lin as electricity prices rise, right now only Jetson Nano is good enough, Raspberry 4 has half-float issues and the GPU is generally too weak. On the backend you need to own the persistent data but not the real-time data, so you will distribute your database on 2x or more home hosted setups and the regional live servers (asia (AWS and GCP), central US (GCP and IONOS) and europe (here anything goes)) will connect to those. You need 1Gb/s up+down fiber on two homes for this. You also need a software/hardware stack that can saturate those 1Gb/s at very low wattage so you can have lead-acid backup power (make sure your appartement building has a UPS on the switch in the basement). The real tricky part is the license you apply to all of this so that others are incentivized to fill the demand for you in the case that blows up! I'm going to go with with monthly payments in proportion to your revenues starting at $20/month. For end customers I'm thinking $10/year.
mark_l_watson 5y ago
I love the sentiments in this blog. I don’t put them into active practice, but I like them, for example I look to Twitter once every morning to see if there is any new tech I should look at or papers to put in my readying list; my Mastodan account languishes. I have small and free VPSs from both Google and Oracle which I appreciate. I totally rely on the publishing platform https://leanpub.com/u/markwatson https://leanpub.com/u/markwatson for writing and publishing the books I write. What her blog triggered for me is that we can have a better digital life by being conscious and taking control of our assets, control over interactions with people and companies, etc.
k__ 5y ago
For every article written about lock-in, hundreds aren't written because people get overwhelmed by technology.
- emodendroket 5y ago
  Let’s face it, even with no explicit lock-in or proprietary features in use, changing providers is still a pain.
  - that_guy_iain 5y ago
    Especially if you're a company and you've integrated their services into yours. I was talking about a business idea with someone and they literally mentioned that once you get into a business relationship with a company you're super hard to remove and used the example of customer service outsourcing they experienced. That's not even tech related and they had serious trouble moving away from the provider.
    - emodendroket 5y ago
      Yeah, I’ve watched a lot of effort going in to avoiding vendor lock-in that seemed like it was basically a waste of everyone’s time.
    - hinkley 5y ago
      If you’re going to half-ass the thing, outsourcing might be better. But if you intend to do it right, you have to do it yourself. Doing it right means achieving multipliers. Using each thing you do to improve everything else you do. How do you improve your product when you’ve outsourced customer support? How do you align it with your business philosophy?
- deleted 5y ago
  [deleted]
aborsy 5y ago
Self-hosting is time-consuming and potentially dangerous with respect to security. You need to know what you are doing. x—————- Example: Dropbox is open to the world. You can share files with everyone. Can you properly secure a nextcloud instance? VPN may not be applicable, because you have to share files with others. Even then, you need to have fair amount of knowledge about networking, protocols, security, current software, vulnerabilities, etc. Even with SSH, you need to be careful. And this is only the security part, I am not getting into a dozen of other concerns. Overall, as software complexity grows, self-hosting will be increasingly harder. Encrypting client-side and using a managed solution is a compelling option.
- kilroy123 5y ago
  This is my issue with self-hosting. I am so damn paranoid. I'm not a sysadmin or a security expert. I don't keep vital or sensitive stuff on anything I'm hosting but it's still frighting.
- api 5y ago
  Until we have self hosting as simple as app installation and without having to fiddle with security, it will be a niche thing.
  - stewbrew 5y ago
    It's not much more difficult. Many hosting companies provide installers like e.g. cPanel that allow you to set up a Nextcloud instance within a minute.
  - input_sh 5y ago
    Plenty of home/SMB NAS offer that. Plus there are projects like https://www.freedombox.org/ https://www.freedombox.org/. On top of that, many hosting providers offer to set up popular open source projects for you.
    - abdullahkhalids 5y ago
      https://sandstorm.io/ https://sandstorm.io/ as well
lbrito 5y ago
One word (okay several): CGNAT. I've been self hosting happily on my android for years until I moved and my current ISP puts me behind a CGNAT. No way to get around it.
- adamnew123456 5y ago
  > No way to get around it. Well...none of them are particularly easy, compared to punching holes in your local firewall. CGNAT takes you one step closer to digital serfdom (all hail our managed lords!) That said, I'd say IPv6 would work if you have a public address and a tunnel broker for v4 only networks. Failing that, some kind of overlay (maybe a .onion?) or a reverse tunnel from someone who does have a public v4 address.
  - donmcronald 5y ago
    > Well...none of them are particularly easy, compared to punching holes in your local firewall. Yeah. You'll have to use something like Cloudflare's Argo to punch out to the world and let them route the traffic back in. That's more complex and could cost and they probably won't like it if you put your media server behind it. Lol.
- 8K832d7tNmiQ 5y ago
  What's stopping you from utilizing vpn tunneling? My server is also behind a CGNAT but can be accessed through Wireguard tunnel with the cheapest vps I found in my place to be the main gateway.
- keyme 5y ago
  I've dealt with this before. It's a pain. Look at this security research about bypassing NATs: https://www.armis.com/research/nat-slipstreaming-v20/ https://www.armis.com/research/nat-slipstreaming-v20/ Look at the section "Creating NAT pinholes to any internal IP using the H.323 ALG" for example. This is using a bug ("feature") that your CGNAT may have implemented (depending on the brand of CGNAT used). Fairly likely that one of those NAT slipstreaming vectors will allow you to punch a hole through it. Is this reliable enough to actually use for self hosting stuff? Probably not. If you do, tell me :) Edit: even the oldest versions of this technique (https://samy.pl/natpin/ https://samy.pl/natpin/) may work for you. Depends if you're lucky. You don't need any of the exploit details that make this into an attack, only the basic concept of using NAT ALGs for unintended purposes.
snowwrestler 5y ago
When you self-host, the government has to come to you for your data. When you use third-party services, the government can go to them. The third party might not fight the request the same way you would. And, you might not even know it happened. The third party might be expressly forbidden from telling you it happened, in fact. This was why Hillary Clinton wanted to host her personal email in her basement. A physical server that she owned, on property she owned; there was no legal way to request that data without going to her personally. If she had used the State Dept server for her personal email, Congress could have accessed all her personal emails simply by asking State to send them over. That’s a controversial example, but the same principle is followed by many companies and organizations who have kept some portion of their data self-hosted. It’s often email or some core of file storage that they consider legally sensitive. This is getting harder to do, though. Look at the recent revelation that the government tried to get newspaper email metadata from Proofpoint, a spam filter provider. Self-hosting a good spam/phishing filter seems almost impossible in 2021, because of the huge amounts of data needed to train filters well.
- ceejayoz 5y ago
  > When you self-host, the government has to come to you for your data. Sure, but my ability to stop them is probably substantially smaller than, say, Amazon’s legal departments capabilities.
  - christophilus 5y ago
    Amazon probably won’t even try.
    - ceejayoz 5y ago
      They’ve clearly and openly committed to trying for years. https://www.computerworld.com/article/2705826/amazon-web-services--we-ll-go-to-court-to-fight-gov-t-requests-for-data.html https://www.computerworld.com/article/2705826/amazon-web-ser... Even Twitter doesn’t like to roll over, and they’ve got a lot less at stake. https://www.latimes.com/politics/story/2021-05-17/twitter-fights-justice-department-subpoena-over-rep-nunes-parody-account https://www.latimes.com/politics/story/2021-05-17/twitter-fi...
stared 5y ago
Your website goes down because there is a temporary power shortage. Your website goes down because it lands on HN... but still, there is a non-trivial cost when there are no visitors. Your website goes down (or worse: gets hijacked) because you forgot to install the newest security update. Your website goes down because, at some point, you lose effort in maintaining it actively. My opinion is that the best way is to host through GitHub (or a similar service). It is up, the way you want. Yet, if anything goes wrong, you still have everything, and there is little friction to push it somewhere else. Sure, if you want to self-host as you have as a DIY project, excellent. For a reliable, safe, cheap, long-term way of sharing data, it is unlikely to be an efficient solution.
- Avamander 5y ago
  > Your website goes down because it lands on HN... but still, there is a non-trivial cost when there are no visitors. A bit of optimization goes a long way, but yes there are limits. Now the question is this risk worth mitigating? I think for small sites that rarely end up on HN, not more so than avoiding reliance on GH for example.
  - stared 5y ago
    If you want to optimize your site for no visitors, you don't need to self-host it anyway.
    - ekianjo 5y ago
      Self hosting is not just for ONE service. You can run an array of local services as well on the same machine.
- ekianjo 5y ago
  > Your website goes down because there is a temporary power shortage. UPS are cheap these days and easy to set up. > Your website goes down because it lands on HN... but still, there is a non-trivial cost when there are no visitors. It would not if you host a static page. > Your website goes down (or worse: gets hijacked) because you forgot to install the newest security update. backups, clean and re-install? Painful the first time you have to do it, then you build good habits from it. > My opinion is that the best way is to host through GitHub (or a similar service) Enjoy your DMCA take-downs for no reason now and then, and good luck fixing that on your own.
rob_c 5y ago
Yes, more of this!
jmull 5y ago
Self-hosting is a great learning experience and can be fun for personal projects. Beyond than that, I think this quote sums it up pretty well: "Asking everyone to host their own services is not even utopia but rather stupid" (BTW, this is from the linked article.)
that_guy_iain 5y ago
I think with GDPR we'll end up seeing more self hosting at companies. Outsourcing it to SaaS platforms is easy but I think we'll start seeing more and more hassles with privacy and data breaches from SaaS platforms and companies who use them getting fined for not doing due dillegence.
stayux 5y ago
Self hosting is logical if you want to have control over your brand. Even if circumstances arise (scalability, bigger audience) you can always design your architecture with focus over maximum control. In this moment in time, with this prices if you are software professional and you are not self hosting your blog (or having maximum control over your intellectual property) you are lazy. :)
bovermyer 5y ago
This article touches on something interesting: community hosting. I'd like to explore that. Specifically, the idea of small communities where a group of people maintains the underlying tech, and - kicker here - everyone in the community knows more or less everyone else in the community. That offers a bit more security/safety/continuity than just self-hosting everything, while still not ceding control to a faceless corporation. Granted, there will always be other reliances outside of the community - like internet and electricity providers - but a line has to be drawn somewhere.
- mnahkies 5y ago
  I can't remember the name now, but I picked up a flyer for a place just like this based in Amsterdam at fosdem once. It was very reasonably priced, you were able to have physical access and is more of an enthusiast club than a business. Seriously considered it, but I don't live in Amsterdam and they recommended being able to speak Dutch to participate properly.
  - ValentineC 5y ago
    > It was very reasonably priced, you were able to have physical access and is more of an enthusiast club than a business. Sounds like it might be one of the hackerspaces there: https://wiki.hackerspaces.org/Amsterdam https://wiki.hackerspaces.org/Amsterdam
- derbOac 5y ago
  This has been on my mind lately a lot with Nextdoor. I have really mixed feelings about Nextdoor which is a slightly separate issue, but it always seemed to me that something like Mastodon or maybe even SSB would be an ideal use case in the same space as Nextdoor. You could have local communities around local servers, that have some natural reason to organize about that (geography), but are still loosely federated. I'm not sure where my thoughts are going, as I'm not exactly surprised Nextdoor has more use than a more decentralized system for this use, but it's salient to me as I'd think something like SSB or Mastodon would ideally occupy the space that Nextdoor is occupying. I'm not sure if it is highlighting the legwork that Nextdoor did to build up its userbase (physically mailing people in a community), or the lack of technical sophistication of users in general, or the relative infancy of Mastodon/SSB/etc, or something inherent about getting a foot in the door with decentralized stuff in terms of mindset, or some inherent limitations of decentralization (can you really just compel/convince people to use decentralized services? People just use them). I'm trying to imagine, for example, local police posting to Mastodon about some local safety issue in the same way as on Nextdoor. With Nextdoor, it's something known nationally, the state probably gives them recommendations, they just post to Nextdoor. Nextdoor might have even reached out to them. With e.g., Mastodon, I suppose I could see it being recognized as a thing if use got up, but where are they posting? The local popular servers? Do they run their own police server? Some kind of city government server? This isn't a criticism of decentralization -- I'd like to see everything more decentralized. I just think something like Nextdoor is an interesting case to me to think through these issues because Nextdoor is so localized, and it seems like that's kind of the ideal use case for decentralized services.
gdsdfe 5y ago
This kinda make it sound like everyone have time, money and know how to setup, host, secure and maintain all this stuff ... Is it important yeah absolutely, is convenience important ? also yes ... Which is more important, well choose your own adventure!
superbaconman 5y ago
I've never heard of CGNAT before this thread but add that to ISPs downgrading upload speeds, and refusal to allocate residential v6 space... Our whole industry is out to kill technical independence.
- gruez 5y ago
  >Our whole industry is out to kill technical independence. Get a grip. The reasons you listed either don't really impede self-hosting, or have more benign explanations than "they're out to kill technical independence". >CGNAT Because ipv6 rollout is hard, and even if you do have ipv6 rolled out in your network, you'll still need ivp4 for vast portions of the internet >ISPs downgrading upload speeds Because spectrum on the wire is limited, and most consumers download. It's not really logical to allocate the spectrum evenly across upload/download just so a few people self-hosting can benefit to the detriment of everyone else. >refusal to allocate residential v6 space I'm presuming you're talking about ISPs that only allocate a /128? It's not really clear how this impedes self-hosting. You just have to set up port-forwarding, which presumably you have tho skills to do if you're the type of person to self-host.
  - throw0101a 5y ago
    > Because ipv6 rollout is hard, and even if you do have ipv6 rolled out in your network, you'll still need ivp4 for vast portions of the internet The experience of the ISP Free in France: After having had a succinct presentation of the 6rd idea, a major French Internet service provider (ISP), Free of the Iliad group (hereafter Free), did all of the following in an impressively short delay of only five weeks (November 7th to December 11th 2007): 1. obtained from its regional Internet Registry (RIR) an IPv6 prefix, the length of which was that allocated without a justification and a delay to examine it, namely /32; 2. added 6rd support to the software of its Freebox home-gateway (upgrading for this an available 6to4 code); 3. provisioned PC-compatible platform with a 6to4 gateway software; 4. modified it to support 6rd; 5. tested IPv6 operation with several operating systems and applications; 6. finished operational deployment, by means of new version of the downloadable software of their Freeboxes; 7. announced IPv6 Internet connectivity, at no extra charge, for all its customers wishing to activate it. More than 1,500,000 residential customers thus became able to use IPv6 if they wished, with all the look and feel of native IPv6 addresses routed in IPv6. The only condition was an activation of IPv6 in their Freeboxes, and of course in their IPv6-capable hosts. * https://datatracker.ietf.org/doc/html/rfc5569 https://datatracker.ietf.org/doc/html/rfc5569 * https://en.wikipedia.org/wiki/IPv6_rapid_deployment https://en.wikipedia.org/wiki/IPv6_rapid_deployment This was ten years ago, so the Internet was less integral to people's lives (relatively speaking). Some more testing may be needed nowadays for IP end-nodes, but I'm not sure if things in the network infrastructure would be any more challenging.
unixhero 5y ago
Cheers from /r/selfhosted[0]! 0, Awesomelist selfhosted, https://github.com/awesome-selfhosted/awesome-selfhosted https://github.com/awesome-selfhosted/awesome-selfhosted
scottydelta 5y ago
As someone who self-hosts a lot of different apps, self-hosting is really a slippery slope. Once you start enjoying the control over the system and data, you want to self host everything. The most important aspect is the security and you learn this by doing it. My entire self-host apps are hosted behind a private VPN called Pritunl, it provides self-hosted corporate VPN like setup where you can manage users and access to servers. I host these following apps/products right now: - Pritunl (corporate like VPN) - Superset (Analytics) - Bitwarden (Password Manager) - OpenVPN with Pihole (Personal VPN with Adblock) - Wireguard with Pihole (Personal VPN with Adblock) - Drone.io (CI/CD) - Posthog (Web Traffic Analytics) - Papercups (Web chat support)
- hinkley 5y ago
  I think the team dynamic can be just as important from the “host everything” standpoint. Hosts generally have incentives to automate manual processes, and a diverse set of customers pushing to make that automation sane, for some value of sanity. There’s a struggle against manual processes in self hosted environments, or aggressive automation with bespoke or otherwise incomplete tools. What you want is glue code holding together open source tools without too much abstraction over the top. You should always have a hint what’s going on underneath. I find myself having to spend way too much social capital on this. While I much prefer self hosted, there is a clear advantage of third parties inasmuch as you can bond over the stupid things their solutions do, instead of driving wedges between teams by engaging in that kind if catharsis.
- gizdan 5y ago
  Out of curiousity, what is reason to host three different VPNs?
  - scottydelta 5y ago
    I have my own bootstrapped startup since a year and half and I manage the entire infrastructure for my startup. My startup focuses on big data projects and is currently building a web-based Bloomberg Terminal alternative(https://quantale.io https://quantale.io) which is an infrastructure heavy project. here is why I use 3 VPNs: 1. Pritunl is used as an enterprise VPN setup. Using this VPN, I provide access to different webapps/self-host to different users in my team/clients. for eg, the stage version of Quantale is hosted on a server(with ufw rule to only allow connection from pritunl VPN IP) and I can provide access to you by creating a vpn config for you and you will only be able to access stage server, you wont be able to access any other servers behind this VPN. 2. OpenVPN with Pihole, I use this VPN as a personal VPN. This VPN blocks ads and trackers using PiHole and my self-hosted password manager is only accessible via this VPN 3. Wirehole with PiHole, this is a backup VPN to access my password manager in case I loose access to my OpenVPN server.
candiddevmike 5y ago
If you're looking for a self hosted "household management" solution, checkout Domestica--https://about.domestica.app https://about.domestica.app. Calendars, budgets, recipes, tasks and more, all integrated and hosted on your stuff.
- hackernudes 5y ago
  To save everyone else a click: "You must have an active Domestica Premium subscription to use the self-hosted version." From https://hub.docker.com/r/candiddev/domestica https://hub.docker.com/r/candiddev/domestica
  - candiddevmike 5y ago
    Yep, however a bit further down it says you can do this as a one time purchase and keep using Domestica, forever, but you won't get updates after your subscription expires. I should probably update the docs to emphasize that more. This is a side project of mine, hopefully someday I can make the self hosted version free or OSS.
  - donmcronald 5y ago
    I don't have a problem with the way that one's set up because it's not some ridiculous pay per user per feature per month scheme like most SaaS. Paying $X per year for continual software updates without limitations on user count or features is good value and the self hosting option creates a natural limit on the scale (ie: no one's going to host 10k users on an instance). Compare it to something like GitLab where I have the burden of maintaining it plus I still have to pay per user per month and deal with ridiculous feature tiers. Which one is better value and less hassle for me? There's a big difference IMO.
kosasbest 5y ago
Isn't self-hosting usually forbidden by most ISPs? I've read countless times in the ToS of ISPs that running your own server is against their terms. Also: if you suffer from frequent power outages, you have to have a server that 'bounces back' and survives the outage, with all the necessary services up and running after a reboot. Good luck writing scripts for that scenario.
- hellbannedguy 5y ago
  I have heard the same for years. I have a feeling they forbid it in tos, but don't enforce it in most areas? Does anyone know for sure if Xfinity in Marin County, Ca allows self-hosting?
  - hoppyhoppy2 5y ago
    >[You may not] use or run dedicated, stand-alone equipment or servers from the Premises that provide network content or any other services to anyone outside of your Premises local area network (“Premises LAN”), also commonly referred to as public services or servers. Examples of prohibited equipment and servers include, but are not limited to, email, web hosting, file sharing, and proxy services and servers https://www.xfinity.com/Corporate/Customers/Policies/HighSpeedInternetAUP https://www.xfinity.com/Corporate/Customers/Policies/HighSpe... This applies to Comcast's (at least residential) customers nationwide. But I, like you, get the sense that it's there so they can shut down excessive bandwidth use; I haven't heard about them trying to shut down a low-traffic webserver or self-hosted stuff for one's own personal use (though I probably wouldn't try to run a gaming server or anything high-traffic). Just be aware that it is technically against Comcast's Acceptable Use Policy, and try to switch to a better internet provider if you have the option.
- alamortsubite 5y ago
  > if you suffer from frequent power outages, you have to have a server that 'bounces back' and survives the outage In my experience self-hosting a number of services, it's really no more challenging than managing the services themselves. A BIOS that supports wake-on-power is obviously a key hardware requirement. On the software side, I find systemd and DuckDNS to be very helpful and easy to use. One of my sites is a remote cabin that suffers blackouts all the time. It has the burden of an LTE connection, which means it requires an ssh tunnel to get around the ISP's firewall. This makes autossh an additional, if simple and reliable component.
hinkley 5y ago
I have a more practical take on self hosting. Developers should be talking to the Ops folk. It informs your architecture decisions with practical considerations, like physics, and how many NICs you can plug into a homogenous switch before you have network hops screwing up your pretty but naive designs. When you stop self hosting, the number and quality of those people goes away when they realize they should find someplace else to be. And when we need fewer of them, we stop making new ones. I try to push architectures that allow for a degree of heterogeneity, where we have one data center we own, and use others we don’t for geographic redundancy and speed of light concerns. For a read mostly system 5a Reading an entire zip file's contents and writing out a brand new zip file could be an extremely slow process. For read-mostly systems, that may mean for instance that we keep the system of record (I’m doing just this to bootstrap a personal project that has a read-mostly information architecture) but distribute the UI out into the Someone Else’s Computers.
normac2 5y ago
It's depressing, because I can imagine a world where the government would access people's data, but only under extreme circumstances like investigating terrorist networks. Instead, they use it in outrageously corrupt ways, like collecting vast swathes of communications from everyone and storing them with poor limitations on access. As Chomsky (who I don't agree with on everything by any means) has written, when the government talks about doing things for "security," that usually means security of the government from its own people.
sgt 5y ago
A lot of people here are worried about constantly having to worry about hardware instability for the services they are self hosting. I've been thinking of is to rent colo space for an 1U enclosure, with two Mac Mini M1's inside. Then run them with a tiny Mikrotik router in front that can either load balance or perhaps some other reliable piece of hardware to do HAproxy style functionality. From that one should be able to provide a very reliable server even in the cases of hardware issues on one of them, assuming the DC has decent connectivity and redundant power - all inside one neat little package. As much as I love macOS, I would prefer to run those two Mac Minis on Linux though. Wonder how mature the port is?
- moondev 5y ago
  If you are focused on arm infrastructure for linux, you can do a lot better than a Mac mini m1, they are limited to 16GB of RAM and booting Linux is experimental at best. Check out something like the solidrun honeycomb lx2 which can take 2 32G sodimms: https://shop.solid-run.com/product/SRLX216S00D00GE064H08CH/ https://shop.solid-run.com/product/SRLX216S00D00GE064H08CH/ Designed for linux out of the box, and you can even run VMware esxi for arm and go wild with vms. https://flings.vmware.com/esxi-arm-edition https://flings.vmware.com/esxi-arm-edition
  - sgt 5y ago
    Yeah but without the outstanding performance of the M1 chip I then I might as well just buy non-ARM. Saying M1 is comparable with any run of the mill ARM based CPU is the understatement of the decade. I will look at how stable the port is.
kaydub 5y ago
The biggest most successful and best run organizations I've worked for have not worried about hosting their software. They're concerned with delivering value to customers and doing it quickly. The most fractured and worst run companies have been concerned with self-hosting and frankly they sympathized with a lot of the stuff I'm seeing on here. But I recognize we still need people self-hosting because it drives innovation and competition. I believe there are ways of doing things well while self-hosting, but I'm not sure what those ways are.
- deleted 5y ago
  [deleted]
- donmcronald 5y ago
  I don't think it's a yes or no, right or wrong kind of thing. For an independent / small developer I think there's value in self hosting your tools, but I think it would be crazy to self host a customer facing app. There's scaling and redundancy that you can take advantage that you could never come close to hosting on your own. However, for tooling like VCS and CI, I think maintaining control is extra important when you don't have any negotiating power. For mid-sized companies where you're paying someone to maintain things, whether that's an employee or a 3rd party, I think you need to assess your tools in terms of your negotiating power and how important it is to maintain control of everything. What if GitHub bans you? I don't think I'd try to self host a customer facing app at this scale either. I think you can be short-term successful by throwing caution to the wind and using every shortcut available, but will the first to market advantage be enough to offset the price competition of people that aren't locked in to some proprietary API gateway or WAF? For example, what if I take extra time to build on OpenFaaS and you build on AWS everything. Who wins long term? You're faster, but I have better negotiating power (ie: less costs) by threatening to switch vendors. Or is the idea of switching hosting vendors detached from reality at this point? Is hosting cost so negligible it doesn't matter? All I know is that everything looks crazy expensive from where I am.
  - kaydub 5y ago
    > What if GitHub bans you? This is one of those things that I was talking about them sympathizing with. It's unrealistic. If you're getting banned from github or a cloud provider maybe you should reconsider what you're doing. > You're faster, but I have better negotiating power (ie: less costs) by threatening to switch vendors. Okay, but AWS isn't expensive as it is. If you build to the way these resources are meant to be built you can seriously minimize your costs. I've seen plenty of lift and shifted apps get their costs dramatically reduced once redesigned in a "cloud native" architecture. The lift and shifted design was in the thousands of dollars a month. Rebuilding for AWS resources brought our costs to almost literally nothing on this same app. I'm talking $5,000 a month to $5 per month. So I have a hard time following the premise that you'll do it cheaper with an on-prem or self-managed system.
duped 5y ago
My company recently migrated from OTS 3rd party apps hosted on their clouds that had frequent downtime and high costs, and self hosted a highly touted and battle tested solution instead. It's a disaster and costs us far more time in developer hours than we spent before. It turns out infrastructure isn't free and paying someone else to care about it scales very nicely.
deleted 5y ago
[deleted]
surfsvammel 5y ago
I go back and forth between hosting everything myself for a year or two, then deciding its not worth the time and effort and just signing up for what ever services I need. Then I go back to self hosting again a year later... I am not sure why I self host. I think it's the engineer in me that just needs to understand how things work, and I need to tinker. It's a hobby more than anything.
Havoc 5y ago
I’ve also found it massively beneficial for learning. A virtualisation server at home makes it much easier to experiment. After a bit of gathering knowledge and ansible recipes the various bits start getting synergies and more becomes possible a la Lego.
kaetemi 5y ago
More important than that is just using portable technologies that you can easily use a hosted version of, as well as be able to run the whole thing on your own machine. If you can't even run your own code yourself you're going to be in debugging hell (these kinds of teams do exist).
jlkuester7 5y ago
IMHO, containerization technology (Docker) has changed the game for self-hosting. I run a bunch of different services all segregated in their own containers behind a Traefik container that rounts to all of them based on domain/subdomain. I don't have to spend a bunch of time wording about incompatible libraries or versions because everything is self-contained and I can upgrade each service individually without worrying about affecting the other services for the same reason. I can configure it all up simply with docker-compose.
nazrulmum10 5y ago
Why is self hosting exactly? § Self hosting is about freedom, you can choose what server you want to run, which version, which features and which configuration you want. If you self host at home, You can also pick the hardware to match your needs (more Ram ? More Disk? RAID?). Self hosting is not a perfect solution, you have to buy the hardware, replace faulty components, do the system maintenance to keep the software part alive.
akho 5y ago
I have a home NAS, accessible through a Wireguard network set up on the router. Am I self-hosting Synology's things (for photos / music / video / downloads)? Synology also supports Docker, so I have - Gitea - Vaultwarden - Nextcloud - pihole - Miniflux (+ rssbridge) running there. These I would definitely say I self-host. I'm also happier with these choices (with the exception of Nextcloud, which I'm replacing with Syncthing) than I would be with any commercial / cloud replacement. Storing passwords outside your control just is not sane. RSS readers on the web suck and want money to read a well-established file format from a public source. Gitea is really nice for things I don't want on GitHub or public, but want to be able to look up on the go (my Org notes, dotfiles, any short-term miniprojects, &c). Synology's apps are used for high-volume stuff, which would combine into quite a large regular bill if done "in the cloud".
jesterson 5y ago
Can't agree more with the sentiment. We are self-hosting everything we can and quite happy with it. From security and management standpoint it's incomparably better. It does have it's management overhead, however for years we managed to create processes and workflows keeping it's to bare minimum. Contrary to popular opinion I believe that self-hosting will grow, at least for solutions requiring immense security and reliability

566 points by hucste 5y ago

Or something is a mix of both: semi-self-hosting, like this app: midinote.me, all your data is on your own computer, but you still can connect to the server, to backup your data, in case you break your computer.

saurik 5y ago
offline != self-hosting (which this product doesn't seem to support? I might have just failed to find it, though)

woliveirajr 5y ago

> Self hosting is better when it's done in community, (...) are a good way to create a resilient Internet while not giving away your rights to capitalist companies. If you take away this political (?) tone you'll remember that being dependent of government have the same issues. Or being dependent of a group of friends. Or your family. Anytime you rely on somebody to hold your data, you have to trust that it won't disappear on purpose or by accident. And both things can happen. And, yes, it's hard to deal with everything by yourself. Chose wisely whether you're being cautious and paranoic.

helsinkiandrew 5y ago
The weakest point in any system is the incompetent person with admin privileges.
foobar33333 5y ago
>Anytime you rely on somebody to hold your data, you have to trust that it won't disappear on purpose or by accident. And both things can happen. This is why I use the feature on Google takeout which emails me a link to a dump of all my data every 2 months. That way if something happens I just sign up to the competitor service and upload my data.
goodpoint 5y ago
> being dependent of government have the same issues In some societies there is a [more-or-less] democratic process that is completely absent in private companies. > being dependent of a group ...that you trust. Unlike a company conglomerate that profits from selling your data.
- woliveirajr 5y ago
  I'd say "that trust you". When a group do something that breaks you, it happens when you trusted them but they excluded you somehow (even not intentionally). If you don't trust, you'd have "protected" yourself.

Santosh83 5y ago

We're well on the path of universal, always connected, high-speed devices with cheap data. Software is also surging in sophistication and complexity. I really don't see why most applications cannot be truly self-hosted (which means your own device or hardware, not a VPS or colocated) these days, except for video. I can only speculate that the abysmal state of self-hosted software for the general public is because there is not enough money to be made in terms of recurring subscriptions or constant inflow of data.

FooBarWidget 5y ago
The appeal of hosting is that somebody else takes care of infra, OS, application management for you. Until this can be meaningfully solved for self-hosted situations, self-hosting will always be at a disadvantage.
- Santosh83 5y ago
  Why can it not be meaningfully solved right now, technically? Do we need AGI to solve easy to use self-hosted apps? I don't think so. I think the blockers are more economic than technical. Which is why in fact the field is heading the other way, towards the universal cloud and thin clients.
  - FooBarWidget 5y ago
    Self-hosting means that the person who does the administering, has no control over the infrastructure, configuration, etc. There are millions of ways in which something could be configured. The user could have installed a kernel extension that panics every 2 days, for all he knows. This means that the administrator must be sufficiently skilled to be able to handle all anything that might come at him. Which means that he's expensive. This makes it economically hard to compete against hosting, where they administrators can be cheaper due to having more controlled environments. One solution is for administrators to insist on that environment conforms to some sort of standard. But no meaningful standardization currently exist for this context. Making the device/software resilient enough, is also very hard, and suffers the same problems as with human administrators. If you install a device in a network with a faulty router, then what is that device supposed to do? How does it even know the router is the culprit?

chmike 5y ago

I wouldn't call it self hosting. It is more associative hosting. Self hosting is when you do everything yourself and that can be really cheap. It's more work and require more competence but you have the minimum dependency. I had a bad experience in using non self hosting. I used weebly for my blog because it was free and convenient. Without warning they disallowed free access. I can't modify my data and can't export it. That gives me an unpleasant feeling about weebly and such type of free service. I now do true self hosting as far as I can. I wouldn't even trust an association.

juandazapata 5y ago
> Self hosting is when you do everything yourself and that can be really cheap “Cheap”, only if you don’t value your own time.
- gsich 5y ago
  Learning is valuable time.
  - WJW 5y ago
    It can also just be enjoyable and therefore not wasted time. That said, the learned skills are only actually valuable if you can use what you learned later on in life. I've done my fair share of fiddling around with raspberry pis and kernel compiling when I was younger, but can't think of a single time in the last few years where I had to use that knowledge in my day job now that everything is containers+k8s+<some cloud hoster>. Maybe we can argue that it gave me a slight speedup when trying to grok the container execution model or something like that, but I could have gained that knowledge much more efficiently in other ways.
  - dmitriid 5y ago
    There are infinite things to learn. Why should I prioritize learning all the broken things that will allow we to self-host, and not, say, carpentry. Or knitting. Or the history and evolution of a non-y language. Or...

bullen 5y ago

Here is my roadmap to the "metaverse" or the final medium if you like: The clients will be X86/Win and slowly migrate to ARM/Lin as electricity prices rise, right now only Jetson Nano is good enough, Raspberry 4 has half-float issues and the GPU is generally too weak. On the backend you need to own the persistent data but not the real-time data, so you will distribute your database on 2x or more home hosted setups and the regional live servers (asia (AWS and GCP), central US (GCP and IONOS) and europe (here anything goes)) will connect to those. You need 1Gb/s up+down fiber on two homes for this. You also need a software/hardware stack that can saturate those 1Gb/s at very low wattage so you can have lead-acid backup power (make sure your appartement building has a UPS on the switch in the basement). The real tricky part is the license you apply to all of this so that others are incentivized to fill the demand for you in the case that blows up! I'm going to go with with monthly payments in proportion to your revenues starting at $20/month. For end customers I'm thinking $10/year.

mark_l_watson 5y ago

I love the sentiments in this blog. I don’t put them into active practice, but I like them, for example I look to Twitter once every morning to see if there is any new tech I should look at or papers to put in my readying list; my Mastodan account languishes. I have small and free VPSs from both Google and Oracle which I appreciate. I totally rely on the publishing platform https://leanpub.com/u/markwatson https://leanpub.com/u/markwatson for writing and publishing the books I write. What her blog triggered for me is that we can have a better digital life by being conscious and taking control of our assets, control over interactions with people and companies, etc.

k__ 5y ago

For every article written about lock-in, hundreds aren't written because people get overwhelmed by technology.

emodendroket 5y ago
Let’s face it, even with no explicit lock-in or proprietary features in use, changing providers is still a pain.
- that_guy_iain 5y ago
  Especially if you're a company and you've integrated their services into yours. I was talking about a business idea with someone and they literally mentioned that once you get into a business relationship with a company you're super hard to remove and used the example of customer service outsourcing they experienced. That's not even tech related and they had serious trouble moving away from the provider.
  - emodendroket 5y ago
    Yeah, I’ve watched a lot of effort going in to avoiding vendor lock-in that seemed like it was basically a waste of everyone’s time.
  - hinkley 5y ago
    If you’re going to half-ass the thing, outsourcing might be better. But if you intend to do it right, you have to do it yourself. Doing it right means achieving multipliers. Using each thing you do to improve everything else you do. How do you improve your product when you’ve outsourced customer support? How do you align it with your business philosophy?
deleted 5y ago
[deleted]

aborsy 5y ago

Self-hosting is time-consuming and potentially dangerous with respect to security. You need to know what you are doing. x—————- Example: Dropbox is open to the world. You can share files with everyone. Can you properly secure a nextcloud instance? VPN may not be applicable, because you have to share files with others. Even then, you need to have fair amount of knowledge about networking, protocols, security, current software, vulnerabilities, etc. Even with SSH, you need to be careful. And this is only the security part, I am not getting into a dozen of other concerns. Overall, as software complexity grows, self-hosting will be increasingly harder. Encrypting client-side and using a managed solution is a compelling option.

kilroy123 5y ago
This is my issue with self-hosting. I am so damn paranoid. I'm not a sysadmin or a security expert. I don't keep vital or sensitive stuff on anything I'm hosting but it's still frighting.
api 5y ago
Until we have self hosting as simple as app installation and without having to fiddle with security, it will be a niche thing.
- stewbrew 5y ago
  It's not much more difficult. Many hosting companies provide installers like e.g. cPanel that allow you to set up a Nextcloud instance within a minute.
- input_sh 5y ago
  Plenty of home/SMB NAS offer that. Plus there are projects like https://www.freedombox.org/ https://www.freedombox.org/. On top of that, many hosting providers offer to set up popular open source projects for you.
  - abdullahkhalids 5y ago
    https://sandstorm.io/ https://sandstorm.io/ as well

lbrito 5y ago

One word (okay several): CGNAT. I've been self hosting happily on my android for years until I moved and my current ISP puts me behind a CGNAT. No way to get around it.

adamnew123456 5y ago
> No way to get around it. Well...none of them are particularly easy, compared to punching holes in your local firewall. CGNAT takes you one step closer to digital serfdom (all hail our managed lords!) That said, I'd say IPv6 would work if you have a public address and a tunnel broker for v4 only networks. Failing that, some kind of overlay (maybe a .onion?) or a reverse tunnel from someone who does have a public v4 address.
- donmcronald 5y ago
  > Well...none of them are particularly easy, compared to punching holes in your local firewall. Yeah. You'll have to use something like Cloudflare's Argo to punch out to the world and let them route the traffic back in. That's more complex and could cost and they probably won't like it if you put your media server behind it. Lol.
8K832d7tNmiQ 5y ago
What's stopping you from utilizing vpn tunneling? My server is also behind a CGNAT but can be accessed through Wireguard tunnel with the cheapest vps I found in my place to be the main gateway.
keyme 5y ago
I've dealt with this before. It's a pain. Look at this security research about bypassing NATs: https://www.armis.com/research/nat-slipstreaming-v20/ https://www.armis.com/research/nat-slipstreaming-v20/ Look at the section "Creating NAT pinholes to any internal IP using the H.323 ALG" for example. This is using a bug ("feature") that your CGNAT may have implemented (depending on the brand of CGNAT used). Fairly likely that one of those NAT slipstreaming vectors will allow you to punch a hole through it. Is this reliable enough to actually use for self hosting stuff? Probably not. If you do, tell me :) Edit: even the oldest versions of this technique (https://samy.pl/natpin/ https://samy.pl/natpin/) may work for you. Depends if you're lucky. You don't need any of the exploit details that make this into an attack, only the basic concept of using NAT ALGs for unintended purposes.

snowwrestler 5y ago

When you self-host, the government has to come to you for your data. When you use third-party services, the government can go to them. The third party might not fight the request the same way you would. And, you might not even know it happened. The third party might be expressly forbidden from telling you it happened, in fact. This was why Hillary Clinton wanted to host her personal email in her basement. A physical server that she owned, on property she owned; there was no legal way to request that data without going to her personally. If she had used the State Dept server for her personal email, Congress could have accessed all her personal emails simply by asking State to send them over. That’s a controversial example, but the same principle is followed by many companies and organizations who have kept some portion of their data self-hosted. It’s often email or some core of file storage that they consider legally sensitive. This is getting harder to do, though. Look at the recent revelation that the government tried to get newspaper email metadata from Proofpoint, a spam filter provider. Self-hosting a good spam/phishing filter seems almost impossible in 2021, because of the huge amounts of data needed to train filters well.

ceejayoz 5y ago
> When you self-host, the government has to come to you for your data. Sure, but my ability to stop them is probably substantially smaller than, say, Amazon’s legal departments capabilities.
- christophilus 5y ago
  Amazon probably won’t even try.
  - ceejayoz 5y ago
    They’ve clearly and openly committed to trying for years. https://www.computerworld.com/article/2705826/amazon-web-services--we-ll-go-to-court-to-fight-gov-t-requests-for-data.html https://www.computerworld.com/article/2705826/amazon-web-ser... Even Twitter doesn’t like to roll over, and they’ve got a lot less at stake. https://www.latimes.com/politics/story/2021-05-17/twitter-fights-justice-department-subpoena-over-rep-nunes-parody-account https://www.latimes.com/politics/story/2021-05-17/twitter-fi...

stared 5y ago

Your website goes down because there is a temporary power shortage. Your website goes down because it lands on HN... but still, there is a non-trivial cost when there are no visitors. Your website goes down (or worse: gets hijacked) because you forgot to install the newest security update. Your website goes down because, at some point, you lose effort in maintaining it actively. My opinion is that the best way is to host through GitHub (or a similar service). It is up, the way you want. Yet, if anything goes wrong, you still have everything, and there is little friction to push it somewhere else. Sure, if you want to self-host as you have as a DIY project, excellent. For a reliable, safe, cheap, long-term way of sharing data, it is unlikely to be an efficient solution.

Avamander 5y ago
> Your website goes down because it lands on HN... but still, there is a non-trivial cost when there are no visitors. A bit of optimization goes a long way, but yes there are limits. Now the question is this risk worth mitigating? I think for small sites that rarely end up on HN, not more so than avoiding reliance on GH for example.
- stared 5y ago
  If you want to optimize your site for no visitors, you don't need to self-host it anyway.
  - ekianjo 5y ago
    Self hosting is not just for ONE service. You can run an array of local services as well on the same machine.
ekianjo 5y ago
> Your website goes down because there is a temporary power shortage. UPS are cheap these days and easy to set up. > Your website goes down because it lands on HN... but still, there is a non-trivial cost when there are no visitors. It would not if you host a static page. > Your website goes down (or worse: gets hijacked) because you forgot to install the newest security update. backups, clean and re-install? Painful the first time you have to do it, then you build good habits from it. > My opinion is that the best way is to host through GitHub (or a similar service) Enjoy your DMCA take-downs for no reason now and then, and good luck fixing that on your own.

rob_c 5y ago

Yes, more of this!

jmull 5y ago

Self-hosting is a great learning experience and can be fun for personal projects. Beyond than that, I think this quote sums it up pretty well: "Asking everyone to host their own services is not even utopia but rather stupid" (BTW, this is from the linked article.)

that_guy_iain 5y ago

I think with GDPR we'll end up seeing more self hosting at companies. Outsourcing it to SaaS platforms is easy but I think we'll start seeing more and more hassles with privacy and data breaches from SaaS platforms and companies who use them getting fined for not doing due dillegence.

stayux 5y ago

Self hosting is logical if you want to have control over your brand. Even if circumstances arise (scalability, bigger audience) you can always design your architecture with focus over maximum control. In this moment in time, with this prices if you are software professional and you are not self hosting your blog (or having maximum control over your intellectual property) you are lazy. :)

bovermyer 5y ago

This article touches on something interesting: community hosting. I'd like to explore that. Specifically, the idea of small communities where a group of people maintains the underlying tech, and - kicker here - everyone in the community knows more or less everyone else in the community. That offers a bit more security/safety/continuity than just self-hosting everything, while still not ceding control to a faceless corporation. Granted, there will always be other reliances outside of the community - like internet and electricity providers - but a line has to be drawn somewhere.

mnahkies 5y ago
I can't remember the name now, but I picked up a flyer for a place just like this based in Amsterdam at fosdem once. It was very reasonably priced, you were able to have physical access and is more of an enthusiast club than a business. Seriously considered it, but I don't live in Amsterdam and they recommended being able to speak Dutch to participate properly.
- ValentineC 5y ago
  > It was very reasonably priced, you were able to have physical access and is more of an enthusiast club than a business. Sounds like it might be one of the hackerspaces there: https://wiki.hackerspaces.org/Amsterdam https://wiki.hackerspaces.org/Amsterdam
derbOac 5y ago
This has been on my mind lately a lot with Nextdoor. I have really mixed feelings about Nextdoor which is a slightly separate issue, but it always seemed to me that something like Mastodon or maybe even SSB would be an ideal use case in the same space as Nextdoor. You could have local communities around local servers, that have some natural reason to organize about that (geography), but are still loosely federated. I'm not sure where my thoughts are going, as I'm not exactly surprised Nextdoor has more use than a more decentralized system for this use, but it's salient to me as I'd think something like SSB or Mastodon would ideally occupy the space that Nextdoor is occupying. I'm not sure if it is highlighting the legwork that Nextdoor did to build up its userbase (physically mailing people in a community), or the lack of technical sophistication of users in general, or the relative infancy of Mastodon/SSB/etc, or something inherent about getting a foot in the door with decentralized stuff in terms of mindset, or some inherent limitations of decentralization (can you really just compel/convince people to use decentralized services? People just use them). I'm trying to imagine, for example, local police posting to Mastodon about some local safety issue in the same way as on Nextdoor. With Nextdoor, it's something known nationally, the state probably gives them recommendations, they just post to Nextdoor. Nextdoor might have even reached out to them. With e.g., Mastodon, I suppose I could see it being recognized as a thing if use got up, but where are they posting? The local popular servers? Do they run their own police server? Some kind of city government server? This isn't a criticism of decentralization -- I'd like to see everything more decentralized. I just think something like Nextdoor is an interesting case to me to think through these issues because Nextdoor is so localized, and it seems like that's kind of the ideal use case for decentralized services.

gdsdfe 5y ago

This kinda make it sound like everyone have time, money and know how to setup, host, secure and maintain all this stuff ... Is it important yeah absolutely, is convenience important ? also yes ... Which is more important, well choose your own adventure!

superbaconman 5y ago

I've never heard of CGNAT before this thread but add that to ISPs downgrading upload speeds, and refusal to allocate residential v6 space... Our whole industry is out to kill technical independence.

gruez 5y ago
>Our whole industry is out to kill technical independence. Get a grip. The reasons you listed either don't really impede self-hosting, or have more benign explanations than "they're out to kill technical independence". >CGNAT Because ipv6 rollout is hard, and even if you do have ipv6 rolled out in your network, you'll still need ivp4 for vast portions of the internet >ISPs downgrading upload speeds Because spectrum on the wire is limited, and most consumers download. It's not really logical to allocate the spectrum evenly across upload/download just so a few people self-hosting can benefit to the detriment of everyone else. >refusal to allocate residential v6 space I'm presuming you're talking about ISPs that only allocate a /128? It's not really clear how this impedes self-hosting. You just have to set up port-forwarding, which presumably you have tho skills to do if you're the type of person to self-host.
- throw0101a 5y ago
  > Because ipv6 rollout is hard, and even if you do have ipv6 rolled out in your network, you'll still need ivp4 for vast portions of the internet The experience of the ISP Free in France: After having had a succinct presentation of the 6rd idea, a major French Internet service provider (ISP), Free of the Iliad group (hereafter Free), did all of the following in an impressively short delay of only five weeks (November 7th to December 11th 2007): 1. obtained from its regional Internet Registry (RIR) an IPv6 prefix, the length of which was that allocated without a justification and a delay to examine it, namely /32; 2. added 6rd support to the software of its Freebox home-gateway (upgrading for this an available 6to4 code); 3. provisioned PC-compatible platform with a 6to4 gateway software; 4. modified it to support 6rd; 5. tested IPv6 operation with several operating systems and applications; 6. finished operational deployment, by means of new version of the downloadable software of their Freeboxes; 7. announced IPv6 Internet connectivity, at no extra charge, for all its customers wishing to activate it. More than 1,500,000 residential customers thus became able to use IPv6 if they wished, with all the look and feel of native IPv6 addresses routed in IPv6. The only condition was an activation of IPv6 in their Freeboxes, and of course in their IPv6-capable hosts. * https://datatracker.ietf.org/doc/html/rfc5569 https://datatracker.ietf.org/doc/html/rfc5569 * https://en.wikipedia.org/wiki/IPv6_rapid_deployment https://en.wikipedia.org/wiki/IPv6_rapid_deployment This was ten years ago, so the Internet was less integral to people's lives (relatively speaking). Some more testing may be needed nowadays for IP end-nodes, but I'm not sure if things in the network infrastructure would be any more challenging.

unixhero 5y ago

Cheers from /r/selfhosted[0]! 0, Awesomelist selfhosted, https://github.com/awesome-selfhosted/awesome-selfhosted https://github.com/awesome-selfhosted/awesome-selfhosted

scottydelta 5y ago

As someone who self-hosts a lot of different apps, self-hosting is really a slippery slope. Once you start enjoying the control over the system and data, you want to self host everything. The most important aspect is the security and you learn this by doing it. My entire self-host apps are hosted behind a private VPN called Pritunl, it provides self-hosted corporate VPN like setup where you can manage users and access to servers. I host these following apps/products right now: - Pritunl (corporate like VPN) - Superset (Analytics) - Bitwarden (Password Manager) - OpenVPN with Pihole (Personal VPN with Adblock) - Wireguard with Pihole (Personal VPN with Adblock) - Drone.io (CI/CD) - Posthog (Web Traffic Analytics) - Papercups (Web chat support)

hinkley 5y ago
I think the team dynamic can be just as important from the “host everything” standpoint. Hosts generally have incentives to automate manual processes, and a diverse set of customers pushing to make that automation sane, for some value of sanity. There’s a struggle against manual processes in self hosted environments, or aggressive automation with bespoke or otherwise incomplete tools. What you want is glue code holding together open source tools without too much abstraction over the top. You should always have a hint what’s going on underneath. I find myself having to spend way too much social capital on this. While I much prefer self hosted, there is a clear advantage of third parties inasmuch as you can bond over the stupid things their solutions do, instead of driving wedges between teams by engaging in that kind if catharsis.
gizdan 5y ago
Out of curiousity, what is reason to host three different VPNs?
- scottydelta 5y ago
  I have my own bootstrapped startup since a year and half and I manage the entire infrastructure for my startup. My startup focuses on big data projects and is currently building a web-based Bloomberg Terminal alternative(https://quantale.io https://quantale.io) which is an infrastructure heavy project. here is why I use 3 VPNs: 1. Pritunl is used as an enterprise VPN setup. Using this VPN, I provide access to different webapps/self-host to different users in my team/clients. for eg, the stage version of Quantale is hosted on a server(with ufw rule to only allow connection from pritunl VPN IP) and I can provide access to you by creating a vpn config for you and you will only be able to access stage server, you wont be able to access any other servers behind this VPN. 2. OpenVPN with Pihole, I use this VPN as a personal VPN. This VPN blocks ads and trackers using PiHole and my self-hosted password manager is only accessible via this VPN 3. Wirehole with PiHole, this is a backup VPN to access my password manager in case I loose access to my OpenVPN server.

candiddevmike 5y ago

If you're looking for a self hosted "household management" solution, checkout Domestica--https://about.domestica.app https://about.domestica.app. Calendars, budgets, recipes, tasks and more, all integrated and hosted on your stuff.

hackernudes 5y ago
To save everyone else a click: "You must have an active Domestica Premium subscription to use the self-hosted version." From https://hub.docker.com/r/candiddev/domestica https://hub.docker.com/r/candiddev/domestica
- candiddevmike 5y ago
  Yep, however a bit further down it says you can do this as a one time purchase and keep using Domestica, forever, but you won't get updates after your subscription expires. I should probably update the docs to emphasize that more. This is a side project of mine, hopefully someday I can make the self hosted version free or OSS.
- donmcronald 5y ago
  I don't have a problem with the way that one's set up because it's not some ridiculous pay per user per feature per month scheme like most SaaS. Paying $X per year for continual software updates without limitations on user count or features is good value and the self hosting option creates a natural limit on the scale (ie: no one's going to host 10k users on an instance). Compare it to something like GitLab where I have the burden of maintaining it plus I still have to pay per user per month and deal with ridiculous feature tiers. Which one is better value and less hassle for me? There's a big difference IMO.

kosasbest 5y ago

Isn't self-hosting usually forbidden by most ISPs? I've read countless times in the ToS of ISPs that running your own server is against their terms. Also: if you suffer from frequent power outages, you have to have a server that 'bounces back' and survives the outage, with all the necessary services up and running after a reboot. Good luck writing scripts for that scenario.

hellbannedguy 5y ago
I have heard the same for years. I have a feeling they forbid it in tos, but don't enforce it in most areas? Does anyone know for sure if Xfinity in Marin County, Ca allows self-hosting?
- hoppyhoppy2 5y ago
  >[You may not] use or run dedicated, stand-alone equipment or servers from the Premises that provide network content or any other services to anyone outside of your Premises local area network (“Premises LAN”), also commonly referred to as public services or servers. Examples of prohibited equipment and servers include, but are not limited to, email, web hosting, file sharing, and proxy services and servers https://www.xfinity.com/Corporate/Customers/Policies/HighSpeedInternetAUP https://www.xfinity.com/Corporate/Customers/Policies/HighSpe... This applies to Comcast's (at least residential) customers nationwide. But I, like you, get the sense that it's there so they can shut down excessive bandwidth use; I haven't heard about them trying to shut down a low-traffic webserver or self-hosted stuff for one's own personal use (though I probably wouldn't try to run a gaming server or anything high-traffic). Just be aware that it is technically against Comcast's Acceptable Use Policy, and try to switch to a better internet provider if you have the option.
alamortsubite 5y ago
> if you suffer from frequent power outages, you have to have a server that 'bounces back' and survives the outage In my experience self-hosting a number of services, it's really no more challenging than managing the services themselves. A BIOS that supports wake-on-power is obviously a key hardware requirement. On the software side, I find systemd and DuckDNS to be very helpful and easy to use. One of my sites is a remote cabin that suffers blackouts all the time. It has the burden of an LTE connection, which means it requires an ssh tunnel to get around the ISP's firewall. This makes autossh an additional, if simple and reliable component.

hinkley 5y ago

I have a more practical take on self hosting. Developers should be talking to the Ops folk. It informs your architecture decisions with practical considerations, like physics, and how many NICs you can plug into a homogenous switch before you have network hops screwing up your pretty but naive designs. When you stop self hosting, the number and quality of those people goes away when they realize they should find someplace else to be. And when we need fewer of them, we stop making new ones. I try to push architectures that allow for a degree of heterogeneity, where we have one data center we own, and use others we don’t for geographic redundancy and speed of light concerns. For a read mostly system 5a Reading an entire zip file's contents and writing out a brand new zip file could be an extremely slow process. For read-mostly systems, that may mean for instance that we keep the system of record (I’m doing just this to bootstrap a personal project that has a read-mostly information architecture) but distribute the UI out into the Someone Else’s Computers.

normac2 5y ago

It's depressing, because I can imagine a world where the government would access people's data, but only under extreme circumstances like investigating terrorist networks. Instead, they use it in outrageously corrupt ways, like collecting vast swathes of communications from everyone and storing them with poor limitations on access. As Chomsky (who I don't agree with on everything by any means) has written, when the government talks about doing things for "security," that usually means security of the government from its own people.

sgt 5y ago

A lot of people here are worried about constantly having to worry about hardware instability for the services they are self hosting. I've been thinking of is to rent colo space for an 1U enclosure, with two Mac Mini M1's inside. Then run them with a tiny Mikrotik router in front that can either load balance or perhaps some other reliable piece of hardware to do HAproxy style functionality. From that one should be able to provide a very reliable server even in the cases of hardware issues on one of them, assuming the DC has decent connectivity and redundant power - all inside one neat little package. As much as I love macOS, I would prefer to run those two Mac Minis on Linux though. Wonder how mature the port is?

moondev 5y ago
If you are focused on arm infrastructure for linux, you can do a lot better than a Mac mini m1, they are limited to 16GB of RAM and booting Linux is experimental at best. Check out something like the solidrun honeycomb lx2 which can take 2 32G sodimms: https://shop.solid-run.com/product/SRLX216S00D00GE064H08CH/ https://shop.solid-run.com/product/SRLX216S00D00GE064H08CH/ Designed for linux out of the box, and you can even run VMware esxi for arm and go wild with vms. https://flings.vmware.com/esxi-arm-edition https://flings.vmware.com/esxi-arm-edition
- sgt 5y ago
  Yeah but without the outstanding performance of the M1 chip I then I might as well just buy non-ARM. Saying M1 is comparable with any run of the mill ARM based CPU is the understatement of the decade. I will look at how stable the port is.

kaydub 5y ago

The biggest most successful and best run organizations I've worked for have not worried about hosting their software. They're concerned with delivering value to customers and doing it quickly. The most fractured and worst run companies have been concerned with self-hosting and frankly they sympathized with a lot of the stuff I'm seeing on here. But I recognize we still need people self-hosting because it drives innovation and competition. I believe there are ways of doing things well while self-hosting, but I'm not sure what those ways are.

deleted 5y ago
[deleted]
donmcronald 5y ago
I don't think it's a yes or no, right or wrong kind of thing. For an independent / small developer I think there's value in self hosting your tools, but I think it would be crazy to self host a customer facing app. There's scaling and redundancy that you can take advantage that you could never come close to hosting on your own. However, for tooling like VCS and CI, I think maintaining control is extra important when you don't have any negotiating power. For mid-sized companies where you're paying someone to maintain things, whether that's an employee or a 3rd party, I think you need to assess your tools in terms of your negotiating power and how important it is to maintain control of everything. What if GitHub bans you? I don't think I'd try to self host a customer facing app at this scale either. I think you can be short-term successful by throwing caution to the wind and using every shortcut available, but will the first to market advantage be enough to offset the price competition of people that aren't locked in to some proprietary API gateway or WAF? For example, what if I take extra time to build on OpenFaaS and you build on AWS everything. Who wins long term? You're faster, but I have better negotiating power (ie: less costs) by threatening to switch vendors. Or is the idea of switching hosting vendors detached from reality at this point? Is hosting cost so negligible it doesn't matter? All I know is that everything looks crazy expensive from where I am.
- kaydub 5y ago
  > What if GitHub bans you? This is one of those things that I was talking about them sympathizing with. It's unrealistic. If you're getting banned from github or a cloud provider maybe you should reconsider what you're doing. > You're faster, but I have better negotiating power (ie: less costs) by threatening to switch vendors. Okay, but AWS isn't expensive as it is. If you build to the way these resources are meant to be built you can seriously minimize your costs. I've seen plenty of lift and shifted apps get their costs dramatically reduced once redesigned in a "cloud native" architecture. The lift and shifted design was in the thousands of dollars a month. Rebuilding for AWS resources brought our costs to almost literally nothing on this same app. I'm talking $5,000 a month to $5 per month. So I have a hard time following the premise that you'll do it cheaper with an on-prem or self-managed system.

duped 5y ago

My company recently migrated from OTS 3rd party apps hosted on their clouds that had frequent downtime and high costs, and self hosted a highly touted and battle tested solution instead. It's a disaster and costs us far more time in developer hours than we spent before. It turns out infrastructure isn't free and paying someone else to care about it scales very nicely.

deleted 5y ago

[deleted]

surfsvammel 5y ago

I go back and forth between hosting everything myself for a year or two, then deciding its not worth the time and effort and just signing up for what ever services I need. Then I go back to self hosting again a year later... I am not sure why I self host. I think it's the engineer in me that just needs to understand how things work, and I need to tinker. It's a hobby more than anything.

Havoc 5y ago

I’ve also found it massively beneficial for learning. A virtualisation server at home makes it much easier to experiment. After a bit of gathering knowledge and ansible recipes the various bits start getting synergies and more becomes possible a la Lego.

kaetemi 5y ago

More important than that is just using portable technologies that you can easily use a hosted version of, as well as be able to run the whole thing on your own machine. If you can't even run your own code yourself you're going to be in debugging hell (these kinds of teams do exist).

jlkuester7 5y ago

IMHO, containerization technology (Docker) has changed the game for self-hosting. I run a bunch of different services all segregated in their own containers behind a Traefik container that rounts to all of them based on domain/subdomain. I don't have to spend a bunch of time wording about incompatible libraries or versions because everything is self-contained and I can upgrade each service individually without worrying about affecting the other services for the same reason. I can configure it all up simply with docker-compose.

nazrulmum10 5y ago

Why is self hosting exactly? § Self hosting is about freedom, you can choose what server you want to run, which version, which features and which configuration you want. If you self host at home, You can also pick the hardware to match your needs (more Ram ? More Disk? RAID?). Self hosting is not a perfect solution, you have to buy the hardware, replace faulty components, do the system maintenance to keep the software part alive.

akho 5y ago

I have a home NAS, accessible through a Wireguard network set up on the router. Am I self-hosting Synology's things (for photos / music / video / downloads)? Synology also supports Docker, so I have - Gitea - Vaultwarden - Nextcloud - pihole - Miniflux (+ rssbridge) running there. These I would definitely say I self-host. I'm also happier with these choices (with the exception of Nextcloud, which I'm replacing with Syncthing) than I would be with any commercial / cloud replacement. Storing passwords outside your control just is not sane. RSS readers on the web suck and want money to read a well-established file format from a public source. Gitea is really nice for things I don't want on GitHub or public, but want to be able to look up on the go (my Org notes, dotfiles, any short-term miniprojects, &c). Synology's apps are used for high-volume stuff, which would combine into quite a large regular bill if done "in the cloud".

jesterson 5y ago

Can't agree more with the sentiment. We are self-hosting everything we can and quite happy with it. From security and management standpoint it's incomparably better. It does have it's management overhead, however for years we managed to create processes and workflows keeping it's to bare minimum. Contrary to popular opinion I believe that self-hosting will grow, at least for solutions requiring immense security and reliability