4 ms·

Self hosting is hard because we are limited by our ISPs and what they allow us to do with our connections. Why don't they allow people to self host and when tra

by ozfive 5y ago

Self hosting is hard because we are limited by our ISPs and what they allow us to do with our connections. Why don't they allow people to self host and when traffic starts to get big have an option to migrate that data to the cloud instead of clogging the municipal pipes if that's really the problem. The way companies like Comcast and DSL providers can succeed in this arena is to make self hosting easier. Why do I as an end user need to understand the infrastructure when all I am trying to do is share data or information?!?! I am playing the role of an uneducated user here and not as a Software Engineer as a career. It's disheartening that basic things like this aren't solved for people on the ISP level. I should have to find a service like Weebly or, DO or AWS, etc. I just want to be able to share my info and my ISP at the very least should be able to provide the basic framework for me to do it. When my content becomes populate then adjust accordingly.

aetherspawn 5y ago
I don't think that anyone self-hosting is able to make a single dent in the consumer internet infrastructure. Usually when you're self-hosting I guess you're handling either just your friends and family or perhaps at most up to 1000 strangers that follow your hobby. Hypothetically in a future where self hosting is non hostile, will you see people self-hosting startups and the like? Yeah, maybe feasible. I think at that scale you start to care about things like uptime and maintenance. But I think the biggest winner to self hosting is the photographer who saves $5/mo on the blog that people rarely read or the kid who doesn't have to pay $10 to play Minecraft with their friends, or the family friend with 2TB of family data that doesn't want to pay $20/mo for Dropbox or the like when they already have the hard drive to store it. Yes it needs to be simpler for these people! There's a whole economy in making it difficult for these!
- vincnetas 5y ago
  " I think at that scale you start to care about things like uptime and maintenance." I guess none of members of your mentioned groups will ever grow to care about a scale. And thats a good thing because they are distributed. And i think it creates a better internet, which is kind of wide network instead of shallow graph of couple mega nodes.
- TeMPOraL 5y ago
  The biggest win we could achieve for self-hosting doesn't involve people actually self-hosting. The idea of personally maintaining your own infrastructure is unlikely to scale to general population - but the thing we're really after is ownership of data and the ability to own infrastructure. So, I think, the ideal situation would be to have a combination of big and small companies offering storage and compute as a commodity. You'd pay a fee to keep your stuff hosted somewhere; any time you see a better offer, you can migrate to a different provider without much hassle, and with near-zero downtime. Cloud services would work by shipping their code to your data, not the other way around[0]. And if you were so inclined, you could just build your own infra, or even buy a turn-key "self-hosting in a box" kit. Pieces of that vision are already here. Compute providers are plenty. You can order "self-hosting in a box" kits. Internet architecture makes everyone's computer equal (at least in theory, ISPs mess it up with NAT, and their T&Cs). The only thing missing is the part where you own your data, and SaaS vendors serve you - the bit that makes SaaS truly be Software as a Service, instead of Serfdom as a Service. -- [0] - Preferably with homomorphic encryption preventing SaaS vendors from putting their hands in the cookie jar, if we can get that to work without creating another blockchain-level environmental disaster.
  - rambambram 5y ago
    You mean good old webhosting? What I don't get about the HN crowd is making every simple, already existing, already proven, and already solved problem so damn hard. I'm in EU, I host my websites at multiple local webhosting companies. They're small enough to care about support and big enough to guarantee speedy and reliable service. By law they're not allowed to go through my data (ofcourse they can and I have no way of proving that they did), so the legal deal between me and them is crystal clear. Who do you call when your Amazon Web Shit serverless thingy doesn't work no more? I can and did move several webapps and websites from one webhosting company to the other. It works flawless. And besides waiting a couple of hours for a DNS change, it's almost instant. I get it, you can't do that easily with a system with million users. This so called problem was already solved decades ago. It was called personal computers and the internet. When people started calling the internet 'the cloud' then things went downhill. Excuse me for the slight rant. I should go outside and see some more sun. ;)
- rapnie 5y ago
  > There's a whole economy in making it difficult for these! And there is your niche right there. Individual companies or persons do not need to make 'dents in markets' all by themself. A host of people doing the same might. But for the individual - especially when having sustainable income objectives, not hockeystick growth - there's a good place in the market I think.
- El_RIDO 5y ago
  > Hypothetically in a future where self hosting is non hostile, will you see people self-hosting startups and the like? Huh, is that not a thing anymore for startups today? In my experience, starting in 2003, across two companies, was that if you need any service fast and on the cheap, you're forced to host it yourself. Run your own mail server, web server, DNS, server housing, all the web apps, etc. and of course spend most of the day developing your actual product. Have the hosted options actually gotten so cheap and reliable these days? Is it a mindset thing? My gripe with the cloud or SaaS solutions I've used over the years is always that management of them and backup is difficult tending towards impossible and without those you can't rely on these services, it's just voluntary vendor lock-in. When self-hosting, your actually (forced to) learn how they work and be able to fix them. Self-hosting, to me, is the simpler, more reliable option, if you depend on a service for your business and it enables you to get it fixed yourself, when it is not working. It doesn't prevent you to out source the work, if you have the money, but worst case, you still have direct access to your property. With the hosted options, in my experience, you get all these fancy promises on availability and it being the latest, smartest tech and then the service is down for half a day, your data got restored from days old backups (if at all) and there is nothing you can do except tell your customers that "were sorry and working on it" while you wait for it to come back. :-(
  - fauigerzigerk 5y ago
    >and then the service is down for half a day, your data got restored from days old backups (if at all) It's not unheard of for the same thing to happen with in-house systems. The difference is that you may have a wider range of options to avoid and respond to any outage if you run things yourself. That seems to be a rather theoretical advantage though. Every time there's a new ransomware attack, it's always those in-house deployments that are hit hardest and take the longest to recover. I think under optimal conditions self-hosting is superior. But conditions are rarely optimal. As soon as you have to convince non-technical management to invest in non-productive necessities or in contingency planning you're already in a sub-optimal position.
  - jokethrowaway 5y ago
    It's economically valid reasoning but most startups (even one man band) are happy to pay a premium for the cloud.
TheOtherHobbes 5y ago
Self-hosting isn't hard, it's currently impossible. The real problem is who owns your data. Because if you use Word or Photoshop, your files are locked inside Word and Photoshop. And this is still true if you use FOSS alternatives, because there's only very limited support for metadata-aware sharing between applications of all kinds. It would be super-useful to have (for example...) seamless links between text editors, web design applications, web hosting systems, and even video editors and ebook publishing tools. But that's not where we are now. There's some limited interchange, but most cross-domain transfers are difficult and fragile, and some are impossible. Cloud is just the online version of the same model. When you have proprietary control of user data through proprietary file formats which actively frustrate open sharing of data between applications, it doesn't matter of the data is stored locally or in the cloud. It also doesn't matter if you're using a mobile or desktop UI. The FOSS people have always been looking through the wrong end of the telescope. The real revolution would be open data which is wholly and exclusively owned by users (or user groups for collaboration) and loaned to proprietary software for specific limited tasks. Which is the opposite of things work now. Applications and products own your data and they let you access it - but only if you ask them nicely. And - increasingly - if you pay annually for the privilege. So containerisation or self-hosting or whatever is a non-solution unless it also gives data back to users. Which is also why the fragments idea won't work. There's limited use in trying to automate or manage or otherwise AI-ify access to data that you don't truly own anyway. In fact a new kind of shared Internet would be a very useful thing. But it would need a ground-up redesign of everything, including browsers, mobile apps, desktop applications, operating systems, search, and the financial and legal frameworks surrounding them. I'd love to see that happen. But right now in 2021 it just doesn't seem likely.
- Santosh83 5y ago
  > The real revolution would be open data which is wholly and exclusively owned by users (or user groups for collaboration) and loaned to proprietary software for specific limited tasks. How would that work in practice? By its very nature open data would also be accessible to proprietary programs although the reverse need not be the case. Seems like we will need open data and code. Either one open will not do.

by ozfive 5y ago

aetherspawn 5y ago

I don't think that anyone self-hosting is able to make a single dent in the consumer internet infrastructure. Usually when you're self-hosting I guess you're handling either just your friends and family or perhaps at most up to 1000 strangers that follow your hobby. Hypothetically in a future where self hosting is non hostile, will you see people self-hosting startups and the like? Yeah, maybe feasible. I think at that scale you start to care about things like uptime and maintenance. But I think the biggest winner to self hosting is the photographer who saves $5/mo on the blog that people rarely read or the kid who doesn't have to pay $10 to play Minecraft with their friends, or the family friend with 2TB of family data that doesn't want to pay $20/mo for Dropbox or the like when they already have the hard drive to store it. Yes it needs to be simpler for these people! There's a whole economy in making it difficult for these!

vincnetas 5y ago
" I think at that scale you start to care about things like uptime and maintenance." I guess none of members of your mentioned groups will ever grow to care about a scale. And thats a good thing because they are distributed. And i think it creates a better internet, which is kind of wide network instead of shallow graph of couple mega nodes.
TeMPOraL 5y ago
The biggest win we could achieve for self-hosting doesn't involve people actually self-hosting. The idea of personally maintaining your own infrastructure is unlikely to scale to general population - but the thing we're really after is ownership of data and the ability to own infrastructure. So, I think, the ideal situation would be to have a combination of big and small companies offering storage and compute as a commodity. You'd pay a fee to keep your stuff hosted somewhere; any time you see a better offer, you can migrate to a different provider without much hassle, and with near-zero downtime. Cloud services would work by shipping their code to your data, not the other way around[0]. And if you were so inclined, you could just build your own infra, or even buy a turn-key "self-hosting in a box" kit. Pieces of that vision are already here. Compute providers are plenty. You can order "self-hosting in a box" kits. Internet architecture makes everyone's computer equal (at least in theory, ISPs mess it up with NAT, and their T&Cs). The only thing missing is the part where you own your data, and SaaS vendors serve you - the bit that makes SaaS truly be Software as a Service, instead of Serfdom as a Service. -- [0] - Preferably with homomorphic encryption preventing SaaS vendors from putting their hands in the cookie jar, if we can get that to work without creating another blockchain-level environmental disaster.
- rambambram 5y ago
  You mean good old webhosting? What I don't get about the HN crowd is making every simple, already existing, already proven, and already solved problem so damn hard. I'm in EU, I host my websites at multiple local webhosting companies. They're small enough to care about support and big enough to guarantee speedy and reliable service. By law they're not allowed to go through my data (ofcourse they can and I have no way of proving that they did), so the legal deal between me and them is crystal clear. Who do you call when your Amazon Web Shit serverless thingy doesn't work no more? I can and did move several webapps and websites from one webhosting company to the other. It works flawless. And besides waiting a couple of hours for a DNS change, it's almost instant. I get it, you can't do that easily with a system with million users. This so called problem was already solved decades ago. It was called personal computers and the internet. When people started calling the internet 'the cloud' then things went downhill. Excuse me for the slight rant. I should go outside and see some more sun. ;)
rapnie 5y ago
> There's a whole economy in making it difficult for these! And there is your niche right there. Individual companies or persons do not need to make 'dents in markets' all by themself. A host of people doing the same might. But for the individual - especially when having sustainable income objectives, not hockeystick growth - there's a good place in the market I think.
El_RIDO 5y ago
> Hypothetically in a future where self hosting is non hostile, will you see people self-hosting startups and the like? Huh, is that not a thing anymore for startups today? In my experience, starting in 2003, across two companies, was that if you need any service fast and on the cheap, you're forced to host it yourself. Run your own mail server, web server, DNS, server housing, all the web apps, etc. and of course spend most of the day developing your actual product. Have the hosted options actually gotten so cheap and reliable these days? Is it a mindset thing? My gripe with the cloud or SaaS solutions I've used over the years is always that management of them and backup is difficult tending towards impossible and without those you can't rely on these services, it's just voluntary vendor lock-in. When self-hosting, your actually (forced to) learn how they work and be able to fix them. Self-hosting, to me, is the simpler, more reliable option, if you depend on a service for your business and it enables you to get it fixed yourself, when it is not working. It doesn't prevent you to out source the work, if you have the money, but worst case, you still have direct access to your property. With the hosted options, in my experience, you get all these fancy promises on availability and it being the latest, smartest tech and then the service is down for half a day, your data got restored from days old backups (if at all) and there is nothing you can do except tell your customers that "were sorry and working on it" while you wait for it to come back. :-(
- fauigerzigerk 5y ago
  >and then the service is down for half a day, your data got restored from days old backups (if at all) It's not unheard of for the same thing to happen with in-house systems. The difference is that you may have a wider range of options to avoid and respond to any outage if you run things yourself. That seems to be a rather theoretical advantage though. Every time there's a new ransomware attack, it's always those in-house deployments that are hit hardest and take the longest to recover. I think under optimal conditions self-hosting is superior. But conditions are rarely optimal. As soon as you have to convince non-technical management to invest in non-productive necessities or in contingency planning you're already in a sub-optimal position.
- jokethrowaway 5y ago
  It's economically valid reasoning but most startups (even one man band) are happy to pay a premium for the cloud.

TheOtherHobbes 5y ago

Self-hosting isn't hard, it's currently impossible. The real problem is who owns your data. Because if you use Word or Photoshop, your files are locked inside Word and Photoshop. And this is still true if you use FOSS alternatives, because there's only very limited support for metadata-aware sharing between applications of all kinds. It would be super-useful to have (for example...) seamless links between text editors, web design applications, web hosting systems, and even video editors and ebook publishing tools. But that's not where we are now. There's some limited interchange, but most cross-domain transfers are difficult and fragile, and some are impossible. Cloud is just the online version of the same model. When you have proprietary control of user data through proprietary file formats which actively frustrate open sharing of data between applications, it doesn't matter of the data is stored locally or in the cloud. It also doesn't matter if you're using a mobile or desktop UI. The FOSS people have always been looking through the wrong end of the telescope. The real revolution would be open data which is wholly and exclusively owned by users (or user groups for collaboration) and loaned to proprietary software for specific limited tasks. Which is the opposite of things work now. Applications and products own your data and they let you access it - but only if you ask them nicely. And - increasingly - if you pay annually for the privilege. So containerisation or self-hosting or whatever is a non-solution unless it also gives data back to users. Which is also why the fragments idea won't work. There's limited use in trying to automate or manage or otherwise AI-ify access to data that you don't truly own anyway. In fact a new kind of shared Internet would be a very useful thing. But it would need a ground-up redesign of everything, including browsers, mobile apps, desktop applications, operating systems, search, and the financial and legal frameworks surrounding them. I'd love to see that happen. But right now in 2021 it just doesn't seem likely.

Santosh83 5y ago
> The real revolution would be open data which is wholly and exclusively owned by users (or user groups for collaboration) and loaned to proprietary software for specific limited tasks. How would that work in practice? By its very nature open data would also be accessible to proprietary programs although the reverse need not be the case. Seems like we will need open data and code. Either one open will not do.