15 ms·

Taking back control of my digital life

105 points by fallenhitokiri 8y ago

vxNsr 8y ago
This is a great read, but I'd like to hear more about the nitty gritty, what server hardware you use, it seems like you have 3 pis and then a bunch of really advanced enterprise-level stuff for other work... I'd love a guide on what you did to get here. The internet really needs something like that.
- hkajhfjl 8y ago
  I'm not the OP, but I have been doing this for some years now. Here's how I went about it. I got out of social media in early 2012. Deleted FB and Twitter. I do maintain a presence on LinkedIn, but I never post anything there, nor do I login. It's mostly there because employers expect a LI profile. It's just a bland copy of my resume. I moved my email from Gmail to my own domain from 2013-2015. It took almost two years because I had Gmail since April 2004, a few days after launch. My whole digital life was tied into Gmail, and moving stuff over takes time. I still have that Gmail account, but again, I never log into anything Google, and it simply forwards any email to my real account. I get maybe 1 email a month from Gmail. I host my email with Fastmail, but I've also selfhosted, and tried O365. It's easy to switch, and I keep an offline backup of everything. Now, my hardware and software setup. At home, I have a Synology NAS with ~12 TB of usable storage. This is my primary data store - I have software that automiatically backs up all photos from my Iphone, Time Machine, Veeam, Rysnc, and email backup. I own my own physical server (Dell R720XD) which is colocated in a datacenter in a different country. This server has 96 GB of RAM, 16 TB of storage, and 12 Xeon cores. It usually runs between 12-25 VMs, some of which are production (personal), hosting my websites, git, various projects, and so on. It's connected to the internet through a gigabit connection with a 10 TB monthly bandwidth cap (I never come close to 1 TB!). This costs me ~$85 a month. I don't do BGP, though I could if I paid some more and put in a proper router and got my own IP range.. as it stands, the 8 public IPs I get are enough for me. The colo server is connected via an always-on, site-to-site IPSEC VPN to my home LAN (through my EdgeRouter X). I have symmetric gigabit, so it's practically like having the machine in my house. My NAS backs up to the colo server, and the colo server to the NAS (via rsync). The NAS also backs up to a local USB drive, for quick restores. All my home data lives in four places 1. The local PC/laptop/phone. 2. The NAS 3. The USB Disk 4. The offsite colo server. Likewise, all my colo data lives in the same four places. I can VPN in to the colo server and get connected to my home lan from anywhere in the world. Any sensitive data is encrypted - and anything really sensitive is not stored on a computer. This setup works very well for me and my family, and I control all my data and am not dependent on a cloud provider. I also have a bunch of other servers running in my home, but they aren't part of my core infrastructure. The cost? ~$1200 for the R720. ~$1000 for the NAS. and ~$85 a month for colocation costs.
  - dewey 8y ago
    Do you have measures in place to give access to someone trusted in case something happens to you? With a specialized setup like this hosting, for example all family pictures, I’d always be scared of having a low bus factor.
    - hkajhfjl 8y ago
      Yes and no. My spouse can easily access anything family related - pictures, documents, passwords (keepass) from their computer. The NAS and the Colo Server both present as shared drives on the network and they use it almost every day. The USB drive is plug and play and has copies of anything important. However, all my git repos, websites, tech projects etc will probably not survive my passing. I don't really care about that. They will still be backed up and accessible on the NAS, so if someone in my family (I have a few technically adept people in the family) wanted to check it out, my spouse would be able to give them access. Honestly, I don't think anyone would care, but who knows. All passwords and access details are in KeePass and my spouse uses that everyday so knows how to use it, My colo server is paid a year in advance every year, so it would chug along for a few months before the provider shut it down and shipped it back to my house (again, my spouse knows how to get it shipped back). All in all, I think my family would be able to access anything truly valuable without too much trouble, and who cares about the other stuff anyway.
  - voltagex_ 8y ago
    Few more questions: * How do you backup Fastmail? * How'd you find your colo provider? * If it's in a different country, how does the latency affect you?
    - walterbell 8y ago
      Ditto on the colo provider question. Which virtualization do you use on server and how do you handle remote admin and security updates for guest VMs and host?
      - hkajhfjl 8y ago
- fallenhitokiri 8y ago
  OP here - since there seems to be some interest on the details I will try to answer them in as much detail as possible in a longer blog post. I already wrote about the network setup[1] The server is a Dell Precision T5610 (20 physical cores, 64GB memory, 4 Samsung Pro SSDs and quad gigabit NIC additionally to the internal one) and a QNap TS-531X with Seagate IronWolf drives. I would recommend against using PIs at this point, see other comment[2]. Right now all services are running in docker in a dedicated VM on VMWare Workstation. The server also hosts a full work environment I can RDP and SSH into, depending on what I do, when I am traveling only with my iPad Pro (RDP because my last job involved a lot of Java for which I prefer IntelliJ). At some point I should likely migrate to vSphere, but I had the workstation license already and it was quick to setup. [1] https://screamingatmyscreen.com/2018/7/building-our-home-and-office-network/ https://screamingatmyscreen.com/2018/7/building-our-home-and... [2] https://news.ycombinator.com/item?id=17966264 https://news.ycombinator.com/item?id=17966264
  - 8fingerlouie 8y ago
    I'm probably beating a dead horse here with Docker Policies available and all, but i personally don't trust Docker in production. Each Docker image runs it's own stack, and each stack has the potential to contain vulnerabilities. Even with services such as Watchtower ( https://hub.docker.com/r/v2tec/watchtower/ https://hub.docker.com/r/v2tec/watchtower/ ), you're still not safe. Some images are abandoned for years, others are only updated once their "final" product is updated, meaning you could be hosting a handful of vulnerable services without even knowing it. I host all my internet facing stuff on FreeBSD in jails, though Linux with LXC would do just as good. I have one stack to update, and once that's updated, everything else is updated as well.
    - fallenhitokiri 8y ago
      Nothing I host is exposed to the Internet but only available locally. When on the road I VPN back home. But generally I agree, for Internet facing services I would likely make a few different decisions than in this case.
fouc 8y ago
Recently I was wondering what if all our phones and computers all connected to our own personal "platform" that we have full control over. Route everything on the phone through a personal platform. Gain control of location, IP, maps caching, etc. Centralize at a personal level.
- CalRobert 8y ago
  To an extent, this is kind of the idea behind sandstorm.io. Run the services you want on a box under your control (or a vps, but still yours). Given that there are finally good non-google options for a lot of these (OSM + Thunderforest + Graphhopper gives a really nice map experience for instance) I'd love to see it go further. As always, it's hard to find time to work on this though.
- nwienert 8y ago
  I’m working on almost this exact thing. Still a ways off but get in touch if interested.
- close04 8y ago
  A platform is useful for the other services it provides: Google funnels your data but give you better search, MS swipes some info about your PC to give you better drivers, Facebook takes all your likes and dislikes to give you stories to your taste. [talking about stated interest here] What would your personal platform do? How would it take that data and turn it into something useful while also removing the privacy concerns? Best you can do is have something that blocks or spoofs that data before sending it to the "collector". So you gain the privacy and you lose on the service improvement. If this is legitimate concern you can try to drop the most intrusive platforms from your digital life. These would most likely be Google and Facebook. Alternatively there are options for you to use with more limited effect: ad and tracker blockers, secure or anonymous browsers, VPN, stricter privacy settings on your devices, etc. The end result is usually the same: the less data you provide them, the less useful their service is. Which makes trading your info even less enticing.
  - ACS_Solver 8y ago
bthornbury 8y ago
Very cool, would love to see more content on this topic. Setup guides of individual services, issues you've run into, more on hardware choices, and definitely more on your smart home setup.
- fallenhitokiri 8y ago
  OP here, thanks! I posted a few more details here[1] and will follow up with an in depth blog post covering as much as possible rest. For the smart home setup I am mostly using Elgato Eve SmartPlugs, their door and window sensors and Homebridge for everything that does not integrate with HomeKit natively. Any light source I cannot directly plug into a power outlet (ceiling mounted e.x.) is at least remote controllable via RF, not IR, and I am currently working on figuring out how to send the control signals through an PI or Arduino with an RF transceiver. [1] https://news.ycombinator.com/item?id=17966311 https://news.ycombinator.com/item?id=17966311 [2] https://github.com/nfarina/homebridge https://github.com/nfarina/homebridge
onefish 8y ago
Recently I've been playing around with something like this, or really; getting back to something like this. Years ago I was an idealistic graduate student who didn't have any "free" (you-are-the-product) digital service or social media accounts - I used Lavabit (before it got shut down) for email and a couple of cheap VPS providers to run a little blog and IRC bouncer. I had a dumb "burner" phone, a RockBox-based MP3 player, ran Tomato on my router, and ran Debian on most everything else. I think the main factors that killed it for me, that made me drink the Kool-Aid so to speak, were a combination of getting an Android phone and the shutdown of Lavabit (with all the hassle that incurred - notifying dozens of colleagues, mailing lists, etc of the change). Concurrently with this I had just gotten my first industry gig at a pretty large networking equipment company, on a team with mostly older, mellowed, senior engineers for whom programming was "just a job". I didn't want to seem like a "paranoid weirdo" who had some black-hat alter ego. I sold out. Since, I've pretty much moved wholesale to Google. I still don't use any other digital services - Google has basically become my one-stop shop, for better or worse. I use Android, Chrome, Gmail, Drive, Music, Books, Search, Maps, Keep, Photos, basically the whole damn suite. It's a beautifully unified and seamless experience. I feel in-general, Google gets just about everything right (I don't use Docs - I still write docs in LaTex, haha). It's quite a 180 from what I had before. But ultimately, I think this has caused me a lot of cognitive dissonance. I've spent a lot of time thinking about how to "get back" lately, but this is tempered by how much control I've already given up and, well, what is frankly a pretty damn high quality and convenient experience and there are some things (like Maps, and Photos) which I really don't want to give up. I also don't hold any delusions that anything I do is going to be "more secure" from any threat model, really. I guess I just miss all the DIY. The creativity and control. Anyways, recently I've been building an ARM64-based "mini-homelab" around an Archer AC1750 router running OpenWRT, a stack of three Odroid C2 SBCs, and an Nvidia Jetson TX1 (with a 50K LUT FPGA on the m.2 PCIe slot). I also have an ARM64 VM in the cloud. Once that's all set up, I've been considering how much I can "get back" under my control.
- Arn_Thor 8y ago
  I enjoyed google, and used docs and sheets for budgeting and planning. Then one day it was all gone. I'd had two google accounts, one created a decade ago on YouTube and later connected to my gmail-google account. I used the same login for both, and could switch between them without problem. I had used my Youtube-persona for most of my google docs work. That was all fine until suddenly I could only access docs from my other gmail-persona. They had without warning or reason changed either the account type or the app permissions for my account type. Not a word of warning, or even a message. There's no reverting it, no one at Google has been able to undo the change or recover my documents. NEVER AGAIN will I trust any important information to any cloud company. That was the day I started to take everything off the cloud and access it through a NAS at home (with off-site backup of course). I've never slept better
rcarmo 8y ago
I have a similar home setup, but don’t recommend using Pis for disk-intensive workloads due to SD Card volatility. My HomeKit setup, for instance, is Dockerized and runs on an ODROID board with EMMC storage, and the build server for my ARM containers boots from an SD card but has an USB hard drive.
- fallenhitokiri 8y ago
  At this point I would not recommend using PIs for anything long running at all. I have two v3 and one v2 and all of them crashed at some point, one of the v3 was idle during that time. The one PI I still run as build server operates of a write protected SD card and reboots regularly. I am not 100% sure why the PIs crashed yet (still have to do the investigation), but if errors like this already show up in a server rack with AC attached I'm a bit skeptical.
  - 8fingerlouie 8y ago
    The usual suspect when i comes to Pi SD Card corruption is a bad power supply. The Pi will (attempt to) draw more power than it can, resulting in a corrupted SD card. I've also run a large part of my internal network on RPis, but have since moved to Intel NUC machines, where i replaced a bunch of Pis with 2 NUCs. Storage like you has always been handled on a NAS. Synology in my case. I don't have need for much computing power, so the NUCs handle that just fine. See my other comment : https://news.ycombinator.com/item?id=17966507 https://news.ycombinator.com/item?id=17966507
teekert 8y ago
"I think the only thing I would like to have automated but have not yet is getting photos from our phones on the server, right now they still go through iCloud." I use Nextcloud for this, every picture I take is directly synced to my own basement. I can browse them and share them directly from the ui. Nextcloud also offers a nice webui for your email and allows syncing of you calendar and contacts from iPhone or Android. For me it allowed me to switch away from Google for my phone's back-end.
- fallenhitokiri 8y ago
  OP here - I actually looked into NextCloud but for most file sharing went for Resilio, so this would be the only use case for NextCloud which seems a bit thin. The Qnap NAS actually provides functionality like this, but I didn't have a chance to take a closer look at it. And I assume I still have to manually open the app once in a while for a sync, which is one step below where I'd actually want to get to.
  - teekert 8y ago
    I don't have to open the app, my pictures sync when I am on my home wifi and on the charger. But you can also set it to immediately upload after a picture was taken.
8fingerlouie 8y ago
I've been steadily moving everything back home ever since Snowden revealed just how much snooping is going on, and with the last few years worth of scandals, it seems i've made the right choice. Initially i went the Raspberry Pi route, and in a few months i had 4 or 5 of them running, which was where it started turning into a chore. While i enjoy tinkering, i don't particularly enjoy keeping a host of machine up to date. Fast forward a couple of years, and my home setup now looks like this : - UBNT EdgeRouter for that sweet affordable hardware L2TP/IPSec VPN. - Intel NUC6, Running everything internet facing in FreeBSD Jails. - Intel NUC7, Running everything internally available through Docker, along with Time Machine backups. - Synology DS716+, Holds all data from both NUC machines, as well as personal files (documents/photos/music/movies) Resilio Sync is running on the external NUC to provide an "always on" node. I have a couple of ODroid HC1 boxes that holds my backups, one at home and one at a remote location. The only thing i have yet to migrate is mail, which is still located at GSuite. I've moved several times, but every time i decide it's not worth the effort. Since "everybody" uses GMail every mail i send will eventually get indexed as well, regardless of where i store my own mail. I use GSuite for hosting 5 domains, and i've yet to find an alternative that doesn't require either lots of money or lots of administration from me.
- Angostura 8y ago
  Sorry to go off topic, but could you explain how you;re doing Time Machine backups to the NUC - sounds very interesting.
  - 8fingerlouie 8y ago
    Nothing magic. I've installed netatalk from the guidelines on the wiki http://netatalk.sourceforge.net/wiki/index.php/Install_Netatalk_3.1.11_on_Debian_9_Stretch http://netatalk.sourceforge.net/wiki/index.php/Install_Netat... and simply expose a shared drive through it.
    - tonyedgecombe 8y ago
      Netatalk is an AFP[1] server. [1] https://en.wikipedia.org/wiki/Apple_Filing_Protocol https://en.wikipedia.org/wiki/Apple_Filing_Protocol
sneak 8y ago
The amount of lost time spent sysadminning and configuring and securing these things is not a potential risk, it is a real cost. Getting NSLed or TOSed is a potential risk, not a guarantee. For most people who are doing nothing controversial or interesting and who are just using the cloud to receive service notifications and correspondence from friends (who are also doing nothing controversial or interesting), it isn’t a loss of control. This is a real trade-off, and neither is right. But don’t pretend that keeping a half dozen complex services up, backed up, and secured isn’t a huge time investment. You can’t get that back.
- hkajhfjl 8y ago
  No, not really. It isn't a huge time investment if you know all this well (let's say you sysadmin for work). Of course it is not feasible for most people to do this, just like it's not feasible for me personally to rebuild my car's engine, or grow vegetables in my garden. Both of which my neighbor does with great ease. The point is, I use my skills to make my life better, as everyone does. I enjoy it, and it has the side effect of keeping my data private. For things I'm not good at, I either spend time getting good, or I pay a professional for his time and skills. My mother uses Google and Apple services to back up her photos and data, and I wouldn't dream of forcing my approach on her. My spouse asked me to set all this up for them, since they share my feelings about data privacy.
- fallenhitokiri 8y ago
  OP here - to me, it is not. For most services I picked SQLite, so the whole backup is one file / directory. Software can easily be updated or test instances started since all is driven by docker. For the few services that need a DB there is a central one serving all hosts. Backups regularly (on a nightly basis where sufficient) go to the NAS. The VM docker is running on gets regular snapshots. Some initial setup and that's it. The whole setup is as movable and reproducible as I could justify making it over a weekend. Securing theres is an interesting point. I take the risk of a local attacker. Other than that all I got to keep secure is the VPN, which I would have to do anyway. I am not trying to make it sound like this is something you spend 10 minutes on and call it a day. I am doing this (running prod services) to various degrees for nearly two decades. If you are just getting into all of this, sure, it will take more time and dedication. I see your point, but I do not necessarily agree with it. I would argue that even with "nothing controversial or interesting" it still is a loss of control. You do not control your data. You do not control if the service will stay available to you and / or your friends.
ta76567656 8y ago
The main issue with this approach for me is that Google et. al. clearly have much better physical security than my apartment. In exchange for in-principle privacy improvements and possibly forth amendment protection you take on a huge risk of burglaries, fires, floods, power outages, etc, plus of course the workload of being your own sysadmin. Also, if you're paranoid, your data is more exposed. You can turn off all your devices with disk encryption when they're out of your control (usually) but if you turn off your NAS while your away from home it's useless. And if it's on, physical access, and therefore your data, is easy to obtain by the moderately motivated. Google is like a feudal lord: they might own you, but they'll protect you from everyone else weaker than them.
- confounded 8y ago
  That’s all true. I don’t actually think that anyone is out to get me in particular, I just find unaccountable concentrations of power and authority distasteful (and troubling). So, I try not to encourage them. I’m interested in self-hosting for autonomy more than security. That said, the data Google controls will likely outlive Google the legal entity/business-model of today. Including that huge security apparatus. The long-term (say 10 years from now) security of your data is moot if you can’t control it. Google have been quite good with allowing exports, and promising to delete things (probably more like perma-hash, but I’ll take it). However, they’re increasingly behaving in ways I don’t expect or like. It’s been great, but they no longer seem like good stewards to me.
- 8fingerlouie 8y ago
  As for physical security, you risk burglaries, fire, floods, etc, but that's why you have remote backups. I keep everything except the boot drives on encrypted drives, so that in case of burglary no data is readable. The boot drives hold no data or passwords, only enough to start up and allow SSH logins. It's a small chore to login and manually mount the drives, but IMO worth it. As for physical access, besides the 40kg German Shepherd Dog roaming my house, the same rules apply to access as from the outside: 2FA, and limited login attempts. I do expose more services on the LAN than i do on the internet, but everything requires authentication. For personal cloud stuff i use Resilio Sync. It's not dependent on a single machine being powered on, and i have a couple of machines at different physical locations (both _mine_, as in hardware and sysadm tasks) "hosting" the data.
nickjj 8y ago
The MVP of this approach would be buying a $60 external USB HD and backing everything up to that on a daily basis. Works great if you "only" have a few TB or less of data. Here's a bash script + the set up I'm running to do that: https://nickjanetakis.com/blog/automatic-offline-file-backups-with-bash-and-rsync https://nickjanetakis.com/blog/automatic-offline-file-backup...
confounded 8y ago
Out of interest, have any self-hosters here tried one big machine with lots of VMs, as opposed to a stack of NUCs/Pis? The recent Intel scares have got me interested in firmware and Libreboot again, and a HN commenter[0] pointed me at some extremely beefy (e.g. 16 cores!) AMD motherboard/CPU combos, that it’s possible to run fully-free firmware on. Without doing much actual research, Xen on a big box seems like it could be a good way to have a single physical machine where I ‘deploy’ different services in VMs. Anyone gone this route, or can think of any problems with this approach? It seems like it could be a little easier to manage, but I’m sure there are extra considerations for networking and security. [0]: https://news.ycombinator.com/threads?id=153791098c https://news.ycombinator.com/threads?id=153791098c
- stefanfisk 8y ago
  I’m contemplating something similar! If you want you can even run windows in kvm with pci pass through for native gpu performance.
- HappyRobot 8y ago
  You might be interested in the https://www.reddit.com/r/homelab https://www.reddit.com/r/homelab subreddit. The prevailing wisdom there is to buy used enterprise hardware, install a hyperadvisor, and set up separate VM's for everything. It's easy to scale, easy to tinker with, and really flexible.
- fallenhitokiri 8y ago
  OP here - all of my services run in a VM on the Dell server, including some other VMs used for work. Works pretty well, but you pay a slightly higher energy bill. As long as only a few users (in my case between 1 and 6) use the services you'd be surprised with how little resources you can get away. My initial test VM (VMWare) I used had 1 CPU and 1GB memory - it ran Postgres, miniflux, docker registry, gitea and wallabag without problems. CPU mostly idle, memory without a lot of load ~350MB. (Alpine as base for docker)
- mindslight 8y ago
  There's the security issue of general VM escape, and sidechannel attacks across VMs (eg Spectre). And of course you want redundancy against losing an entire machine (say the PSU fails and overvolts the 12v rail). But more centralized ad-hoc infrastructure should be easier to grok/manage/backup against this. My new router is pfsense running on KVM/Debian on an i5-4430. If I start running more Internet-facing services from home, they'll probably be on that box rather than my Libreboot KGPE. FWIW, Newegg has been clearing out those KGPE boards, occasionally selling them for $80 - https://pcpartpicker.com/product/yvJwrH/asus-motherboard-kgped16 https://pcpartpicker.com/product/yvJwrH/asus-motherboard-kgp... (the last one was on flash.newegg.com and so doesn't show up on PCPP). Non-high-density G34 heatsinks are a bit tough to find these days, but there's a thread where someone details the steps of mounting a 212 Evo. The Raijintek Aidos worked for me just by fashioning the appropriate length of steel bar, drilling four holes (two for the G34 mount, two for securing the existing Raijintek piece), and adding the appropriate screws/springs. The fan mounting on the Aidos does suck though.
NIL8 8y ago
I'd love to see a YT channel or a website that teaches how to do stuff like this for the non-tinkerer. I'm thinking along the lines of Primitive Techknology's channel where he simply shows the process step-by-step.
deleted 8y ago
[deleted]

This is a great read, but I'd like to hear more about the nitty gritty, what server hardware you use, it seems like you have 3 pis and then a bunch of really advanced enterprise-level stuff for other work... I'd love a guide on what you did to get here. The internet really needs something like that.

hkajhfjl 8y ago
I'm not the OP, but I have been doing this for some years now. Here's how I went about it. I got out of social media in early 2012. Deleted FB and Twitter. I do maintain a presence on LinkedIn, but I never post anything there, nor do I login. It's mostly there because employers expect a LI profile. It's just a bland copy of my resume. I moved my email from Gmail to my own domain from 2013-2015. It took almost two years because I had Gmail since April 2004, a few days after launch. My whole digital life was tied into Gmail, and moving stuff over takes time. I still have that Gmail account, but again, I never log into anything Google, and it simply forwards any email to my real account. I get maybe 1 email a month from Gmail. I host my email with Fastmail, but I've also selfhosted, and tried O365. It's easy to switch, and I keep an offline backup of everything. Now, my hardware and software setup. At home, I have a Synology NAS with ~12 TB of usable storage. This is my primary data store - I have software that automiatically backs up all photos from my Iphone, Time Machine, Veeam, Rysnc, and email backup. I own my own physical server (Dell R720XD) which is colocated in a datacenter in a different country. This server has 96 GB of RAM, 16 TB of storage, and 12 Xeon cores. It usually runs between 12-25 VMs, some of which are production (personal), hosting my websites, git, various projects, and so on. It's connected to the internet through a gigabit connection with a 10 TB monthly bandwidth cap (I never come close to 1 TB!). This costs me ~$85 a month. I don't do BGP, though I could if I paid some more and put in a proper router and got my own IP range.. as it stands, the 8 public IPs I get are enough for me. The colo server is connected via an always-on, site-to-site IPSEC VPN to my home LAN (through my EdgeRouter X). I have symmetric gigabit, so it's practically like having the machine in my house. My NAS backs up to the colo server, and the colo server to the NAS (via rsync). The NAS also backs up to a local USB drive, for quick restores. All my home data lives in four places 1. The local PC/laptop/phone. 2. The NAS 3. The USB Disk 4. The offsite colo server. Likewise, all my colo data lives in the same four places. I can VPN in to the colo server and get connected to my home lan from anywhere in the world. Any sensitive data is encrypted - and anything really sensitive is not stored on a computer. This setup works very well for me and my family, and I control all my data and am not dependent on a cloud provider. I also have a bunch of other servers running in my home, but they aren't part of my core infrastructure. The cost? ~$1200 for the R720. ~$1000 for the NAS. and ~$85 a month for colocation costs.
- dewey 8y ago
  Do you have measures in place to give access to someone trusted in case something happens to you? With a specialized setup like this hosting, for example all family pictures, I’d always be scared of having a low bus factor.
  - hkajhfjl 8y ago
    Yes and no. My spouse can easily access anything family related - pictures, documents, passwords (keepass) from their computer. The NAS and the Colo Server both present as shared drives on the network and they use it almost every day. The USB drive is plug and play and has copies of anything important. However, all my git repos, websites, tech projects etc will probably not survive my passing. I don't really care about that. They will still be backed up and accessible on the NAS, so if someone in my family (I have a few technically adept people in the family) wanted to check it out, my spouse would be able to give them access. Honestly, I don't think anyone would care, but who knows. All passwords and access details are in KeePass and my spouse uses that everyday so knows how to use it, My colo server is paid a year in advance every year, so it would chug along for a few months before the provider shut it down and shipped it back to my house (again, my spouse knows how to get it shipped back). All in all, I think my family would be able to access anything truly valuable without too much trouble, and who cares about the other stuff anyway.
- voltagex_ 8y ago
  Few more questions: * How do you backup Fastmail? * How'd you find your colo provider? * If it's in a different country, how does the latency affect you?
  - walterbell 8y ago
    Ditto on the colo provider question. Which virtualization do you use on server and how do you handle remote admin and security updates for guest VMs and host?
    - hkajhfjl 8y ago
fallenhitokiri 8y ago
OP here - since there seems to be some interest on the details I will try to answer them in as much detail as possible in a longer blog post. I already wrote about the network setup[1] The server is a Dell Precision T5610 (20 physical cores, 64GB memory, 4 Samsung Pro SSDs and quad gigabit NIC additionally to the internal one) and a QNap TS-531X with Seagate IronWolf drives. I would recommend against using PIs at this point, see other comment[2]. Right now all services are running in docker in a dedicated VM on VMWare Workstation. The server also hosts a full work environment I can RDP and SSH into, depending on what I do, when I am traveling only with my iPad Pro (RDP because my last job involved a lot of Java for which I prefer IntelliJ). At some point I should likely migrate to vSphere, but I had the workstation license already and it was quick to setup. [1] https://screamingatmyscreen.com/2018/7/building-our-home-and-office-network/ https://screamingatmyscreen.com/2018/7/building-our-home-and... [2] https://news.ycombinator.com/item?id=17966264 https://news.ycombinator.com/item?id=17966264
- 8fingerlouie 8y ago
  I'm probably beating a dead horse here with Docker Policies available and all, but i personally don't trust Docker in production. Each Docker image runs it's own stack, and each stack has the potential to contain vulnerabilities. Even with services such as Watchtower ( https://hub.docker.com/r/v2tec/watchtower/ https://hub.docker.com/r/v2tec/watchtower/ ), you're still not safe. Some images are abandoned for years, others are only updated once their "final" product is updated, meaning you could be hosting a handful of vulnerable services without even knowing it. I host all my internet facing stuff on FreeBSD in jails, though Linux with LXC would do just as good. I have one stack to update, and once that's updated, everything else is updated as well.
  - fallenhitokiri 8y ago
    Nothing I host is exposed to the Internet but only available locally. When on the road I VPN back home. But generally I agree, for Internet facing services I would likely make a few different decisions than in this case.

fouc 8y ago

Recently I was wondering what if all our phones and computers all connected to our own personal "platform" that we have full control over. Route everything on the phone through a personal platform. Gain control of location, IP, maps caching, etc. Centralize at a personal level.

CalRobert 8y ago
To an extent, this is kind of the idea behind sandstorm.io. Run the services you want on a box under your control (or a vps, but still yours). Given that there are finally good non-google options for a lot of these (OSM + Thunderforest + Graphhopper gives a really nice map experience for instance) I'd love to see it go further. As always, it's hard to find time to work on this though.
nwienert 8y ago
I’m working on almost this exact thing. Still a ways off but get in touch if interested.
close04 8y ago
A platform is useful for the other services it provides: Google funnels your data but give you better search, MS swipes some info about your PC to give you better drivers, Facebook takes all your likes and dislikes to give you stories to your taste. [talking about stated interest here] What would your personal platform do? How would it take that data and turn it into something useful while also removing the privacy concerns? Best you can do is have something that blocks or spoofs that data before sending it to the "collector". So you gain the privacy and you lose on the service improvement. If this is legitimate concern you can try to drop the most intrusive platforms from your digital life. These would most likely be Google and Facebook. Alternatively there are options for you to use with more limited effect: ad and tracker blockers, secure or anonymous browsers, VPN, stricter privacy settings on your devices, etc. The end result is usually the same: the less data you provide them, the less useful their service is. Which makes trading your info even less enticing.
- ACS_Solver 8y ago

bthornbury 8y ago

Very cool, would love to see more content on this topic. Setup guides of individual services, issues you've run into, more on hardware choices, and definitely more on your smart home setup.

fallenhitokiri 8y ago
OP here, thanks! I posted a few more details here[1] and will follow up with an in depth blog post covering as much as possible rest. For the smart home setup I am mostly using Elgato Eve SmartPlugs, their door and window sensors and Homebridge for everything that does not integrate with HomeKit natively. Any light source I cannot directly plug into a power outlet (ceiling mounted e.x.) is at least remote controllable via RF, not IR, and I am currently working on figuring out how to send the control signals through an PI or Arduino with an RF transceiver. [1] https://news.ycombinator.com/item?id=17966311 https://news.ycombinator.com/item?id=17966311 [2] https://github.com/nfarina/homebridge https://github.com/nfarina/homebridge

onefish 8y ago

Recently I've been playing around with something like this, or really; getting back to something like this. Years ago I was an idealistic graduate student who didn't have any "free" (you-are-the-product) digital service or social media accounts - I used Lavabit (before it got shut down) for email and a couple of cheap VPS providers to run a little blog and IRC bouncer. I had a dumb "burner" phone, a RockBox-based MP3 player, ran Tomato on my router, and ran Debian on most everything else. I think the main factors that killed it for me, that made me drink the Kool-Aid so to speak, were a combination of getting an Android phone and the shutdown of Lavabit (with all the hassle that incurred - notifying dozens of colleagues, mailing lists, etc of the change). Concurrently with this I had just gotten my first industry gig at a pretty large networking equipment company, on a team with mostly older, mellowed, senior engineers for whom programming was "just a job". I didn't want to seem like a "paranoid weirdo" who had some black-hat alter ego. I sold out. Since, I've pretty much moved wholesale to Google. I still don't use any other digital services - Google has basically become my one-stop shop, for better or worse. I use Android, Chrome, Gmail, Drive, Music, Books, Search, Maps, Keep, Photos, basically the whole damn suite. It's a beautifully unified and seamless experience. I feel in-general, Google gets just about everything right (I don't use Docs - I still write docs in LaTex, haha). It's quite a 180 from what I had before. But ultimately, I think this has caused me a lot of cognitive dissonance. I've spent a lot of time thinking about how to "get back" lately, but this is tempered by how much control I've already given up and, well, what is frankly a pretty damn high quality and convenient experience and there are some things (like Maps, and Photos) which I really don't want to give up. I also don't hold any delusions that anything I do is going to be "more secure" from any threat model, really. I guess I just miss all the DIY. The creativity and control. Anyways, recently I've been building an ARM64-based "mini-homelab" around an Archer AC1750 router running OpenWRT, a stack of three Odroid C2 SBCs, and an Nvidia Jetson TX1 (with a 50K LUT FPGA on the m.2 PCIe slot). I also have an ARM64 VM in the cloud. Once that's all set up, I've been considering how much I can "get back" under my control.

Arn_Thor 8y ago
I enjoyed google, and used docs and sheets for budgeting and planning. Then one day it was all gone. I'd had two google accounts, one created a decade ago on YouTube and later connected to my gmail-google account. I used the same login for both, and could switch between them without problem. I had used my Youtube-persona for most of my google docs work. That was all fine until suddenly I could only access docs from my other gmail-persona. They had without warning or reason changed either the account type or the app permissions for my account type. Not a word of warning, or even a message. There's no reverting it, no one at Google has been able to undo the change or recover my documents. NEVER AGAIN will I trust any important information to any cloud company. That was the day I started to take everything off the cloud and access it through a NAS at home (with off-site backup of course). I've never slept better

rcarmo 8y ago

I have a similar home setup, but don’t recommend using Pis for disk-intensive workloads due to SD Card volatility. My HomeKit setup, for instance, is Dockerized and runs on an ODROID board with EMMC storage, and the build server for my ARM containers boots from an SD card but has an USB hard drive.

fallenhitokiri 8y ago
At this point I would not recommend using PIs for anything long running at all. I have two v3 and one v2 and all of them crashed at some point, one of the v3 was idle during that time. The one PI I still run as build server operates of a write protected SD card and reboots regularly. I am not 100% sure why the PIs crashed yet (still have to do the investigation), but if errors like this already show up in a server rack with AC attached I'm a bit skeptical.
- 8fingerlouie 8y ago
  The usual suspect when i comes to Pi SD Card corruption is a bad power supply. The Pi will (attempt to) draw more power than it can, resulting in a corrupted SD card. I've also run a large part of my internal network on RPis, but have since moved to Intel NUC machines, where i replaced a bunch of Pis with 2 NUCs. Storage like you has always been handled on a NAS. Synology in my case. I don't have need for much computing power, so the NUCs handle that just fine. See my other comment : https://news.ycombinator.com/item?id=17966507 https://news.ycombinator.com/item?id=17966507

teekert 8y ago

"I think the only thing I would like to have automated but have not yet is getting photos from our phones on the server, right now they still go through iCloud." I use Nextcloud for this, every picture I take is directly synced to my own basement. I can browse them and share them directly from the ui. Nextcloud also offers a nice webui for your email and allows syncing of you calendar and contacts from iPhone or Android. For me it allowed me to switch away from Google for my phone's back-end.

fallenhitokiri 8y ago
OP here - I actually looked into NextCloud but for most file sharing went for Resilio, so this would be the only use case for NextCloud which seems a bit thin. The Qnap NAS actually provides functionality like this, but I didn't have a chance to take a closer look at it. And I assume I still have to manually open the app once in a while for a sync, which is one step below where I'd actually want to get to.
- teekert 8y ago
  I don't have to open the app, my pictures sync when I am on my home wifi and on the charger. But you can also set it to immediately upload after a picture was taken.

8fingerlouie 8y ago

I've been steadily moving everything back home ever since Snowden revealed just how much snooping is going on, and with the last few years worth of scandals, it seems i've made the right choice. Initially i went the Raspberry Pi route, and in a few months i had 4 or 5 of them running, which was where it started turning into a chore. While i enjoy tinkering, i don't particularly enjoy keeping a host of machine up to date. Fast forward a couple of years, and my home setup now looks like this : - UBNT EdgeRouter for that sweet affordable hardware L2TP/IPSec VPN. - Intel NUC6, Running everything internet facing in FreeBSD Jails. - Intel NUC7, Running everything internally available through Docker, along with Time Machine backups. - Synology DS716+, Holds all data from both NUC machines, as well as personal files (documents/photos/music/movies) Resilio Sync is running on the external NUC to provide an "always on" node. I have a couple of ODroid HC1 boxes that holds my backups, one at home and one at a remote location. The only thing i have yet to migrate is mail, which is still located at GSuite. I've moved several times, but every time i decide it's not worth the effort. Since "everybody" uses GMail every mail i send will eventually get indexed as well, regardless of where i store my own mail. I use GSuite for hosting 5 domains, and i've yet to find an alternative that doesn't require either lots of money or lots of administration from me.

Angostura 8y ago
Sorry to go off topic, but could you explain how you;re doing Time Machine backups to the NUC - sounds very interesting.
- 8fingerlouie 8y ago
  Nothing magic. I've installed netatalk from the guidelines on the wiki http://netatalk.sourceforge.net/wiki/index.php/Install_Netatalk_3.1.11_on_Debian_9_Stretch http://netatalk.sourceforge.net/wiki/index.php/Install_Netat... and simply expose a shared drive through it.
  - tonyedgecombe 8y ago
    Netatalk is an AFP[1] server. [1] https://en.wikipedia.org/wiki/Apple_Filing_Protocol https://en.wikipedia.org/wiki/Apple_Filing_Protocol

sneak 8y ago

The amount of lost time spent sysadminning and configuring and securing these things is not a potential risk, it is a real cost. Getting NSLed or TOSed is a potential risk, not a guarantee. For most people who are doing nothing controversial or interesting and who are just using the cloud to receive service notifications and correspondence from friends (who are also doing nothing controversial or interesting), it isn’t a loss of control. This is a real trade-off, and neither is right. But don’t pretend that keeping a half dozen complex services up, backed up, and secured isn’t a huge time investment. You can’t get that back.

hkajhfjl 8y ago
No, not really. It isn't a huge time investment if you know all this well (let's say you sysadmin for work). Of course it is not feasible for most people to do this, just like it's not feasible for me personally to rebuild my car's engine, or grow vegetables in my garden. Both of which my neighbor does with great ease. The point is, I use my skills to make my life better, as everyone does. I enjoy it, and it has the side effect of keeping my data private. For things I'm not good at, I either spend time getting good, or I pay a professional for his time and skills. My mother uses Google and Apple services to back up her photos and data, and I wouldn't dream of forcing my approach on her. My spouse asked me to set all this up for them, since they share my feelings about data privacy.
fallenhitokiri 8y ago
OP here - to me, it is not. For most services I picked SQLite, so the whole backup is one file / directory. Software can easily be updated or test instances started since all is driven by docker. For the few services that need a DB there is a central one serving all hosts. Backups regularly (on a nightly basis where sufficient) go to the NAS. The VM docker is running on gets regular snapshots. Some initial setup and that's it. The whole setup is as movable and reproducible as I could justify making it over a weekend. Securing theres is an interesting point. I take the risk of a local attacker. Other than that all I got to keep secure is the VPN, which I would have to do anyway. I am not trying to make it sound like this is something you spend 10 minutes on and call it a day. I am doing this (running prod services) to various degrees for nearly two decades. If you are just getting into all of this, sure, it will take more time and dedication. I see your point, but I do not necessarily agree with it. I would argue that even with "nothing controversial or interesting" it still is a loss of control. You do not control your data. You do not control if the service will stay available to you and / or your friends.

ta76567656 8y ago

The main issue with this approach for me is that Google et. al. clearly have much better physical security than my apartment. In exchange for in-principle privacy improvements and possibly forth amendment protection you take on a huge risk of burglaries, fires, floods, power outages, etc, plus of course the workload of being your own sysadmin. Also, if you're paranoid, your data is more exposed. You can turn off all your devices with disk encryption when they're out of your control (usually) but if you turn off your NAS while your away from home it's useless. And if it's on, physical access, and therefore your data, is easy to obtain by the moderately motivated. Google is like a feudal lord: they might own you, but they'll protect you from everyone else weaker than them.

confounded 8y ago
That’s all true. I don’t actually think that anyone is out to get me in particular, I just find unaccountable concentrations of power and authority distasteful (and troubling). So, I try not to encourage them. I’m interested in self-hosting for autonomy more than security. That said, the data Google controls will likely outlive Google the legal entity/business-model of today. Including that huge security apparatus. The long-term (say 10 years from now) security of your data is moot if you can’t control it. Google have been quite good with allowing exports, and promising to delete things (probably more like perma-hash, but I’ll take it). However, they’re increasingly behaving in ways I don’t expect or like. It’s been great, but they no longer seem like good stewards to me.
8fingerlouie 8y ago
As for physical security, you risk burglaries, fire, floods, etc, but that's why you have remote backups. I keep everything except the boot drives on encrypted drives, so that in case of burglary no data is readable. The boot drives hold no data or passwords, only enough to start up and allow SSH logins. It's a small chore to login and manually mount the drives, but IMO worth it. As for physical access, besides the 40kg German Shepherd Dog roaming my house, the same rules apply to access as from the outside: 2FA, and limited login attempts. I do expose more services on the LAN than i do on the internet, but everything requires authentication. For personal cloud stuff i use Resilio Sync. It's not dependent on a single machine being powered on, and i have a couple of machines at different physical locations (both _mine_, as in hardware and sysadm tasks) "hosting" the data.

nickjj 8y ago

The MVP of this approach would be buying a $60 external USB HD and backing everything up to that on a daily basis. Works great if you "only" have a few TB or less of data. Here's a bash script + the set up I'm running to do that: https://nickjanetakis.com/blog/automatic-offline-file-backups-with-bash-and-rsync https://nickjanetakis.com/blog/automatic-offline-file-backup...

confounded 8y ago

Out of interest, have any self-hosters here tried one big machine with lots of VMs, as opposed to a stack of NUCs/Pis? The recent Intel scares have got me interested in firmware and Libreboot again, and a HN commenter[0] pointed me at some extremely beefy (e.g. 16 cores!) AMD motherboard/CPU combos, that it’s possible to run fully-free firmware on. Without doing much actual research, Xen on a big box seems like it could be a good way to have a single physical machine where I ‘deploy’ different services in VMs. Anyone gone this route, or can think of any problems with this approach? It seems like it could be a little easier to manage, but I’m sure there are extra considerations for networking and security. [0]: https://news.ycombinator.com/threads?id=153791098c https://news.ycombinator.com/threads?id=153791098c

stefanfisk 8y ago
I’m contemplating something similar! If you want you can even run windows in kvm with pci pass through for native gpu performance.
HappyRobot 8y ago
You might be interested in the https://www.reddit.com/r/homelab https://www.reddit.com/r/homelab subreddit. The prevailing wisdom there is to buy used enterprise hardware, install a hyperadvisor, and set up separate VM's for everything. It's easy to scale, easy to tinker with, and really flexible.
fallenhitokiri 8y ago
OP here - all of my services run in a VM on the Dell server, including some other VMs used for work. Works pretty well, but you pay a slightly higher energy bill. As long as only a few users (in my case between 1 and 6) use the services you'd be surprised with how little resources you can get away. My initial test VM (VMWare) I used had 1 CPU and 1GB memory - it ran Postgres, miniflux, docker registry, gitea and wallabag without problems. CPU mostly idle, memory without a lot of load ~350MB. (Alpine as base for docker)
mindslight 8y ago
There's the security issue of general VM escape, and sidechannel attacks across VMs (eg Spectre). And of course you want redundancy against losing an entire machine (say the PSU fails and overvolts the 12v rail). But more centralized ad-hoc infrastructure should be easier to grok/manage/backup against this. My new router is pfsense running on KVM/Debian on an i5-4430. If I start running more Internet-facing services from home, they'll probably be on that box rather than my Libreboot KGPE. FWIW, Newegg has been clearing out those KGPE boards, occasionally selling them for $80 - https://pcpartpicker.com/product/yvJwrH/asus-motherboard-kgped16 https://pcpartpicker.com/product/yvJwrH/asus-motherboard-kgp... (the last one was on flash.newegg.com and so doesn't show up on PCPP). Non-high-density G34 heatsinks are a bit tough to find these days, but there's a thread where someone details the steps of mounting a 212 Evo. The Raijintek Aidos worked for me just by fashioning the appropriate length of steel bar, drilling four holes (two for the G34 mount, two for securing the existing Raijintek piece), and adding the appropriate screws/springs. The fan mounting on the Aidos does suck though.

NIL8 8y ago

I'd love to see a YT channel or a website that teaches how to do stuff like this for the non-tinkerer. I'm thinking along the lines of Primitive Techknology's channel where he simply shows the process step-by-step.

deleted 8y ago

[deleted]