11 ms·
How to Secure a Linux Server
- pettycashstash2 8y agoI didn’t know you could add google Authenticator to your server. Thank you for this write up.
- tyingq 8y agoI'm surprised Authy doesn't have a pam module: https://github.com/authy/authy-ssh/issues/53 https://github.com/authy/authy-ssh/issues/53
- imthenachoman 8y agoAuthy's mobile app should work with Google's TOTP PAM module.
- Timucin 8y agoYes, you can add it to almost anything and the best part is; you don’t have to be online to generate and/or verify the code.
- Fnoord 8y agoYou don't have to use a closed source TOTP client such as Google's. There's RedHat's FreeOTP and there's another open source one called antOTP. You can find them on F-Droid as well. There have been TOTP PAM modules for ages (these work on a Linux client and Linux server via e.g. SSH). You can even add YubiKey to PAM. Same for BSD Auth and macOS.
- noinsight 8y ago> RedHat's FreeOTP It's abandonware these days and hasn't been updated for years. The iOS app doesn't work anymore and can't use the camera for reading codes.
- dfed-mpls 8y agoI use it on ios, and so does most of Red Hat employees (they state it is a preferred version of tfa over the google app.) It's not broken for the 15 or so accounts I use it for.
- noinsight 8y agoTry the "scan code" button which should activate the camera and the app will crash. I just tested it with my iPhone and verified. The iOS App Store application was last updated 4 years ago. There's also comments there saying it doesn't work anymore. Reading / using existing codes might work properly, or manually inputting them.
- dfed-mpls 8y agoNope. I used that feature just today to add another account. Works on my iPhone X.
- dfed-mpls 8y agoAlso, the last commit to the app was a year ago, if you head to their github page. https://github.com/freeotp/freeotp-ios/commits/master https://github.com/freeotp/freeotp-ios/commits/master
- dfed-mpls 8y agoInterestingly my wife's XS is having the problem you describe but not my X. I've sent a note to the maintainers, but there's an open git issue for it. Not sure when their commits will roll to the store. I'm told by multiple colleagues at redhat that they've raised this with internal support, so it should be sorted eventually. Lots of iphones in redhat and this is the app they officially support for TFA internally.
- Avamander 8y agoTOTP/HOTP hasn't changed for years. The app works perfectly for me and needs no updates.
- imthenachoman 8y agoWhen I was researching, literally every article I read mentioned Google'S TOTP PAM module. I can't even find any other ones that have active development. Do you know of any? Would love to use another one if it works with my iOS authenticator app.
- bpye 8y agoIf you have an NFC YubiKey and an Android phone with NFC then the Yubico Authenticator is pretty awesome. You can store your OTP tokens on your YubiKey. Also open source. https://github.com/Yubico/yubioath-android https://github.com/Yubico/yubioath-android
- imthenachoman 8y agoYou're welcome. Glad it helped someone!
- khamba 8y agoI was pleased to notice that the author used DuckDuckGo search link in their article instead of Google.
- imthenachoman 8y agoAlways. Google gives me 5 thousand pages of ads before the results I want. I hate to admit that sometimes I still use Google for searching. Like I love their shopping search.
- deleted 8y ago[deleted]
- jSherz 8y agoThe CIS benchmarks are a great place to start for hardening a system (https://www.cisecurity.org/cis-benchmarks/ https://www.cisecurity.org/cis-benchmarks/) and there's also OpenSCAP gives you a nice way to scan systems for compliance against a set of hardening rules (https://www.open-scap.org/ https://www.open-scap.org/).
- chomp 8y agoYes! This guide is good for securing maybe a personal server, but any business systems should use a server hardening standard that has industry mindshare (CIS, STIG, etc.) I can speak from personal experience that QSAs give you a very skeptical look when you say "our security standard is homebrewed."
- bitexploder 8y agoMeh, QSAs check boxes. In my experience they are not very technically capable. Some are, of course. Most are not.
- Spivak 8y agoSure but the point still stands. Why should anyone trust that you, or your company, is any good at security compared to say CIS? If you have homegrown security and can show your QSA your detailed policy document and that it's a superset of CIS, STIG, NIST, etc. with documented exceptions then it'll be no problem. I avoid homegrown whenever possible because it's a rabbit hole that never ends. If you instead say CIS level 2 then you can clearly define when you've done enough.
- bitexploder 8y agoBecause, checked boxes don’t mean much. CIS level 2 = a lot of checked boxes. Using cyber security frameworks is great, but some of the most compliant and “advanced” organizations have the worst legacy cruft you can imagine. We work with orgs all the time and organizations that use these frameworks with expert guidance can easily secure their most critical assets while only implementing the right parts of a framework. And the best frameworks have risk based targeting for maturity levels (NIST Cybersecurity) of various activities. These frameworks can end up being a bit of security theatre if you are just implementing it for the sake of “having security”. Guess I am just jaded after breaking software and networks for over a decade. Some of the most secure organizations have very adaptive security practices that focus on application security. Some of the worst are ISO, CIS, STIG policy template hardened blah blah blah. Just don’t put these frameworks and policies on a pedastal. The real security work happens in the margins.
- supakeen 8y agoNo mention of shipping off logs to another place? It's probably good to assume someone will gain access and make after-the-fact forensics a primary concern as well. Something a lot of hardening guides seem to skip!
- Someone1234 8y agoA lot of hardening guides skip the long tail for security. Which is to say: So you've shipped logs off, so then what? How are you going to monitor those regularly, what are you looking for, how are you going to make sure important information stands out? Many people set up remote logging and then never check the logs until after there is an issue. An unread log isn't useful. Logs that are too spammy aren't going to be read.
- imthenachoman 8y agoGood idea. I need to figure out how to work that in into the guide. I added to the to-do list. Thanks!
- koolba 8y agoHere’s a bonus one: If you install Docker don’t add your non-admin to the docker group as it’s effectively passwordless sudo.
- bpye 8y agoDoes Docker allow you to enforce user namespaces for all comtainers? I think that avoids the issue.
- snuxoll 8y agoNothing about SELinux or AppArmor? While these are enabled by default on some of the big distros I think a basic guide on managing and troubleshooting would be beneficial.
- tyingq 8y agoGetting apparmor to stop flooding syslog with mysterious complaints is probably a full guide by itself.
- peterwwillis 8y agoYeah, the best impact would come from using SELinux, AppArmor, and GRSec. Everything else is just tweaking defaults, which for up-to-date software probably isn't going to impact your security much at all.
- imthenachoman 8y agoI don't disagree but you gotta start with the basics. SELinux can be rather advanced/complex and probably warrants its own guide. I still have to learn it better before I try to write a guide on it.
- imthenachoman 8y agoIt is on my to-do list. :)
- rasengan 8y agoIt would also be proper to disable non root access to /proc among other things. You can do that by simply mounting with hidepid=2 or adding it to fstab.
- shereadsthenews 8y agoWhy? Disabling access to /proc will disable a huge number of useful features, such as the ability of a process to monitor and manage its memory usage, to debug itself, and so forth.
- mbaeten 8y agoFor example, /proc/self/pagemap can be used for rowhammer attacks. Source: Another flip in the wall of rowhammer defenses (IEEE S&P 2018)
- imthenachoman 8y agoInteresting. I have never read that anywhere. I will research. TY!
- z3t4 8y agoDont forget about namespaces which lets you isolate apps. and Apparmor to restrict file and network access. And also chroot and setuid to drop privileges and change the root path. And unix sockets / named pipes. Instead of letting your daemons access the whole network have them listen on a unix socket. And scan your own network, both from the outside and inside.
- imthenachoman 8y agoI am not familiar with namespaces in this context? Can you please point me in the right direction? I have added AppArmor to my to-do list. I know what chroot and setuid are, not sure what to mention about them in the guide? And I will look into unix sockets. Can you recommend any good network scanning tools? I've been on the hunt for a good one.
- z3t4 8y agoI think the most useful for sysadmins are the network namespace that lets you put an app into it's own network, so it can not see the rest of the network. Container technology makes use of namespaces to create lightweight VM/containers. See man namespaces, lxc nmap is a popular tool for network scanning. I've also found tcpdump to be useful for looking at network traffic. setuid and chroot are useful for programmers, so once the app is up and running, it can chroot into a data-dir and drop root privileges using setuid to a unprivileged user. As a sysadmin you can also start the app from within a chroot and run it as a unprivileged user which is preferably. Most (free)BSD tutorials go though setting up a chroot jail, it's not as common in Linux. Containers, VM's, and chroot will not stop a very determined attacker, but the more restrictions the harder it will get. Security is applied in layers: First you want to prevent people from the outside. Access is most often gained through exploiting some service/daemon/app running on the server. So the app should have as little privileges as possible. System access is often then gained by exploiting another app (so you want all apps to be locked down, not just the network facing ones), like getting Apache to run curl, that sends a internal request to another app that has a known vulnerability and happens to run as root. (nobody thought that app not accessible from the outside needed to get patched). Once the attacker is inside the server, you also want to prevent access to other computers in the network.
- wodny 8y agoThis guide contains (at least) inaccurate statements. It's oversimplifying and omitting important things while putting emphasis on some exotic details where defaults would be sane enough. "One key, the public key, can only encrypt data, not decrypt it" - this is cryptographically inaccurate. One should use it that way, though. "Identity is verified by encrypting and decrypting data that both the client and server know". That's not how signing works. The crucial part of the process is to first establish an encrypted channel, then choose something random (not used previously) and finally verify the signature is correct. What exactly would be the data that "both the client and server know"? "If you're sure there is nobody listening between the client you're on and your server, you can use ssh-copy-id to transfer and append the public key". Almost never happens unless you're in the server room, connected point-to-point. But then the remark about someone listening doesn't make sense. And not a single word about verifying server's fingerprint in the whole guide. "Keep in mind doing this means you can't use the key for automation because you'll have no way to send the passphrase in your scripts". Unless you use an ssh-agent. "umask is a Bash built-in which means a user can change their own umask setting". Wrong implication here. umask is a syscall. It doesn't matter what you use to call it. And what if you're using a different shell? Suggesting painful default umask instead of just doing `chmod go-rwx $HOME` and adjusting /etc/adduser.conf is debatable. Making this guide "distribution agnostic" is IMHO futile and there already are some distribution-specific guides like https://debian-handbook.info/ https://debian-handbook.info/.
- okl 8y agoArch: https://wiki.archlinux.org/index.php/Security https://wiki.archlinux.org/index.php/Security CentOS: https://wiki.centos.org/HowTos/OS_Protection https://wiki.centos.org/HowTos/OS_Protection (limited) Debian: https://www.debian.org/doc/manuals/securing-debian-howto/index.en.html https://www.debian.org/doc/manuals/securing-debian-howto/ind... (old) Fedora: https://docs.fedoraproject.org/en-US/Fedora/19/html/Security_Guide/index.html https://docs.fedoraproject.org/en-US/Fedora/19/html/Security... (old) Mageia: https://wiki.mageia.org/en/Msec https://wiki.mageia.org/en/Msec (limited) Oracle Linux: https://docs.oracle.com/cd/E52668_01/E54670/html/index.html https://docs.oracle.com/cd/E52668_01/E54670/html/index.html Red Hat: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/ https://access.redhat.com/documentation/en-us/red_hat_enterp... Slackware: https://docs.slackware.com/howtos:security:start https://docs.slackware.com/howtos:security:start (limited) SuSE: https://www.suse.com/documentation/sles-15/singlehtml/book_hardening/book_hardening.html https://www.suse.com/documentation/sles-15/singlehtml/book_h... Ubuntu: https://help.ubuntu.com/lts/serverguide/security.html.en https://help.ubuntu.com/lts/serverguide/security.html.en (limited)
- ThePhysicist 8y agoI would not recommend using “ufw” for configuring the firewall on a production server (as it’s not easily composable and lacks configurability for more complex rules). I can recommend “ferm” instead as it allows you to compose multiple config files (important for automation e.g. when using Ansible) and allows describing more complex rules that e.g. involve policies.
- imthenachoman 8y agoHow would you define a production server? Do you mean in the context of a home server or a server used by a large company? The guide is intended for a server for home use. I hope anyone securing a large corporate server is not using information on GitHub. If they are then the company has far bigger problems than security. I think for home use, ufw is probably good enough. I've been using it for 3+ years and it's worked out okay for me okay. I have not heard of ferm but I will check it out. Thanks!
- ThePhysicist 8y agoSorry, very late reply: We deploy all our infrastructure with Ansible, hence we want to have a way to configure the firewall for each role individually without overwriting previous configuration. For example, we have firewall rules for IP-Sec connections, SSH connections from the bastion host and then specific configurations for applications like databases or message queues. With ferm we can just create individual configs for each of these and put them in a directory where they are loaded sequentially and automatically. This allows us to iteratively define firewall rules and deploy different Ansible roles. For a home server ufw is probably good enough, as I said I wouldn't recommend it for "serious" use in a highly automated environment.
- okl 8y agoIs using gmail to send mails from your server a good idea?
- tlb 8y agoIt's better than running your own smtp servers. But the article proposes using your own gmail account, to which the server has an app password. So a server compromise leads to your own email being compromised. It should be a separate gmail account.
- imthenachoman 8y agoSo I use a separate gmail account for my server but forgot to mention in guide. I will add it. Although I wonder if its necessary if you use an app password. What is the worst a bad-actor can do with the app password?
- imthenachoman 8y agoThere is a balance to be had here. In a high volume, sensitive environment you'd run your own mail server. But that is a challenge in-and-of itself. I don't recommend it for a home setup because there are numerous things to worry about like MX forwarding and what happens if your server goes down -- where does the e-mail go? Granted, using Gmail means if your server is compromised the bad-actor would have your gmail password which is why I recommend enabling 2FA/MFA on Gmail and then using an app password. Check http://reddit.com/r/selfhosted http://reddit.com/r/selfhosted if you want to self-host mail.
- dev_dull 8y ago> 2FA for ssh Correct me if I'm wrong, but isn't this what password-protected private key encryption is?
- wiredfool 8y agoNo. Keys are easily (and persistently) added to ssh-agent, at which point it is easy to forward and will generally silently authorize without any further user interaction. That reduces it to a single factor. Compare that to a totp challenge or a yubi key plus a password. Having one of them won’t get you the other.
- mgbmtl 8y agoThe author probably meant "two-step" (out of band verification) rather than two-factor.
- imthenachoman 8y agoA pass phrase on a certificate is two step. Password + TOTP is two factor.
- LIV2 8y agoThe problem with that is that your users can always generate a key without a passphrase at all and you have no way of knowing
- ohithereyou 8y agoYou are incorrect. The only factor used there is the key. The password simply decrypts the key to perform the requested authentication. 2FA would be private key (password-protected) and a separate (most likely one time use) password. Something you have (key that you decrypted) and something you know (the one-time password).
- blamarvt 8y agoWhat I feel like all the responses missed is that while you're right in that a password-protected private key provides a second factor... what is typically meant by 2FA is two factors checked by the server. The server has no way to know you're using a password-protected key so it shouldn't really be regarded as a factor.
- ozim 8y agoSecuring ssh, 2FA for ssh, using key authentication... Yes but first and foremost you don't expose ssh to all internet, it should be allowed only from known IP addresses and you should VPN to have connect to that known addresses. Section about firewall config just tells to open ssh...
- symisc_devel 8y agoDo you have any resources on how to allow ssh access from certain IP ranges? I can't find anything useful on this topic.
- zegl 8y agoiptables, ufw, Security Groups (if you’re on AWS). There is a lot of ways to it.
- laumars 8y agoThere isn’t lots of ways to do it; there’s just the one: firewall rules. There just happens to be lots of different types of filewalls. If you want to get clever then you can enable port knocking but my personal preference is just good old fashioned whitelist of IPs with fail2ban running ready to auto-blacklist any of those IPs that have too many failed login attempts in a given period of time.
- imthenachoman 8y agoLAN IPs or WAN IPs? If you're using public/private keys you can use the "from" option for the keys. But it's not fool proof.
- zlynx 8y agoIt might help protect against an OpenSSH zero-day but those are pretty rare. I've been running my home server with an completely open SSH port since 2012 and haven't been hacked yet. And I know since I check occasionally with Tripwire and/or RPM verification from a clean boot. The only downside to an open SSH port is the thousands and thousands of log spams from connection attempts. The upside is that I can log in from anywhere using my phone directly or as a hotspot for the laptop. Oh yeah, and VPN just pushes the problem back one level. For me SSH is the VPN. Requiring VPN access first to get to SSH is just pushing any security problems back into the VPN server. Because now IT is the one with the open port to the internet.
- bufferoverflow 8y agoRelated: My First 10 Minutes On a Server - Primer for Securing Ubuntu https://www.codelitt.com/blog/my-first-10-minutes-on-a-server-primer-for-securing-ubuntu/ https://www.codelitt.com/blog/my-first-10-minutes-on-a-serve...
- joobus 8y agoDoes anyone have experience with deborphan? Not previously aware of it, I installed it, and it reported 2 programs which I use very frequently (conky and cryptsetup) as being orphans. I dare not run the uninstall script in the linked guide.
- imthenachoman 8y agoI have never had issues with it but now you have me worried. I added a warning to the guide and will do some research.
- imthenachoman 8y agoWould you mind pasting the output of `deborphan` showing those two or creating a new issue on my GitHub page? I want to show it to the author of `deborphan` to see if he knows why its happening. https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/new https://github.com/imthenachoman/How-To-Secure-A-Linux-Serve...
- imthenachoman 8y agoHey folks! I am the author of this guide. I did not know this was here or I would have commented sooner. This is my first time putting a guide like this together. I see a lot of really great feedback and I will be incorporating them into future updates. I don't have time today but I will reply individually to all the comments that I can. I appreciate any/all feedback/advice. If possible, could future issues be submitted to GitHub because it is easier for me to manage if everything is in one place. https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/issues/new https://github.com/imthenachoman/How-To-Secure-A-Linux-Serve... But right now I need to find all the other places this guide is linked to with comments so I can address them too. :)
- wodny 8y agoIt has been mentioned in one of the pull request that this guide was being discussed on HN right after it got here: https://github.com/imthenachoman/How-To-Secure-A-Linux-Server/pull/7 https://github.com/imthenachoman/How-To-Secure-A-Linux-Serve.... There are two fresh threads on Reddit: https://www.reddit.com/r/linuxadmin/comments/arx7xo/howtosecurealinuxserver_an_evolving_howto_guide/ https://www.reddit.com/r/linuxadmin/comments/arx7xo/howtosec... https://www.reddit.com/r/linux/comments/arx7st/howtosecurealinuxserver_an_evolving_howto_guide/ https://www.reddit.com/r/linux/comments/arx7st/howtosecureal...
- imthenachoman 8y agoHoly crap! I did not see those two on Reddit. I had posted to Reddit and was monitoring those comments but didn't know of these. Thanks! Although, I'm sad both of those threads got more comments then the original one I posted. Not that it really matters in life. Heh. And yes, I saw that pull request but I didn't know what HN was at that time. After I saw that comment, and before they said HN = Hacker News, I started Googling around to see if I could find where my repo was being mentioned. That is how I found this post in HN. Thanks!