8 ms·

Securing Ubuntu

177 points by namidark 14y ago

wereHamster 14y ago
Why use iptables instead of ufw? Also, last I heard chroot is not secure for sandboxing, so why bother with it at all? I'd use cgroups or even lxc containers (those are pretty well supported in the latest ubuntu).
- StavrosK 14y ago
  And why a chroot instead of just a dedicated user? If they can get root, it's game over anyway, isn't it?
  - maaku 14y ago
    I don't think chroot does what you think it does.
    - StavrosK 14y ago
      What do you think it does?
      - rjbond3rd 14y ago
        Changes the filesystem root to be inside a certain folder.
        StavrosK 14y ago
        Yeah, so what's the point? If you have a separate user, you can't access other users' files anyway, unless you have root. You can access the system files, but there should be nothing of much interest in there anyway.
        gvb 14y ago
        You should have only the minimum necessary commands and libraries in your chroot jail which minimizes the attack surface. For instance, I use chroot with rssh to only allow sFTP/scp/rsync and not allowing chroot users to get to a shell prompt. In my chroot /usr/bin, there are only two files (rssh, scp). In my chroot /lib there are only 16 files. Compare to: 1670 in /bin /usr/bin /sbin 50507 in /lib /usr/lib /usr/lib64
        munin 14y ago
        Unless some of those files are setuid root ... what does it matter? Your chroot jail is still going to have a very rich attack surface - the kernel.
        jiggy2011 14y ago
        I guess to prevent an application (e.g a web server running as a non root user) from easily reaching outside and running arbitrary shell commands or reading files. For example if you have something horrible like a PHP script that does: echo system($_POST['whatever']) you at least should try and prevent it from doing "ls" and "cat" to start exploring the filesystem.
  - throwaway2048 14y ago
    why not both? After chrooting use setuid()/setguid() for an individual user.
    - StavrosK 14y ago
      Because of the added complexity.
  - perlgeek 14y ago
    You're correct. chroot works on a per-process basis by adding a relative fs root into the process (or process table? dunno) memory. If you're root, you can change it back to /.
- justincormack 14y ago
  It is probably better than nothing (hard to get root if everything in the chroot is running as a user, no suid files, few devices). Probably need a kernel exploit then to get out. A container is better though.
dmix 14y ago
I'd also recommend using http://www.modsecurity.org/ http://www.modsecurity.org/ with nginx/apache.
- jiggy2011 14y ago
  Is there any straightforward guide to getting started with mod_security? I installed it on a box with some ruleset and it seems to basically block everything. The configuration format also seems far from simple.
  - curiouscats 14y ago
    I liked http://www.thefanclub.co.za/how-to/how-install-apache2-modsecurity-and-modevasive-ubuntu-1204-lts-server http://www.thefanclub.co.za/how-to/how-install-apache2-modse... For some rules to allow Wordpress to work http://wpsecure.net/2012/01/using-mod_security-2-with-wordpress/ http://wpsecure.net/2012/01/using-mod_security-2-with-wordpr...
  - druiid 14y ago
    Well, I'm not sure if I'd suggest looking at mod_security in a 'straightforward' manner particularly. Doing so will likely get you frustrated with the interaction you are likely to have with any default rule-set. A good way to look at it is to start by installing it and a good rule-set. A good one could be the gotroot/ASL kit http://updates.atomicorp.com/channels/rules/ http://updates.atomicorp.com/channels/rules/ From that point your best bet would perhaps be one of the freely available 'getting started' guides like https://www.feistyduck.com/books/modsecurity-handbook/gettingStarted.html https://www.feistyduck.com/books/modsecurity-handbook/gettin... Mostly though, my suggestion is to deploy it, have your application break (you should be doing this all in a staging environment), figure out if unbreaking your application can be done with scripting changes, bypassing rules, etc.
- M4v3R 14y ago
  Installing mod_security can have nasty side effects. For example, I lost many hours finding cause of my AJAX file uploads failing. It turned out to be caused by this module.
martinced 14y ago
Is there any drawback to opening /etc/fstab and making it read-only as suggested in the article? Why isn't this the default? (in other words: what does benefit from ro not being the default besides people trying to hack into the box?)
- reidrac 14y ago
  It really doesn't matter, its permissions are 0664 and the uid/guid are 0/0 (root.root). It may protect you in case of a mistake, your editor should warn you that the file is read-only. I don't like the post, it's too complicated for small benefits (if any). Besides there's no "generic secure setup", I think it depends on what you want to do with that server (ie. making /dev/shm read only may have impact on some software). EDIT: I think I misunderstood your comment.
  - deluxaran 14y ago
    I tend to agree with you about the "generic secure setup". The problem with making read only some parts of the file system, as other persons said too, is when a legit application will want to access those locations, a better solution is to remove execute rights on the desired locations.
- agwa 14y ago
  > what does benefit from ro not being the default /dev/shm is used by the POSIX shared memory implementation (i.e. shm_open) and can also be used as a general temporary directory (like /tmp) that's memory-backed. Legitimate applications use it and legitimate applications will break if you follow this advice. There's nothing inherently unsafe with /dev/shm - it's very similar to /tmp. It's possible some applications use POSIX shared memory unsafely, but that's a problem with those applications. Doing something drastic like making /dev/shm read-only, if it even serves a legitimate security purpose, should only be done as a last resort.
  - WestCoastJustin 14y ago
    Read-only would be bad! You want no-execute on /tmp/, /var/tmp, /dev/shm and they should be their own partitions on a secure system. The reasoning is that if someone does break into your system via apache/php or something else, they will almost always want to get a binary to elevate their access level (i.e. root the machine). They download a binary to a temporary location and execute the code (This can happen in apache/php directories too but most of the time these are locked down). Setting these to no-execute does not allow the attacker to execute code on your system. This is just a minor slowdown but does greatly help to prevent automated attacks. The goal is to slow people down to improve your chances of being alerted. I think this is fiarly standard and is listed as a best practice via the NSA RHEL hardening guide [1]. [1] www.nsa.gov/ia/_files/os/redhat/rhel5-guide-i731.pdf
agwa 14y ago
> A common exploit vector is going through shared memory (which can let you change the UID of running programs and other malicious actions). What? Citation needed. /dev/shm is simply a mount point for a memory-backed filesystem, much like /tmp. There's absolutely nothing special you can do with it. The POSIX shared memory (i.e. shm_open) implementation on Linux does put its files in /dev/shm. By making /dev/shm read-only you're effectively disabling POSIX shared memory. Maybe some applications have vulnerabilities because they use POSIX shared memory unsafely? In that case you probably want to identify and upgrade those applications instead of disabling something system-wide that may cause legitimate programs to break.
- reidrac 14y ago
  To be honest, I don't know what the author means; but I've seen web exploits using /dev/shm as alternative location to dump scripts (the other usual suspects are /tmp and /var/tmp). The web server process can write there, spawn a new process, etc. I think it's better to protect the entry point (ie. in PHP, use php.ini open_basedir).
  - bdonlan 14y ago
    I could see making all writable filesystems noexec - _that_ would make sense. Making all of /dev/shm read-only, and thus breaking the POSIX shared memory API? Madness.
- namidark 14y ago
  It is a common hiding place once the box has been broken into, not necessarily something that would be attacked right away. If you look on google you can see plenty of evidence of people's boxes being compromised through insecure shared memory and you can find plenty of examples of shared memory being exploited for privilege escalation and hiding malicious files.
  - agwa 14y ago
    What makes /dev/shm a better hiding place than /tmp or /var/tmp?
pilom 14y ago
Bastille linux hasn't been updated in close to 6 years. A year ago they announced that they were restarting development but there hasn't been a release since. Just saying I'd be a little nervous that it hasn't kept up with changes in Ubuntu.
- nisa 14y ago
  Beware of bastille - not sure if this still an issue but this is a dangerous bug: http://serverfault.com/q/444948/100094 http://serverfault.com/q/444948/100094
joelhaasnoot 14y ago
Another little tidbit you can use in addition to fail2ban is to just change the SSH port. I generally use something in the 2000 range - 2022, 2222 etc. That way your poor VPS isn´t even handling all the people knocking on your SSH door.
- vincentkriek 14y ago
  I don't agree with this. A simple nmap is enough to find the SSH port. This will get you in trouble when you're on a network which blocks port. The ssh port is often open (at least if it's an establishment where programmers work) and a port like 2022 is not.
_stephan 14y ago
Some commenters here seem to suggest that the advice on that blog post is somewhat outdated or suboptimal. I'm curious, is there something like a commonly agreed on best practice for securing/hardening a modern Ubuntu production web server? Would maybe anyone care to provide a bullet point list of the most important steps?
- csense 14y ago
  The most important steps are: 1) Making sure SSH only accepts public keys. This removes the possibility of a hacker guessing a password. 2) Use fail2ban. This rate-limits SSH attempts. 3) Keep your packages up-to-date, especially Web applications. 4) Use a default-deny firewall configuration, and make sure your services bind to the localhost interface. Letting the outside world directly talk to your database, Redis and memcache lets someone see your application data without any security checks, and this should only be possible for the application itself.
  - _stephan 14y ago
    Thanks!
graue 14y ago
Not a fan of articles that say "Here, type this command" and "Put this in that file" without explaining what's going on or even linking to more information. The section that suggests putting tmpfs /dev/shm tmpfs defaults,ro 0 0 into /etc/fstab might as well be quoting a line of ancient scripture. What does that do? Why does it solve the problem described? No answer. Likewise, the part about sshd begins "make sure these lines exist to secure SSH:" but does not explain what the 5 lines that follow do or why it is important. I prefer guides that explain what they are telling you to do and why, providing a basis for real understanding. To that end, I'd recommend Linode's "Securing Your Server" guide[1] over this. [1] https://library.linode.com/securing-your-server https://library.linode.com/securing-your-server
- rotskoff 14y ago
  I agree, it's not at all clear why one should do any of these things.
- namidark 14y ago
  I've added some comments to the ssh config to be more clear and provided some examples and a citation to the shared memory line.
  - tripzilch 14y ago
    I found your article to be an interesting start-off point to read up more about certain tools and techniques on securing Ubuntu Linux machines. So it's absolutely useful, good of you to put it there. Some criticisms on, however: If the above was the goal, for an advanced audience, it could have been way way shorter. More like a checklist of tools and techniques, short paragraph explaining your choice and maybe a link to your particular config file if you think that's applicable. And that'd be fine, even if it means that I personally would have to Google most things to find out what they're for, it's a great starting point. If, however, your goal was introducing securing an Ubuntu box to relative newbies, by which I mean people who are quite familiar on the Linux command line (like me) but lack specific knowledge about networking, iptables and the tools of the trade (again, like me), your article should have been way way longer than it is :) Or maybe not longer necessarily, but way more in-depth. Maybe a series, even (that would be awesome, but perhaps a lot of work, I'm not saying you should put in a lot of time that you maybe don't have, just that what you do write ought to be optimal for the effort put in ;) ). Most of what seems to be missing in your article are answers to the questions "why (should I use this)?" and "what (is this for)?", both for the "advanced" audience, as for the "newbies". Probably most important is that you must state upfront that this article is about securing a web server. Some of the choices you make are relevant only for web servers, but it wasn't until halfway that you mention "since this machine will be running as a web server ..." and before there were some points I was wondering "hm I'm not sure if I'd want to lock my (desktop) box so tight, but hey this guy seems to know what he's talking about" ... maybe I'm stupid and I should have noticed earlier. In all fairness, if you'd have just titled your article "Securing an Ubuntu web server", I probably wouldn't even have clicked the link (cause I don't have much use for that info right now) and you wouldn't have got all this feedback ;-) You should have a short sentence explaining "Fail2ban is a program that ...", and another sentence going "iptables is a tool/configfile part of .../command/(no idea honestly[0]) that is used for ...". Not asking for an in-depth explanation, but a one-sentence "what it is". Most of your checklist of Bastille settings are just cryptic enough that I cannot guess what they would exactly do and whether my settings should be the same as yours. If you target an "advanced" audience, you could suffice with "I'm using Bastille for XYZ because it's good at doing PQR, check its manual, here's my config, in particular pay attention to options ABC". For a more "newbie" audience you might want to say "Bastille is a hardening program, it consists of a set of tools and programs and configures them and your system to harden it. A few examples are PQR, but there's much more". "These are my settings" (do these go into a config file or is it a list of questions that the sudo bastille command asks? you can assume that most of your audience is not using your article as a step-by-step tutorial, so this is not obvious), and then explain succinctly what each of them does. Maybe just copy bits from the Bastille manpage. Or maybe just explain the ones that are different from the default options, and/or the ones that you suppose people might want to change, and say for the others just "make JKL extra secure, leave this set to 'on', <link to docs> for more info". Hope that was useful for you! (And I don't mean you should do a complete rewrite of this article btw, you can also just take this general advice for the next one you write) [0] I'm guessing "configfile" and part of some low level Linux networking thing? If I had been sitting in front of a Linux desktop I'd have been reading the manpage on it by now.
dfc 14y ago
Can anyone share their experiences with the grsecurity/pax patches? I have only used it on firewalls that had no responsibilities other than moving packets. Have you had any issues using the patches on machines with more responsibilities (www/db/mail) and/or greater user interaction ie. desktop use??
- nisa 14y ago
  I used it on several multi-purpose machines, no desktop through. I just patched the stock Ubuntu kernel¹. If you enable pax you have to watch out for applications that are incompatible with it. GRUB, Java, mod_python broke out of the box, but you can change the flags for pax on a per binary basis. It's probably useful to write some apt rules to apply the changes on every update of the program. Otherwise there where no problems. Using RBAC and configuring it on the other hand is a more difficult process. 1: http://compilefailure.blogspot.de/2011/02/grsecurity-patched-ubuntu-server-lts.html http://compilefailure.blogspot.de/2011/02/grsecurity-patched...
donniezazen 14y ago
It seems like Unity and Ubuntu will continue to become more and more social. Social in simple words mean sending private data to Canonical and third party servers. It's going to get very hard to keep Ubuntu box secure.
- axylone 14y ago
  Source? ubuntu-server has nothing to do with Unity.
  - donniezazen 14y ago
    The security article is relevant to any system based on Ubuntu core with server like processes. You don't actually have to be on ubuntu-server running on a dedicated server computer in your basement. That being said. I think there is no harm in discussing Ubuntu Desktop security. Sources:- https://www.eff.org/deeplinks/2012/10/privacy-ubuntu-1210-amazon-ads-and-data-leaks https://www.eff.org/deeplinks/2012/10/privacy-ubuntu-1210-am...
zx2c4 14y ago
Why does nobody ever mention grsecurity/pax? http://www.grsecurity.net http://www.grsecurity.net It's worth your time.
- namidark 14y ago
  I mention it at the end of the article, unfortunately the latest linux kernel and latest patches from GRSecurity don't compile.
  - zokier 14y ago
    Do you really need bleeding edge kernel? Why not use the stable grsec version?
fduran 14y ago
I appreciate the effort but this advice is not especially good. For example fail2ban is fine but it's almost always better to just use ssh key pairs. His firewall configuration doesn't do anything basically, just gives access to the network services that are already running, and logging firewall errors is only good if they are actionable (they rarely are) otherwise they are just filling up disk space. The sysctl entries, I haven't checked one by one by the look like the defaults in a modern distro.
- namidark 14y ago
  Both are used, and password logins are disabled. As I linked to in the article, those sysctl tweaks are recommended by the NSA handbook.
  - fduran 14y ago
    You are right, keys are used in the OP, not sure then why you'd use fail2ban for ssh but OK. Regarding the sysctl entries, I reviewed them down to execshield (after that there are tweaking entries) in a Amazon EC2 vanilla Ubuntu distro and all the entries are the same except log_martian (debatable, again, are you going to look at the logs, if then what are you going to do with them) and accept_redirects. Execshield is a RedHat thing, it's not used in Ubuntu.
moe 14y ago
Lots of questionable advice here (/dev/shm, fail2ban, PermitRootLogin no, "sysctl hardening") with little explanation and glaring omissions (SSH-port left unchanged, no check for listening services, no pointer to an iptables-wrapper). None of the bullets seems immediately harmful security-wise, but many of them have the potential to break legitimate applications or lock you out of your host. If you intend to follow this guide then make sure you fully understand each step before applying it.
- iuguy 14y ago
  To be fair, the value of changing SSH ports versus using only public key authentication is somewhat debatable. The pros always come down to eliminating noise from automated attacks. The cons always come down to having to specify a non-standard port to clients. If you only support public key based authentication, then I'd argue there's little to gain by moving ports.
  - pekk 14y ago
    Doesn't reducing log noise from automated attacks improve your ability to see significant events in the log? And why would it be hard to use a non-standard port?
    - count 14y ago
      It did, and probably still does to a lesser extent. It doesn't cost any real time or effort to scan most port ranges these days - especially with EC2 and other big cloud vendors and super fast multi-gigabit network connections.
  - moe 14y ago
    the value of changing SSH ports versus using only public key authentication is somewhat debatable. No, there never was a debate. The two changes address different concerns and you should always do both. You turn off password authentication to prevent targeted brute force attacks, keyloggers and (more common) accidents such as typing your password into the shell. You change the SSH-port to improve your odds in case of a vulnerability in the SSH-daemon itself (sweep-scans tend to probe only the default port).
TwistedWave 14y ago
I rely a lot on apparmor to secure an Ubuntu installation. It offers a very fine grained control on what processes are allowed to do, what files they can access. I find it more powerful and easier to setup than a chroot.
WestCoastJustin 14y ago
Anyone interested in solid advise / best parctices should read the NSA RHEL hardening guide [1]. It is great and provides nice details. They also have a tips pamphlet for quick reading but limited details [2]. [1] http://www.nsa.gov/ia/_files/os/redhat/rhel5-guide-i731.pdf http://www.nsa.gov/ia/_files/os/redhat/rhel5-guide-i731.pdf [2] http://www.nsa.gov/ia/_files/factsheets/rhel5-pamphlet-i731.pdf http://www.nsa.gov/ia/_files/factsheets/rhel5-pamphlet-i731....
- glazskunrukitis 14y ago
  This is really solid, thanks for sharing.
- runjake 14y ago
  You may have missed it, but the article mentions and links to the NSA guide.
- meaty 14y ago
  A big note to add: if you want a secure system, forget Ubuntu entirely and go with CentOS or RHEL with SELinux enabled and locked down correctly. It's as damn near bullet proof as you can get.
  - jiggy2011 14y ago
    Perhaps, though I am more familiar with Ubuntu/Debian as I have used it extensively on the desktop. This means I am more confident in the implications of my actions and also have more consistent dev/deploy platforms.
    - tesmar2 14y ago
      It is worth switching, in my opinion, because of all the man-hours put into securing RHEL. The systems aren't THAT different and if you are pretty linux saavy already, I'd guess that you could have an understanding of the differences learned in ~6 weeks.
  - fulafel 14y ago
nisa 14y ago
If you really need security consider using the grsecurity patchset¹ and pax². It's possible to patch the Ubuntu default kernel without problems. However understanding and configuring this takes time and patience. There is a great guide on Wikibooks through³ 1: http://grsecurity.net/ http://grsecurity.net/ 2: http://pax.grsecurity.net/ http://pax.grsecurity.net/ 3: http://en.wikibooks.org/wiki/Grsecurity http://en.wikibooks.org/wiki/Grsecurity
rlpb 14y ago
What are your credentials? How do I know that your instructions are correct and do not actually compromise my security? Have you filed security bugs for all these ways in which Ubuntu is apparently insecure by default? What does the security team have to say about this?
- jiggy2011 14y ago
  You should do due diligence on any advice you read on the internet if you are using it for something important, not just copy & paste. These are not "security bugs" in the sense of 0-day remote code execution vulnerabilities. These are simply configuration options which may increase security for certain use cases. Setting up specific chroots , firewall rules and restrictions could well not make sense for many use cases. Also stuff like fail2ban can lock you out of your server and requiring key based auth can be more complicated to get end users to configure if you are setting up for something like a shared hosting service.
berlinbrown 14y ago
I am not an admin or anything. But won't the firewall rules lock down any access to : subversion, git, irc, or pretty much anything that doesn't use http or https.
- tomschlick 14y ago
  Yes, inbound at least unless you specifically open those ports or allow all from a specific ip. Thats how it should be.
- jiggy2011 14y ago
  git can work over SSH IIRC.
ElliotH 14y ago
Rather than manually configuring iptables, save yourself some time and use UFW (the Uncomplicated FireWall) https://help.ubuntu.com/10.04/serverguide/firewall.html https://help.ubuntu.com/10.04/serverguide/firewall.html which is far easier and just as secure (and reduces the risk of misconfiguration).
systematical 14y ago
I'm more interested in Ubuntu being more stable as a desktop environment than secure. If I was worried about security I'd use Debian stable or some other operating system more well known for its security like OpenBSD.
- jiggy2011 14y ago
  I imagine Canonical makes more revenue from supporting server installations than it gets from the desktop.
JimmaDaRustla 14y ago
Thanks! I use RamNode, which means I can't use Arch Linux on the OpenVZ, so this is appreciated.
geuis 14y ago
Oh boy, there's a couple of good points in here but so much that is wrong. Off to bat, don't edit iptables manually anymore. Use ufw. I just setup a new Ubuntu server (12.10) this weekend and while I'm still tweaking it, I've been documenting my process. I'll make it available here: https://docs.google.com/document/d/1J2o6Lmq00Lk3k3HPUVKnOvSKaIp-S6NPu8VPdsUSR8k https://docs.google.com/document/d/1J2o6Lmq00Lk3k3HPUVKnOvSK...
- crdoconnor 14y ago
  >Off to bat, don't edit iptables manually anymore. Use ufw. For some reason this doesn't work too well in OpenVZ VMs. You have to go via iptables directly.
deleted 14y ago
[deleted]
borplk 14y ago
Is there a ready-made trusted image that people can use as a secure base? I'm not exactly good with dev ops and tweaking operating sytems. Even If I do attempt to fix it up and secure it, it will leave me uneasy because I don't fully understand the things I've done. If there was something trusted and ready-made would be great.

177 points by namidark 14y ago

Why use iptables instead of ufw? Also, last I heard chroot is not secure for sandboxing, so why bother with it at all? I'd use cgroups or even lxc containers (those are pretty well supported in the latest ubuntu).

StavrosK 14y ago
And why a chroot instead of just a dedicated user? If they can get root, it's game over anyway, isn't it?
- maaku 14y ago
  I don't think chroot does what you think it does.
  - StavrosK 14y ago
    What do you think it does?
    - rjbond3rd 14y ago
      Changes the filesystem root to be inside a certain folder.
      - StavrosK 14y ago
        Yeah, so what's the point? If you have a separate user, you can't access other users' files anyway, unless you have root. You can access the system files, but there should be nothing of much interest in there anyway.
        gvb 14y ago
        You should have only the minimum necessary commands and libraries in your chroot jail which minimizes the attack surface. For instance, I use chroot with rssh to only allow sFTP/scp/rsync and not allowing chroot users to get to a shell prompt. In my chroot /usr/bin, there are only two files (rssh, scp). In my chroot /lib there are only 16 files. Compare to: 1670 in /bin /usr/bin /sbin 50507 in /lib /usr/lib /usr/lib64
        munin 14y ago
        Unless some of those files are setuid root ... what does it matter? Your chroot jail is still going to have a very rich attack surface - the kernel.
        jiggy2011 14y ago
        I guess to prevent an application (e.g a web server running as a non root user) from easily reaching outside and running arbitrary shell commands or reading files. For example if you have something horrible like a PHP script that does: echo system($_POST['whatever']) you at least should try and prevent it from doing "ls" and "cat" to start exploring the filesystem.
- throwaway2048 14y ago
  why not both? After chrooting use setuid()/setguid() for an individual user.
  - StavrosK 14y ago
    Because of the added complexity.
- perlgeek 14y ago
  You're correct. chroot works on a per-process basis by adding a relative fs root into the process (or process table? dunno) memory. If you're root, you can change it back to /.
justincormack 14y ago
It is probably better than nothing (hard to get root if everything in the chroot is running as a user, no suid files, few devices). Probably need a kernel exploit then to get out. A container is better though.

dmix 14y ago

I'd also recommend using http://www.modsecurity.org/ http://www.modsecurity.org/ with nginx/apache.

jiggy2011 14y ago
Is there any straightforward guide to getting started with mod_security? I installed it on a box with some ruleset and it seems to basically block everything. The configuration format also seems far from simple.
- curiouscats 14y ago
  I liked http://www.thefanclub.co.za/how-to/how-install-apache2-modsecurity-and-modevasive-ubuntu-1204-lts-server http://www.thefanclub.co.za/how-to/how-install-apache2-modse... For some rules to allow Wordpress to work http://wpsecure.net/2012/01/using-mod_security-2-with-wordpress/ http://wpsecure.net/2012/01/using-mod_security-2-with-wordpr...
- druiid 14y ago
  Well, I'm not sure if I'd suggest looking at mod_security in a 'straightforward' manner particularly. Doing so will likely get you frustrated with the interaction you are likely to have with any default rule-set. A good way to look at it is to start by installing it and a good rule-set. A good one could be the gotroot/ASL kit http://updates.atomicorp.com/channels/rules/ http://updates.atomicorp.com/channels/rules/ From that point your best bet would perhaps be one of the freely available 'getting started' guides like https://www.feistyduck.com/books/modsecurity-handbook/gettingStarted.html https://www.feistyduck.com/books/modsecurity-handbook/gettin... Mostly though, my suggestion is to deploy it, have your application break (you should be doing this all in a staging environment), figure out if unbreaking your application can be done with scripting changes, bypassing rules, etc.
M4v3R 14y ago
Installing mod_security can have nasty side effects. For example, I lost many hours finding cause of my AJAX file uploads failing. It turned out to be caused by this module.

martinced 14y ago

Is there any drawback to opening /etc/fstab and making it read-only as suggested in the article? Why isn't this the default? (in other words: what does benefit from ro not being the default besides people trying to hack into the box?)

reidrac 14y ago
It really doesn't matter, its permissions are 0664 and the uid/guid are 0/0 (root.root). It may protect you in case of a mistake, your editor should warn you that the file is read-only. I don't like the post, it's too complicated for small benefits (if any). Besides there's no "generic secure setup", I think it depends on what you want to do with that server (ie. making /dev/shm read only may have impact on some software). EDIT: I think I misunderstood your comment.
- deluxaran 14y ago
  I tend to agree with you about the "generic secure setup". The problem with making read only some parts of the file system, as other persons said too, is when a legit application will want to access those locations, a better solution is to remove execute rights on the desired locations.
agwa 14y ago
> what does benefit from ro not being the default /dev/shm is used by the POSIX shared memory implementation (i.e. shm_open) and can also be used as a general temporary directory (like /tmp) that's memory-backed. Legitimate applications use it and legitimate applications will break if you follow this advice. There's nothing inherently unsafe with /dev/shm - it's very similar to /tmp. It's possible some applications use POSIX shared memory unsafely, but that's a problem with those applications. Doing something drastic like making /dev/shm read-only, if it even serves a legitimate security purpose, should only be done as a last resort.
- WestCoastJustin 14y ago
  Read-only would be bad! You want no-execute on /tmp/, /var/tmp, /dev/shm and they should be their own partitions on a secure system. The reasoning is that if someone does break into your system via apache/php or something else, they will almost always want to get a binary to elevate their access level (i.e. root the machine). They download a binary to a temporary location and execute the code (This can happen in apache/php directories too but most of the time these are locked down). Setting these to no-execute does not allow the attacker to execute code on your system. This is just a minor slowdown but does greatly help to prevent automated attacks. The goal is to slow people down to improve your chances of being alerted. I think this is fiarly standard and is listed as a best practice via the NSA RHEL hardening guide [1]. [1] www.nsa.gov/ia/_files/os/redhat/rhel5-guide-i731.pdf

agwa 14y ago

> A common exploit vector is going through shared memory (which can let you change the UID of running programs and other malicious actions). What? Citation needed. /dev/shm is simply a mount point for a memory-backed filesystem, much like /tmp. There's absolutely nothing special you can do with it. The POSIX shared memory (i.e. shm_open) implementation on Linux does put its files in /dev/shm. By making /dev/shm read-only you're effectively disabling POSIX shared memory. Maybe some applications have vulnerabilities because they use POSIX shared memory unsafely? In that case you probably want to identify and upgrade those applications instead of disabling something system-wide that may cause legitimate programs to break.

reidrac 14y ago
To be honest, I don't know what the author means; but I've seen web exploits using /dev/shm as alternative location to dump scripts (the other usual suspects are /tmp and /var/tmp). The web server process can write there, spawn a new process, etc. I think it's better to protect the entry point (ie. in PHP, use php.ini open_basedir).
- bdonlan 14y ago
  I could see making all writable filesystems noexec - _that_ would make sense. Making all of /dev/shm read-only, and thus breaking the POSIX shared memory API? Madness.
namidark 14y ago
It is a common hiding place once the box has been broken into, not necessarily something that would be attacked right away. If you look on google you can see plenty of evidence of people's boxes being compromised through insecure shared memory and you can find plenty of examples of shared memory being exploited for privilege escalation and hiding malicious files.
- agwa 14y ago
  What makes /dev/shm a better hiding place than /tmp or /var/tmp?

pilom 14y ago

Bastille linux hasn't been updated in close to 6 years. A year ago they announced that they were restarting development but there hasn't been a release since. Just saying I'd be a little nervous that it hasn't kept up with changes in Ubuntu.

nisa 14y ago
Beware of bastille - not sure if this still an issue but this is a dangerous bug: http://serverfault.com/q/444948/100094 http://serverfault.com/q/444948/100094

joelhaasnoot 14y ago

Another little tidbit you can use in addition to fail2ban is to just change the SSH port. I generally use something in the 2000 range - 2022, 2222 etc. That way your poor VPS isn´t even handling all the people knocking on your SSH door.

vincentkriek 14y ago
I don't agree with this. A simple nmap is enough to find the SSH port. This will get you in trouble when you're on a network which blocks port. The ssh port is often open (at least if it's an establishment where programmers work) and a port like 2022 is not.

_stephan 14y ago

Some commenters here seem to suggest that the advice on that blog post is somewhat outdated or suboptimal. I'm curious, is there something like a commonly agreed on best practice for securing/hardening a modern Ubuntu production web server? Would maybe anyone care to provide a bullet point list of the most important steps?

csense 14y ago
The most important steps are: 1) Making sure SSH only accepts public keys. This removes the possibility of a hacker guessing a password. 2) Use fail2ban. This rate-limits SSH attempts. 3) Keep your packages up-to-date, especially Web applications. 4) Use a default-deny firewall configuration, and make sure your services bind to the localhost interface. Letting the outside world directly talk to your database, Redis and memcache lets someone see your application data without any security checks, and this should only be possible for the application itself.
- _stephan 14y ago
  Thanks!

graue 14y ago

Not a fan of articles that say "Here, type this command" and "Put this in that file" without explaining what's going on or even linking to more information. The section that suggests putting tmpfs /dev/shm tmpfs defaults,ro 0 0 into /etc/fstab might as well be quoting a line of ancient scripture. What does that do? Why does it solve the problem described? No answer. Likewise, the part about sshd begins "make sure these lines exist to secure SSH:" but does not explain what the 5 lines that follow do or why it is important. I prefer guides that explain what they are telling you to do and why, providing a basis for real understanding. To that end, I'd recommend Linode's "Securing Your Server" guide[1] over this. [1] https://library.linode.com/securing-your-server https://library.linode.com/securing-your-server

rotskoff 14y ago
I agree, it's not at all clear why one should do any of these things.
namidark 14y ago
I've added some comments to the ssh config to be more clear and provided some examples and a citation to the shared memory line.
- tripzilch 14y ago
  I found your article to be an interesting start-off point to read up more about certain tools and techniques on securing Ubuntu Linux machines. So it's absolutely useful, good of you to put it there. Some criticisms on, however: If the above was the goal, for an advanced audience, it could have been way way shorter. More like a checklist of tools and techniques, short paragraph explaining your choice and maybe a link to your particular config file if you think that's applicable. And that'd be fine, even if it means that I personally would have to Google most things to find out what they're for, it's a great starting point. If, however, your goal was introducing securing an Ubuntu box to relative newbies, by which I mean people who are quite familiar on the Linux command line (like me) but lack specific knowledge about networking, iptables and the tools of the trade (again, like me), your article should have been way way longer than it is :) Or maybe not longer necessarily, but way more in-depth. Maybe a series, even (that would be awesome, but perhaps a lot of work, I'm not saying you should put in a lot of time that you maybe don't have, just that what you do write ought to be optimal for the effort put in ;) ). Most of what seems to be missing in your article are answers to the questions "why (should I use this)?" and "what (is this for)?", both for the "advanced" audience, as for the "newbies". Probably most important is that you must state upfront that this article is about securing a web server. Some of the choices you make are relevant only for web servers, but it wasn't until halfway that you mention "since this machine will be running as a web server ..." and before there were some points I was wondering "hm I'm not sure if I'd want to lock my (desktop) box so tight, but hey this guy seems to know what he's talking about" ... maybe I'm stupid and I should have noticed earlier. In all fairness, if you'd have just titled your article "Securing an Ubuntu web server", I probably wouldn't even have clicked the link (cause I don't have much use for that info right now) and you wouldn't have got all this feedback ;-) You should have a short sentence explaining "Fail2ban is a program that ...", and another sentence going "iptables is a tool/configfile part of .../command/(no idea honestly[0]) that is used for ...". Not asking for an in-depth explanation, but a one-sentence "what it is". Most of your checklist of Bastille settings are just cryptic enough that I cannot guess what they would exactly do and whether my settings should be the same as yours. If you target an "advanced" audience, you could suffice with "I'm using Bastille for XYZ because it's good at doing PQR, check its manual, here's my config, in particular pay attention to options ABC". For a more "newbie" audience you might want to say "Bastille is a hardening program, it consists of a set of tools and programs and configures them and your system to harden it. A few examples are PQR, but there's much more". "These are my settings" (do these go into a config file or is it a list of questions that the sudo bastille command asks? you can assume that most of your audience is not using your article as a step-by-step tutorial, so this is not obvious), and then explain succinctly what each of them does. Maybe just copy bits from the Bastille manpage. Or maybe just explain the ones that are different from the default options, and/or the ones that you suppose people might want to change, and say for the others just "make JKL extra secure, leave this set to 'on', <link to docs> for more info". Hope that was useful for you! (And I don't mean you should do a complete rewrite of this article btw, you can also just take this general advice for the next one you write) [0] I'm guessing "configfile" and part of some low level Linux networking thing? If I had been sitting in front of a Linux desktop I'd have been reading the manpage on it by now.

dfc 14y ago

Can anyone share their experiences with the grsecurity/pax patches? I have only used it on firewalls that had no responsibilities other than moving packets. Have you had any issues using the patches on machines with more responsibilities (www/db/mail) and/or greater user interaction ie. desktop use??

nisa 14y ago
I used it on several multi-purpose machines, no desktop through. I just patched the stock Ubuntu kernel¹. If you enable pax you have to watch out for applications that are incompatible with it. GRUB, Java, mod_python broke out of the box, but you can change the flags for pax on a per binary basis. It's probably useful to write some apt rules to apply the changes on every update of the program. Otherwise there where no problems. Using RBAC and configuring it on the other hand is a more difficult process. 1: http://compilefailure.blogspot.de/2011/02/grsecurity-patched-ubuntu-server-lts.html http://compilefailure.blogspot.de/2011/02/grsecurity-patched...

donniezazen 14y ago

It seems like Unity and Ubuntu will continue to become more and more social. Social in simple words mean sending private data to Canonical and third party servers. It's going to get very hard to keep Ubuntu box secure.

axylone 14y ago
Source? ubuntu-server has nothing to do with Unity.
- donniezazen 14y ago
  The security article is relevant to any system based on Ubuntu core with server like processes. You don't actually have to be on ubuntu-server running on a dedicated server computer in your basement. That being said. I think there is no harm in discussing Ubuntu Desktop security. Sources:- https://www.eff.org/deeplinks/2012/10/privacy-ubuntu-1210-amazon-ads-and-data-leaks https://www.eff.org/deeplinks/2012/10/privacy-ubuntu-1210-am...

zx2c4 14y ago

Why does nobody ever mention grsecurity/pax? http://www.grsecurity.net http://www.grsecurity.net It's worth your time.

namidark 14y ago
I mention it at the end of the article, unfortunately the latest linux kernel and latest patches from GRSecurity don't compile.
- zokier 14y ago
  Do you really need bleeding edge kernel? Why not use the stable grsec version?

fduran 14y ago

I appreciate the effort but this advice is not especially good. For example fail2ban is fine but it's almost always better to just use ssh key pairs. His firewall configuration doesn't do anything basically, just gives access to the network services that are already running, and logging firewall errors is only good if they are actionable (they rarely are) otherwise they are just filling up disk space. The sysctl entries, I haven't checked one by one by the look like the defaults in a modern distro.

namidark 14y ago
Both are used, and password logins are disabled. As I linked to in the article, those sysctl tweaks are recommended by the NSA handbook.
- fduran 14y ago
  You are right, keys are used in the OP, not sure then why you'd use fail2ban for ssh but OK. Regarding the sysctl entries, I reviewed them down to execshield (after that there are tweaking entries) in a Amazon EC2 vanilla Ubuntu distro and all the entries are the same except log_martian (debatable, again, are you going to look at the logs, if then what are you going to do with them) and accept_redirects. Execshield is a RedHat thing, it's not used in Ubuntu.

moe 14y ago

Lots of questionable advice here (/dev/shm, fail2ban, PermitRootLogin no, "sysctl hardening") with little explanation and glaring omissions (SSH-port left unchanged, no check for listening services, no pointer to an iptables-wrapper). None of the bullets seems immediately harmful security-wise, but many of them have the potential to break legitimate applications or lock you out of your host. If you intend to follow this guide then make sure you fully understand each step before applying it.

iuguy 14y ago
To be fair, the value of changing SSH ports versus using only public key authentication is somewhat debatable. The pros always come down to eliminating noise from automated attacks. The cons always come down to having to specify a non-standard port to clients. If you only support public key based authentication, then I'd argue there's little to gain by moving ports.
- pekk 14y ago
  Doesn't reducing log noise from automated attacks improve your ability to see significant events in the log? And why would it be hard to use a non-standard port?
  - count 14y ago
    It did, and probably still does to a lesser extent. It doesn't cost any real time or effort to scan most port ranges these days - especially with EC2 and other big cloud vendors and super fast multi-gigabit network connections.
- moe 14y ago
  the value of changing SSH ports versus using only public key authentication is somewhat debatable. No, there never was a debate. The two changes address different concerns and you should always do both. You turn off password authentication to prevent targeted brute force attacks, keyloggers and (more common) accidents such as typing your password into the shell. You change the SSH-port to improve your odds in case of a vulnerability in the SSH-daemon itself (sweep-scans tend to probe only the default port).

TwistedWave 14y ago

I rely a lot on apparmor to secure an Ubuntu installation. It offers a very fine grained control on what processes are allowed to do, what files they can access. I find it more powerful and easier to setup than a chroot.

WestCoastJustin 14y ago

Anyone interested in solid advise / best parctices should read the NSA RHEL hardening guide [1]. It is great and provides nice details. They also have a tips pamphlet for quick reading but limited details [2]. [1] http://www.nsa.gov/ia/_files/os/redhat/rhel5-guide-i731.pdf http://www.nsa.gov/ia/_files/os/redhat/rhel5-guide-i731.pdf [2] http://www.nsa.gov/ia/_files/factsheets/rhel5-pamphlet-i731.pdf http://www.nsa.gov/ia/_files/factsheets/rhel5-pamphlet-i731....

glazskunrukitis 14y ago
This is really solid, thanks for sharing.
runjake 14y ago
You may have missed it, but the article mentions and links to the NSA guide.
meaty 14y ago
A big note to add: if you want a secure system, forget Ubuntu entirely and go with CentOS or RHEL with SELinux enabled and locked down correctly. It's as damn near bullet proof as you can get.
- jiggy2011 14y ago
  Perhaps, though I am more familiar with Ubuntu/Debian as I have used it extensively on the desktop. This means I am more confident in the implications of my actions and also have more consistent dev/deploy platforms.
  - tesmar2 14y ago
    It is worth switching, in my opinion, because of all the man-hours put into securing RHEL. The systems aren't THAT different and if you are pretty linux saavy already, I'd guess that you could have an understanding of the differences learned in ~6 weeks.
- fulafel 14y ago

nisa 14y ago

If you really need security consider using the grsecurity patchset¹ and pax². It's possible to patch the Ubuntu default kernel without problems. However understanding and configuring this takes time and patience. There is a great guide on Wikibooks through³ 1: http://grsecurity.net/ http://grsecurity.net/ 2: http://pax.grsecurity.net/ http://pax.grsecurity.net/ 3: http://en.wikibooks.org/wiki/Grsecurity http://en.wikibooks.org/wiki/Grsecurity

rlpb 14y ago

What are your credentials? How do I know that your instructions are correct and do not actually compromise my security? Have you filed security bugs for all these ways in which Ubuntu is apparently insecure by default? What does the security team have to say about this?

jiggy2011 14y ago
You should do due diligence on any advice you read on the internet if you are using it for something important, not just copy & paste. These are not "security bugs" in the sense of 0-day remote code execution vulnerabilities. These are simply configuration options which may increase security for certain use cases. Setting up specific chroots , firewall rules and restrictions could well not make sense for many use cases. Also stuff like fail2ban can lock you out of your server and requiring key based auth can be more complicated to get end users to configure if you are setting up for something like a shared hosting service.

berlinbrown 14y ago

I am not an admin or anything. But won't the firewall rules lock down any access to : subversion, git, irc, or pretty much anything that doesn't use http or https.

tomschlick 14y ago
Yes, inbound at least unless you specifically open those ports or allow all from a specific ip. Thats how it should be.
jiggy2011 14y ago
git can work over SSH IIRC.

ElliotH 14y ago

Rather than manually configuring iptables, save yourself some time and use UFW (the Uncomplicated FireWall) https://help.ubuntu.com/10.04/serverguide/firewall.html https://help.ubuntu.com/10.04/serverguide/firewall.html which is far easier and just as secure (and reduces the risk of misconfiguration).

systematical 14y ago

I'm more interested in Ubuntu being more stable as a desktop environment than secure. If I was worried about security I'd use Debian stable or some other operating system more well known for its security like OpenBSD.

jiggy2011 14y ago
I imagine Canonical makes more revenue from supporting server installations than it gets from the desktop.

JimmaDaRustla 14y ago

Thanks! I use RamNode, which means I can't use Arch Linux on the OpenVZ, so this is appreciated.

geuis 14y ago

Oh boy, there's a couple of good points in here but so much that is wrong. Off to bat, don't edit iptables manually anymore. Use ufw. I just setup a new Ubuntu server (12.10) this weekend and while I'm still tweaking it, I've been documenting my process. I'll make it available here: https://docs.google.com/document/d/1J2o6Lmq00Lk3k3HPUVKnOvSKaIp-S6NPu8VPdsUSR8k https://docs.google.com/document/d/1J2o6Lmq00Lk3k3HPUVKnOvSK...

crdoconnor 14y ago
>Off to bat, don't edit iptables manually anymore. Use ufw. For some reason this doesn't work too well in OpenVZ VMs. You have to go via iptables directly.

deleted 14y ago

[deleted]

borplk 14y ago

Is there a ready-made trusted image that people can use as a secure base? I'm not exactly good with dev ops and tweaking operating sytems. Even If I do attempt to fix it up and secure it, it will leave me uneasy because I don't fully understand the things I've done. If there was something trusted and ready-made would be great.