3 ms·

Yeah noticed that as well. First thing I do is (on debian), update packages and the kernel, install ufw (allow only 22), fail2ban, postfix and logwatch. Then se

by bananas 13y ago

Yeah noticed that as well. First thing I do is (on debian), update packages and the kernel, install ufw (allow only 22), fail2ban, postfix and logwatch. Then setup a non privileged user, sudo, turn of root ssh and password auth and move to key based auth. Then I can sleep/configure the rest of the system. Been meaning to package this all up in an ansible playbook but I can't be bothered :( I'd be using FreeBSD if they offered it. Much like OpenBSD it's a lot more pleasant for sysadmins!

hiphopyo 13y ago
Awesome, please do! Don't forget https://github.com/kickstarter/rack-attack https://github.com/kickstarter/rack-attack as well though.
dmix 13y ago
I'm amazed noone ever mentions GRSecurity as must have. It will reduce the risks of an actual comprise significantly. Non-patched Linux is always full of rootkit potential.
- bananas 13y ago
  Well they don't issue GRSecurity kernels. To be fair I'd just use FreeBSD which has a little better kernel architecture with respect to security.
- hiphopyo 13y ago
  Some more regarding GRSecurity: https://mail-archive.com/misc@openbsd.org/msg127100.html https://mail-archive.com/misc@openbsd.org/msg127100.html
jnsaff2 13y ago
It took me about an hour to put all of this into an ansible playbook. I think there are couple of examples out on blogs about this as well. If there is interest I'll share my playbook. Writing this playbook paid off already for the second installation of the box.
- nikhilalmeida 13y ago
  Please do.
- radioact1ve 13y ago
  Don't mind me, I'll just wait here till you're ready.
- OWaz 13y ago
  The most I know about securing a Linux vps I picked up from this: http://feross.org/how-to-setup-your-linode/ http://feross.org/how-to-setup-your-linode/ . If there is an ultimate resource on how to start securing a *nix vps then please do let me know. If I knew what to search for I could but a lot of this is unknown unknowns to me.
  - mverwijs 13y ago
    After having fixed a sudo user (and confirm it works!), please also disable PermitRootLogin by setting it to 'no'. Restart ssh after editing sshd_conf. If you're squeemish, run dropbear on another port so you can log into your machine in case you made a boo-boo configuring openssh. Also, more resources for security settings: https://benchmarks.cisecurity.org/downloads/multiform/index.cfm https://benchmarks.cisecurity.org/downloads/multiform/index....
- flexd 13y ago
  I'd like a copy of that playbook too :-), publish it somewhere?
- mimiflynn 13y ago
  yes please
- nolite 13y ago
  ditto
- chris_mahan 13y ago
  Please! Give us your address so we can send you money.
iamtew 13y ago
Moving your SSH server away from port 22 and disabling password logins is also something you should consider doing. Small thing but at least then it doesn't shout "Hey I'm a *nix box with remote login enabled"
- api 13y ago
  Is it actually likely that someone would brute force a password? I always install fail2ban, but it's more just to stop bots from filling up my logs with junk and eating my bandwidth/CPU. Unless you use dumb passwords, the odds are phenomenally low.
  - mindcrime 13y ago
    I'm not sure, but I've come around to realize that it's always worth installing and configuring fail2ban just for the reasons you mentioned, if nothing else. I have a demo server that, for a while, was being hammered so hard by ssh brute-force bots that it kept getting knocked offline. I installed fail2ban and no more problems since. I personally can't recommend fail2ban highly enough.
    - MertsA 13y ago
      fail2ban is based on the horribly flawed premise of correctly parsing arbitrary text logs from tons of different programs that wrote their log format without any concerns about parsing that output later. Not surprisingly there have been numerous vulnerabilities in fail2ban that let an attacker ban arbitrary hosts such as your DNS server, database, etc. https://www.google.com/search?q=fail2ban+dos+bug https://www.google.com/search?q=fail2ban+dos+bug A better approach is something like pam_abl which is a pam module that will accomplish mostly the same thing but only for login attempts and without the crappy plain text log parsing.
    - flexd 13y ago
      Have you tried denyhosts? I've always been installing denyhosts but I have not compared the two.
phaker 13y ago
> update packages and the kernel PSA: If you just update the kernel using system tools within the instance, it'll still boot the old one. (At least it did few months ago.) The grub (or other bootloader) and kernels that are installed inside the instance's disk image are silently ignored and the vm uses a kernel stored outside of your instance's image. You can select which kernel image you want to boot in the control panel. The problem is that the kernels available in there are updated very, very rarely. For popular distributions like Ubuntu they can lag several months, for others it can get up to, say, _two years_ [1]. Since digitalocean "doesn't support custom kernels" it appears that the only way to use something up to date (and stay secure) is to write a script that loads a custom kernel using kexec. :S edit: The fact that they use kvm (and that kexec works) which means real visualization but can't boot user's kernels is just very, very weird. The only technical reason i can think of is that they can't manage to make it work with their control panel. If that's the reason, then it's very worrying. (The "right" way is easier to code and it's the solution that you'd think of first and it makes custom kernels a non-problem, but they picked one that both has obvious problems and is harder to setup, i can't think of a reason why.) [1]: https://www.digitalocean.com/community/questions/how-can-i-boot-my-compiled-kernel-image-on-gentoo https://www.digitalocean.com/community/questions/how-can-i-b...
- bananas 13y ago
  It does still boot the old one. You have to power it off, change the kernel and power it up again.
- sn 13y ago
  Second shameless plug on this page: if running your own linux kernel or BSD is really important to you, http://prgmr.com http://prgmr.com can do that and we have out of band console access so you can debug why it's not working (I was having problems with that when I was checking whether some linux+xen bugs were also present on ec2.) At this time we only support xen paravirtualization(PV) though, which almost, but not quite, all major distributions have. Notably we can't run 64-bit freebsd right now because they only have HVM support. I'd like to change that but it needs more testing first. Here is a partial list of distributions supporting xen: http://wiki.xen.org/wiki/DomU_Support_for_Xen http://wiki.xen.org/wiki/DomU_Support_for_Xen Really any linux distribution with a modern kernel can be made to support it though as support has been in mainline for a long time now.

by bananas 13y ago

hiphopyo 13y ago

Awesome, please do! Don't forget https://github.com/kickstarter/rack-attack https://github.com/kickstarter/rack-attack as well though.

dmix 13y ago

I'm amazed noone ever mentions GRSecurity as must have. It will reduce the risks of an actual comprise significantly. Non-patched Linux is always full of rootkit potential.

bananas 13y ago
Well they don't issue GRSecurity kernels. To be fair I'd just use FreeBSD which has a little better kernel architecture with respect to security.
hiphopyo 13y ago
Some more regarding GRSecurity: https://mail-archive.com/misc@openbsd.org/msg127100.html https://mail-archive.com/misc@openbsd.org/msg127100.html

jnsaff2 13y ago

It took me about an hour to put all of this into an ansible playbook. I think there are couple of examples out on blogs about this as well. If there is interest I'll share my playbook. Writing this playbook paid off already for the second installation of the box.

nikhilalmeida 13y ago
Please do.
radioact1ve 13y ago
Don't mind me, I'll just wait here till you're ready.
OWaz 13y ago
The most I know about securing a Linux vps I picked up from this: http://feross.org/how-to-setup-your-linode/ http://feross.org/how-to-setup-your-linode/ . If there is an ultimate resource on how to start securing a *nix vps then please do let me know. If I knew what to search for I could but a lot of this is unknown unknowns to me.
- mverwijs 13y ago
  After having fixed a sudo user (and confirm it works!), please also disable PermitRootLogin by setting it to 'no'. Restart ssh after editing sshd_conf. If you're squeemish, run dropbear on another port so you can log into your machine in case you made a boo-boo configuring openssh. Also, more resources for security settings: https://benchmarks.cisecurity.org/downloads/multiform/index.cfm https://benchmarks.cisecurity.org/downloads/multiform/index....
flexd 13y ago
I'd like a copy of that playbook too :-), publish it somewhere?
mimiflynn 13y ago
yes please
nolite 13y ago
ditto
chris_mahan 13y ago
Please! Give us your address so we can send you money.

iamtew 13y ago

Moving your SSH server away from port 22 and disabling password logins is also something you should consider doing. Small thing but at least then it doesn't shout "Hey I'm a *nix box with remote login enabled"

api 13y ago
Is it actually likely that someone would brute force a password? I always install fail2ban, but it's more just to stop bots from filling up my logs with junk and eating my bandwidth/CPU. Unless you use dumb passwords, the odds are phenomenally low.
- mindcrime 13y ago
  I'm not sure, but I've come around to realize that it's always worth installing and configuring fail2ban just for the reasons you mentioned, if nothing else. I have a demo server that, for a while, was being hammered so hard by ssh brute-force bots that it kept getting knocked offline. I installed fail2ban and no more problems since. I personally can't recommend fail2ban highly enough.
  - MertsA 13y ago
    fail2ban is based on the horribly flawed premise of correctly parsing arbitrary text logs from tons of different programs that wrote their log format without any concerns about parsing that output later. Not surprisingly there have been numerous vulnerabilities in fail2ban that let an attacker ban arbitrary hosts such as your DNS server, database, etc. https://www.google.com/search?q=fail2ban+dos+bug https://www.google.com/search?q=fail2ban+dos+bug A better approach is something like pam_abl which is a pam module that will accomplish mostly the same thing but only for login attempts and without the crappy plain text log parsing.
  - flexd 13y ago
    Have you tried denyhosts? I've always been installing denyhosts but I have not compared the two.

phaker 13y ago

> update packages and the kernel PSA: If you just update the kernel using system tools within the instance, it'll still boot the old one. (At least it did few months ago.) The grub (or other bootloader) and kernels that are installed inside the instance's disk image are silently ignored and the vm uses a kernel stored outside of your instance's image. You can select which kernel image you want to boot in the control panel. The problem is that the kernels available in there are updated very, very rarely. For popular distributions like Ubuntu they can lag several months, for others it can get up to, say, _two years_ [1]. Since digitalocean "doesn't support custom kernels" it appears that the only way to use something up to date (and stay secure) is to write a script that loads a custom kernel using kexec. :S edit: The fact that they use kvm (and that kexec works) which means real visualization but can't boot user's kernels is just very, very weird. The only technical reason i can think of is that they can't manage to make it work with their control panel. If that's the reason, then it's very worrying. (The "right" way is easier to code and it's the solution that you'd think of first and it makes custom kernels a non-problem, but they picked one that both has obvious problems and is harder to setup, i can't think of a reason why.) [1]: https://www.digitalocean.com/community/questions/how-can-i-boot-my-compiled-kernel-image-on-gentoo https://www.digitalocean.com/community/questions/how-can-i-b...

bananas 13y ago
It does still boot the old one. You have to power it off, change the kernel and power it up again.
sn 13y ago
Second shameless plug on this page: if running your own linux kernel or BSD is really important to you, http://prgmr.com http://prgmr.com can do that and we have out of band console access so you can debug why it's not working (I was having problems with that when I was checking whether some linux+xen bugs were also present on ec2.) At this time we only support xen paravirtualization(PV) though, which almost, but not quite, all major distributions have. Notably we can't run 64-bit freebsd right now because they only have HVM support. I'd like to change that but it needs more testing first. Here is a partial list of distributions supporting xen: http://wiki.xen.org/wiki/DomU_Support_for_Xen http://wiki.xen.org/wiki/DomU_Support_for_Xen Really any linux distribution with a modern kernel can be made to support it though as support has been in mainline for a long time now.