4 ms·

And the real issue was (is?) the fact that web application process had write access to a location it was able to execute code from. It is well-known for decade

And the real issue was (is?) the fact that web application process had write access to a location it was able to execute code from. It is well-known for decades (at least since PHP got popular, which was in '90s!) that such locations should only be writeable by the user(s) who maintain the code and not anyone else. Sometimes, such setup can't be done on dirt-cheap FTP-only shared hosting services (shrugs), but certainly not on VPSes. Ignorance is bliss.

jlgaddis 13y ago
Yep, this can be a lesson to developers who read this. If your web application must write to files, keep them outside of the DocumentRoot! edit: I know a lot of HN'ers are fans of Ubuntu but, FWIW, SELinux (default on RHEL and derivatives) would have prevented this.
- drdaeman 13y ago
  Outside of any executable location, not only DocumentRoot. Latter is just an easier and more popular target. I mean, all writeable directories (if any) must not only be not runnable by some server (like Apache with mod_php), but also be on devices mounted with noexec, like this is the case with /tmp.
  - jlgaddis 13y ago
    I'm not sure that'll help like you think it will. I can't verify it at the moment, but create a /tmp/phpinfo.php file with "<?php phpinfo(); ?>" in it and then run "php /tmp/phpinfo.php". Regardless of whether /tmp is mounted noexec I think you'll find that it executes (because the php binary is under /usr which almost certainly isn't mounted noexec).
    - drdaeman 13y ago
      Whoops, seems that you're right, I was wrong about interpreters taking noexec into account - either I misremembered something or things changed, but the fact is they don't. Sorry. Anyway, it's also better to have noexec than not. :) At least, it'll help if there's no complete shell access but only ability to run some binary by name, without args, or modify environment variables when running some executable.
- mhurron 13y ago
  BTW, is there any reason not to hook something like ClamAV into your upload function and scan files before they are saved?
  - trippy_biscuits 13y ago
    Obligatory xkcd: http://xkcd.com/463/ http://xkcd.com/463/ Why add more moving parts when they don't do anything but make more work? Scanning is a helpful idea, but not AV scanning. Regular vulnerability scanning can assess the platform security. At the very least, it can warn about potential security holes. It might also be plagued with false positives, causing more work for no added benefit. Safely running services on the internet is hard.
    - mhurron 13y ago
      > Why add more moving parts when they don't do anything but make more work? Because in this case we're talking about intentionally accepting files from users to either integrate into the system or offer to other users. Why would you not at least check files for cleanness and reject any that fail instead of blindly accepting them because you wanted to enable file uploads. Vulnerability scanning isn't going to tell you much when you want to accept files from users.
- Apes 13y ago
  Ubuntu has AppArmor, which could also do the task. The problem is that no one actually uses SELinux or AppArmor.
  - derekp7 13y ago
    For SELinux on RHEL, you have to take explicit action to disable it. Now it is probably likely that admins who haven't added any SELinux skills to their skillset only learn how to turn it off and not set permissions correctly. However, that is almost as bad as running chmod 777 on everything (which I've seen plenty of people do). The minimum that anyone admining a RHEL-derived system should learn is the chcon (change security context) command, and running the tools to diagnose SELinux issues. If you must, start off with setting SELinux to "Permissive" instead of disabling it completely. Then after a few days of running, go through your audit logs, fix any of the errors that come up, then set it back to Enforcing.
    - blueskin_ 13y ago
      >If you must, start off with setting SELinux to "Permissive" instead of disabling it completely. Then after a few days of running, go through your audit logs, fix any of the errors that come up, then set it back to Enforcing. This is the best bet for something that's in production already, but ideally, you want to have SELinux set to enforcing in testing environments and create the policies there in the first place.
- blueskin_ 13y ago
  CentOS/RHEL guy here, and I have to say, SELinux FTW. So many times I've been frustrated and annoyed when people's first reaction is 'disable it' when they hit some problem (almost always due to using stuff in a wrong or nonstandard way)* instead of spending 5 min to get it working and having far more security. * (That, and custom programs that use perversely wrong paths, e.g. web content in /home, logs in the application directory, etc.)
dclara 13y ago
I totally agree with you. Security is the no. 1 concern before we start a public launch. The issues you mentioned is very basic that should be avoided in the first place. VPS should be well maintained by the owner himself or have professional service to help. For a better architecture, a web server should be separated from the app server and database server, because the HTTP port is always open on the firewall, while the rest can only be accessed locally. I have a blog page showing step by step how to configure application development environment on Windows and deployment environment on Linux. Maybe it's helpful. http://bingobo.info/blog/contents/how-to-configure-application-development-environment-on-windows.jsp http://bingobo.info/blog/contents/how-to-configure-applicati...
MichaelGG 13y ago
>web application process had write access to a location it was able to execute code from Like many CMSes seem intent on doing?
lis 13y ago
I agree that this would be the best option, but what is the point of remote deployment APIs then? Eg my CI deploys war files on a tomcat using tomcats remote deployment API, so tomcat must be able to write in its webapp directory. Would you suggest to deploy wars always via scp and a restart of the tomcat instance instead?
- drdaeman 13y ago
  Sorry, I don't know anything about Tomcat. Maybe you could separate its deployment API and run it as a separate worker process that has elevated privileges? I really have no idea, sorry. Guess there are always exceptional cases to anything.

jlgaddis 13y ago

Yep, this can be a lesson to developers who read this. If your web application must write to files, keep them outside of the DocumentRoot! edit: I know a lot of HN'ers are fans of Ubuntu but, FWIW, SELinux (default on RHEL and derivatives) would have prevented this.

drdaeman 13y ago
Outside of any executable location, not only DocumentRoot. Latter is just an easier and more popular target. I mean, all writeable directories (if any) must not only be not runnable by some server (like Apache with mod_php), but also be on devices mounted with noexec, like this is the case with /tmp.
- jlgaddis 13y ago
  I'm not sure that'll help like you think it will. I can't verify it at the moment, but create a /tmp/phpinfo.php file with "<?php phpinfo(); ?>" in it and then run "php /tmp/phpinfo.php". Regardless of whether /tmp is mounted noexec I think you'll find that it executes (because the php binary is under /usr which almost certainly isn't mounted noexec).
  - drdaeman 13y ago
    Whoops, seems that you're right, I was wrong about interpreters taking noexec into account - either I misremembered something or things changed, but the fact is they don't. Sorry. Anyway, it's also better to have noexec than not. :) At least, it'll help if there's no complete shell access but only ability to run some binary by name, without args, or modify environment variables when running some executable.
mhurron 13y ago
BTW, is there any reason not to hook something like ClamAV into your upload function and scan files before they are saved?
- trippy_biscuits 13y ago
  Obligatory xkcd: http://xkcd.com/463/ http://xkcd.com/463/ Why add more moving parts when they don't do anything but make more work? Scanning is a helpful idea, but not AV scanning. Regular vulnerability scanning can assess the platform security. At the very least, it can warn about potential security holes. It might also be plagued with false positives, causing more work for no added benefit. Safely running services on the internet is hard.
  - mhurron 13y ago
    > Why add more moving parts when they don't do anything but make more work? Because in this case we're talking about intentionally accepting files from users to either integrate into the system or offer to other users. Why would you not at least check files for cleanness and reject any that fail instead of blindly accepting them because you wanted to enable file uploads. Vulnerability scanning isn't going to tell you much when you want to accept files from users.
Apes 13y ago
Ubuntu has AppArmor, which could also do the task. The problem is that no one actually uses SELinux or AppArmor.
- derekp7 13y ago
  For SELinux on RHEL, you have to take explicit action to disable it. Now it is probably likely that admins who haven't added any SELinux skills to their skillset only learn how to turn it off and not set permissions correctly. However, that is almost as bad as running chmod 777 on everything (which I've seen plenty of people do). The minimum that anyone admining a RHEL-derived system should learn is the chcon (change security context) command, and running the tools to diagnose SELinux issues. If you must, start off with setting SELinux to "Permissive" instead of disabling it completely. Then after a few days of running, go through your audit logs, fix any of the errors that come up, then set it back to Enforcing.
  - blueskin_ 13y ago
    >If you must, start off with setting SELinux to "Permissive" instead of disabling it completely. Then after a few days of running, go through your audit logs, fix any of the errors that come up, then set it back to Enforcing. This is the best bet for something that's in production already, but ideally, you want to have SELinux set to enforcing in testing environments and create the policies there in the first place.
blueskin_ 13y ago
CentOS/RHEL guy here, and I have to say, SELinux FTW. So many times I've been frustrated and annoyed when people's first reaction is 'disable it' when they hit some problem (almost always due to using stuff in a wrong or nonstandard way)* instead of spending 5 min to get it working and having far more security. * (That, and custom programs that use perversely wrong paths, e.g. web content in /home, logs in the application directory, etc.)

dclara 13y ago

I totally agree with you. Security is the no. 1 concern before we start a public launch. The issues you mentioned is very basic that should be avoided in the first place. VPS should be well maintained by the owner himself or have professional service to help. For a better architecture, a web server should be separated from the app server and database server, because the HTTP port is always open on the firewall, while the rest can only be accessed locally. I have a blog page showing step by step how to configure application development environment on Windows and deployment environment on Linux. Maybe it's helpful. http://bingobo.info/blog/contents/how-to-configure-application-development-environment-on-windows.jsp http://bingobo.info/blog/contents/how-to-configure-applicati...

MichaelGG 13y ago

>web application process had write access to a location it was able to execute code from Like many CMSes seem intent on doing?

lis 13y ago

I agree that this would be the best option, but what is the point of remote deployment APIs then? Eg my CI deploys war files on a tomcat using tomcats remote deployment API, so tomcat must be able to write in its webapp directory. Would you suggest to deploy wars always via scp and a restart of the tomcat instance instead?

drdaeman 13y ago
Sorry, I don't know anything about Tomcat. Maybe you could separate its deployment API and run it as a separate worker process that has elevated privileges? I really have no idea, sorry. Guess there are always exceptional cases to anything.