4 ms·

On Monday? Assume your systems are compromised and act accordingly.

by mantoto 7y ago

root_axis 7y ago
Some people choose not to work on weekends. Work/life balance etc.
- PeterisP 7y ago
  Sure, but the price of not having 24/7 support may be that instead of applying a patch you get to nuke everything from orbit and rebuild from backups.
  - root_axis 7y ago
    Sure, but just because the company didn't want to shell out the dough for 24/7 support doesn't mean that the employees should necessarily take it upon themselves to work during their off time.
    - PeterisP 7y ago
      It probably comes down to the environment for other work. If the company will "pay the price" then that's okay, but if you will "pay the price" i.e. if a need to nuke everything from orbit and rebuild from backup will simply result in a lot of unpaid overtime for you, that sucks, but in that case you might prefer to do less unpaid work in your off time today instead of more unpaid work in your off time throughout the next week.
      - root_axis 7y ago
        Well sure, if restoring from backups means you will be working unpaid overtime then it'd be worth working less overtime to stave off more, but in practice restoring from backups is a time-consuming process for computers, not for people. Realistically though, enjoying your Sunday and digging into work issues on Monday is probably not going to be a big deal.
        meowface 7y ago
        There's also the small issue that every minute that passes is another potential minute an attacker is stealing sensitive data, PII, and email/IM logs from your company's internal network, and backdooring other servers, installing ransomware, etc. That requires far more than a wipe and restore to deal with, and could potentially result in a massive financial and reputational loss.
        root_axis 7y ago
        The stream of new security threats is never-ending, thus security must be a process that incorporates the reality that employees sometimes have days when they aren't working. Clearly, the OP is not being paid for 24/7 on-call security consulting, so why should they sacrifice their day off to investigate and patch security vulnerabilities?
        arpa 7y ago
        That seems far fetched in most saneish setups of PHP. The only risk, really, is the apps' own data. Which is also where microservices shine - chaining attacks like this is exponentially more difficult then. Unless, of course, your app is a monolith, isn't sandboxed and segregated from the rest of internal network (i.e. on the same server), and the rest of the network is very, very vulnerable so the attacker can chain these exploits just right. The possibility is not that high if you're not a high profile target and if you're a high profile target, well, you should know better than to keep all of your eggs in the same basket. And if it's a shared vps where such things can actually happen, the hosting provider should take care if it.
        mantoto 7y ago
        And who has seen proper Microservicearchitektur? I haven't. Only now how it should look like. Soo you use microservices? Yes! Sooo why can I not find more then one database?
        meowface 7y ago
        I think you very greatly overestimate the typical level of isolation and "sanity" of most setups in general, let alone most PHP setups (which are likely generally much worse than most other setups).
  - arpa 7y ago
    Eh, as long as the database is intact, just nuke the container, rebuild base image and you're free.
deleted 7y ago
[deleted]
tetha 7y ago
In this case we're talking about 5 VMs running php-fpm and no nginx in sight, so these VMs aren't immediately affected by said exploit. Also, these VMs only consume some public, unauthenticated APIs of the company and render the concent into some pretty HTML. These boxes have no persistence, no access to PII and no access to anything you can't get with curl right now. Worst-Case, they can try sending some spam mails or DDoS-attacks, in which case the hoster would zero-route/force-stop them as soon as that's detected. And then I'd have to rebuild them with ~30 minutes waiting for teraform. So yes, I'm going to act accordingly. By not bothering on a sunday, because the systems are properly isolated and there are procedures in place.

On Monday? Assume your systems are compromised and act accordingly.

root_axis 7y ago

Some people choose not to work on weekends. Work/life balance etc.

PeterisP 7y ago
Sure, but the price of not having 24/7 support may be that instead of applying a patch you get to nuke everything from orbit and rebuild from backups.
- root_axis 7y ago
  Sure, but just because the company didn't want to shell out the dough for 24/7 support doesn't mean that the employees should necessarily take it upon themselves to work during their off time.
  - PeterisP 7y ago
    It probably comes down to the environment for other work. If the company will "pay the price" then that's okay, but if you will "pay the price" i.e. if a need to nuke everything from orbit and rebuild from backup will simply result in a lot of unpaid overtime for you, that sucks, but in that case you might prefer to do less unpaid work in your off time today instead of more unpaid work in your off time throughout the next week.
    - root_axis 7y ago
      Well sure, if restoring from backups means you will be working unpaid overtime then it'd be worth working less overtime to stave off more, but in practice restoring from backups is a time-consuming process for computers, not for people. Realistically though, enjoying your Sunday and digging into work issues on Monday is probably not going to be a big deal.
      - meowface 7y ago
        There's also the small issue that every minute that passes is another potential minute an attacker is stealing sensitive data, PII, and email/IM logs from your company's internal network, and backdooring other servers, installing ransomware, etc. That requires far more than a wipe and restore to deal with, and could potentially result in a massive financial and reputational loss.
        root_axis 7y ago
        The stream of new security threats is never-ending, thus security must be a process that incorporates the reality that employees sometimes have days when they aren't working. Clearly, the OP is not being paid for 24/7 on-call security consulting, so why should they sacrifice their day off to investigate and patch security vulnerabilities?
        arpa 7y ago
        That seems far fetched in most saneish setups of PHP. The only risk, really, is the apps' own data. Which is also where microservices shine - chaining attacks like this is exponentially more difficult then. Unless, of course, your app is a monolith, isn't sandboxed and segregated from the rest of internal network (i.e. on the same server), and the rest of the network is very, very vulnerable so the attacker can chain these exploits just right. The possibility is not that high if you're not a high profile target and if you're a high profile target, well, you should know better than to keep all of your eggs in the same basket. And if it's a shared vps where such things can actually happen, the hosting provider should take care if it.
        mantoto 7y ago
        And who has seen proper Microservicearchitektur? I haven't. Only now how it should look like. Soo you use microservices? Yes! Sooo why can I not find more then one database?
        meowface 7y ago
        I think you very greatly overestimate the typical level of isolation and "sanity" of most setups in general, let alone most PHP setups (which are likely generally much worse than most other setups).
- arpa 7y ago
  Eh, as long as the database is intact, just nuke the container, rebuild base image and you're free.

deleted 7y ago

[deleted]

tetha 7y ago

In this case we're talking about 5 VMs running php-fpm and no nginx in sight, so these VMs aren't immediately affected by said exploit. Also, these VMs only consume some public, unauthenticated APIs of the company and render the concent into some pretty HTML. These boxes have no persistence, no access to PII and no access to anything you can't get with curl right now. Worst-Case, they can try sending some spam mails or DDoS-attacks, in which case the hoster would zero-route/force-stop them as soon as that's detected. And then I'd have to rebuild them with ~30 minutes waiting for teraform. So yes, I'm going to act accordingly. By not bothering on a sunday, because the systems are properly isolated and there are procedures in place.