4 ms·

Also introduces that same time delay for getting security patches released by package maintainers into your build pipeline.

yardstick 5y ago
Not true. We assess each update in case of high priority issues that need a quick update. Also we are cautious of using dependencies that don’t provide long term support/back fixes - where possible I pick stable, responsibly-managed dependencies. Postgres is a great example, security fixes are applied across multiple major versions, not just the latest.
- rexelhoff 5y ago
  Sounds exactly like what we do for medical device software. Version pinning, self-managed forks and code reviews of the dependency on upgrade.
- geofft 5y ago
  How do you get informed about whether a high-priority issue exists? In particular, who is auditing the old version of the code that you happen to be running to make sure it doesn't have vulnerabilities that are now gone after a non-security-motivated change like a refactoring or a feature removal? Probably not the upstream maintainers, who generally only maintain HEAD.
  - zomgwat 5y ago
    Security mailing lists. GitHub has security alerts. Good old fashioned RSS. For example, rubygems.org supports RSS for releases. GitHub Release pages also support RSS. An easy way to create a shared RSS feeds is to create a #news channel in Slack that has all the feeds. It can be a pain but that pain might be motivation to not pull in dependencies with little thought.
    - geofft 5y ago
      Again, how does any of that tell you whether a high-priority issue exists in the old version of the code that you're running, as opposed to in the latest release?
      - zomgwat 5y ago
        I understand your point. I'd expect the old version to have been reviewed when it was introduced into the system just as the new version should be. Of course, that doesn't guarantee something won't slip in. Running private package infrastructure with audited dependencies isn't a panacea to stopping supply chain attacks. I do believe it's an effective defense-in-depth tactic for the reasons others have discussed. An additional supporting tactic that should be done is to tightly control egress traffic. Like ingress traffic, all egress traffic should be denied by default. From there, traffic should be whitelisted. That makes it more difficult to exfiltrate data or communicate with command and control infrastructure. Tight control on egress traffic also makes it easier to alert on unexpected connection attempts. That all said, locking down egress traffic can be a pain. It also isn't a panacea. Where there’s a will there’s a way.
tracker1 5y ago
This... worked in an org, where it literally took nearly 3 months to get package updates and additions approved... and that was for licensing review, not even proper security audits. It may be safer to just accept that you live in the wild west and have systems boundaries in place to limit exposure/impact. I've been pushing for closed-box logging targets, for example, where all logging goes to at least two systems (local and remote) so that the logging target is generally only available to submit log lines entries to. Not perfect, but where my head has been at. Another thing is to PULL backups, not push them out.
- jacquesm 5y ago
  > Another thing is to PULL backups, not push them out. That's an absolute must. A push backup may not be a backup at all when you need it, or it may be compromised. It also requires the system you push into to be accessible for inbound connections, which in itself may be problematic.
  - andrewchambers 5y ago
    Really once the secure channel between two processes running on each host is setup, there is no difference between the two methods. A pull backup can be tricked just as easily by presenting a false filesystem or backdooring the 'pull process' running on the compromised machine. You want append only backups with secure timestamps and do forensics to find the last good snapshot. It is easier to set that up with pull backups, but not so hard with the right tools for push backups.
  - zomgwat 5y ago
    Another option is to use a cloud storage service that supports fine grained access control (e.g. write-only vs read-only) and retention polices.
- beefjerkins 5y ago
  I'm trying to wrap my head around the concept of 'pulling' backups, rather than pushing them. In my mind, once you make a backup, you should then transfer it to a separate system for archival. Where am I going wrong?

yardstick 5y ago

Not true. We assess each update in case of high priority issues that need a quick update. Also we are cautious of using dependencies that don’t provide long term support/back fixes - where possible I pick stable, responsibly-managed dependencies. Postgres is a great example, security fixes are applied across multiple major versions, not just the latest.

rexelhoff 5y ago
Sounds exactly like what we do for medical device software. Version pinning, self-managed forks and code reviews of the dependency on upgrade.
geofft 5y ago
How do you get informed about whether a high-priority issue exists? In particular, who is auditing the old version of the code that you happen to be running to make sure it doesn't have vulnerabilities that are now gone after a non-security-motivated change like a refactoring or a feature removal? Probably not the upstream maintainers, who generally only maintain HEAD.
- zomgwat 5y ago
  Security mailing lists. GitHub has security alerts. Good old fashioned RSS. For example, rubygems.org supports RSS for releases. GitHub Release pages also support RSS. An easy way to create a shared RSS feeds is to create a #news channel in Slack that has all the feeds. It can be a pain but that pain might be motivation to not pull in dependencies with little thought.
  - geofft 5y ago
    Again, how does any of that tell you whether a high-priority issue exists in the old version of the code that you're running, as opposed to in the latest release?
    - zomgwat 5y ago
      I understand your point. I'd expect the old version to have been reviewed when it was introduced into the system just as the new version should be. Of course, that doesn't guarantee something won't slip in. Running private package infrastructure with audited dependencies isn't a panacea to stopping supply chain attacks. I do believe it's an effective defense-in-depth tactic for the reasons others have discussed. An additional supporting tactic that should be done is to tightly control egress traffic. Like ingress traffic, all egress traffic should be denied by default. From there, traffic should be whitelisted. That makes it more difficult to exfiltrate data or communicate with command and control infrastructure. Tight control on egress traffic also makes it easier to alert on unexpected connection attempts. That all said, locking down egress traffic can be a pain. It also isn't a panacea. Where there’s a will there’s a way.

tracker1 5y ago

This... worked in an org, where it literally took nearly 3 months to get package updates and additions approved... and that was for licensing review, not even proper security audits. It may be safer to just accept that you live in the wild west and have systems boundaries in place to limit exposure/impact. I've been pushing for closed-box logging targets, for example, where all logging goes to at least two systems (local and remote) so that the logging target is generally only available to submit log lines entries to. Not perfect, but where my head has been at. Another thing is to PULL backups, not push them out.

jacquesm 5y ago
> Another thing is to PULL backups, not push them out. That's an absolute must. A push backup may not be a backup at all when you need it, or it may be compromised. It also requires the system you push into to be accessible for inbound connections, which in itself may be problematic.
- andrewchambers 5y ago
  Really once the secure channel between two processes running on each host is setup, there is no difference between the two methods. A pull backup can be tricked just as easily by presenting a false filesystem or backdooring the 'pull process' running on the compromised machine. You want append only backups with secure timestamps and do forensics to find the last good snapshot. It is easier to set that up with pull backups, but not so hard with the right tools for push backups.
- zomgwat 5y ago
  Another option is to use a cloud storage service that supports fine grained access control (e.g. write-only vs read-only) and retention polices.
beefjerkins 5y ago
I'm trying to wrap my head around the concept of 'pulling' backups, rather than pushing them. In my mind, once you make a backup, you should then transfer it to a separate system for archival. Where am I going wrong?