4 ms·

> I don’t operate production systems, but I have helped to design a couple of them. So I understand something about the assumptions you make when building them.

> I don’t operate production systems, but I have helped to design a couple of them. So I understand something about the assumptions you make when building them. Start by operating production systems. You will rapidly discover that patching is not a technical issue (as I already said last time there was a patching discussion on HN). Patching is technically "easy". But if the business does not prioritize (and allocate time towards) patching (and testing that systems work with after) patching won't get done. There are features to develop and deadlines to meet and new systems to turn on and old systems to retire and oh-my-god don't touch those servers otherwise the vendor will not support the prod environment anymore (no longer certified, yay!) and it costs us 30000$ to get certified again. Too many times I see HN assuming that everyone is running on a public cloud with B/G deployments and immutable infrastructure you can rebuild and redeploy easily. Unfortunately that's not the case.

sqldba 9y ago
Those "don't touch" systems are death. It's usually in contracts with the big players like Honeywell and others. You literally can't touch them without breaking the contract. The second you call for support and they find a patch - you have to uninstall before they continue - even if it's unrelated. BTW hands up if that includes WannnaCrypt. Yep.
- alkonaut 9y ago
  This is the same situation as those MRI machines with XP on them. The answer is don't buy those machines or don't connect machines you can't patch to the network. If hospitals can't buy them because those are the criteria, then industry will change. Don't sign contracts that leaves you in the situation that you can't patch machines.
  - ams6110 9y ago
    If a hospital doesn't have an MRI machine, it won't be able to compete. It's really not an option to say "I won't buy it unless X." You buy the one that meets most of your needs for the best price. Not buying is not really a choice.
Spooky23 9y ago
I am responsible for about 200,000 production systems, about 90/10 client to server. Delivery is not difficult. My teams can deliver anything to 95% of those systems in <24 hours. The hard part is testing, validation and coordinating rollouts. It's also more effective to understand and have mitigating controls. Many java related exploits cannot be patched with off the shelf software -- you need to wait for the vendor. So you to segment or monitor to live at risk. Companies who are good at this have good asset management, good process and good vulnerability monitoring.
- dec0dedab0de 9y ago
  Good fonts?
  - Spooky23 9y ago
    Thanks for catching that quickly! I don't remember precisely what word i intended to write... but autocorrect butchered it! I suppose adding "quality control" to my original post would be appropriate as well! :)
xorcist 9y ago
> with B/G deployments Why do we call it that now, instead of A/B deployment like we used to?
- _asummers 9y ago
  To me, blue/green is for infrastructure, and A/B is for the user side of it. That is, you should be able to do blue/green with the content not changing, but when you're A/B testing, you're collecting metrics and other usage data about the users to make future product decisions. You can also do A/B testing without having a B/G setup, using things like user gates. This distinction may be incorrect, but that's how I've used them.
- nine_k 9y ago
  A/B implies certain symmetry, while B/G is highly asymmetrical: G is known good, battle-tested, and B is new and suspicious.

sqldba 9y ago

Those "don't touch" systems are death. It's usually in contracts with the big players like Honeywell and others. You literally can't touch them without breaking the contract. The second you call for support and they find a patch - you have to uninstall before they continue - even if it's unrelated. BTW hands up if that includes WannnaCrypt. Yep.

alkonaut 9y ago
This is the same situation as those MRI machines with XP on them. The answer is don't buy those machines or don't connect machines you can't patch to the network. If hospitals can't buy them because those are the criteria, then industry will change. Don't sign contracts that leaves you in the situation that you can't patch machines.
- ams6110 9y ago
  If a hospital doesn't have an MRI machine, it won't be able to compete. It's really not an option to say "I won't buy it unless X." You buy the one that meets most of your needs for the best price. Not buying is not really a choice.

Spooky23 9y ago

I am responsible for about 200,000 production systems, about 90/10 client to server. Delivery is not difficult. My teams can deliver anything to 95% of those systems in <24 hours. The hard part is testing, validation and coordinating rollouts. It's also more effective to understand and have mitigating controls. Many java related exploits cannot be patched with off the shelf software -- you need to wait for the vendor. So you to segment or monitor to live at risk. Companies who are good at this have good asset management, good process and good vulnerability monitoring.

dec0dedab0de 9y ago
Good fonts?
- Spooky23 9y ago
  Thanks for catching that quickly! I don't remember precisely what word i intended to write... but autocorrect butchered it! I suppose adding "quality control" to my original post would be appropriate as well! :)

xorcist 9y ago

> with B/G deployments Why do we call it that now, instead of A/B deployment like we used to?

_asummers 9y ago
To me, blue/green is for infrastructure, and A/B is for the user side of it. That is, you should be able to do blue/green with the content not changing, but when you're A/B testing, you're collecting metrics and other usage data about the users to make future product decisions. You can also do A/B testing without having a B/G setup, using things like user gates. This distinction may be incorrect, but that's how I've used them.
nine_k 9y ago
A/B implies certain symmetry, while B/G is highly asymmetrical: G is known good, battle-tested, and B is new and suspicious.