8 ms·

In the meantime, your PHP app had 40 major security flaws, no meaningful monitoring, a DB that wasn't backed up and major data consistency problems. Also, when

In the meantime, your PHP app had 40 major security flaws, no meaningful monitoring, a DB that wasn't backed up and major data consistency problems. Also, when your box's hdd crashed, you lost all those minor changes you had vim'ed over the years. But for the rest, yeah, all is fine.

shocks 3y ago
If you forget about a PHP container for a few years it will /also/ have 40 new vulnerabilities. Actually, containers are worse because OS updates of core shared libraries do nothing. You have to rebuild every damn container. Setting up monitoring for your docker containers is also a whole thing. :) I think you’re taking my example a little too literally. My point is not that docker/k8s/whatever is bad; just that the ‘new’ adds features at the cost of simplicity.
- makeitdouble 3y ago
  The biggest thing with containers is, these 40 vulnerabilities won't matter as much if they're about erasing your directories or killing your machine. It will get rebooted in a pristine state and an attacker would need to stick there killing it at every reboot to have lasting effect. Which is also the point of monitoring a container, which is fairly reliable nowadays, compared to managing your own health check service to ping your box to check if it's alive. All in all, I think k8s is way too complicated for what it does for most people, but administrating servers was also a complex task to begin with, and the things sys admins were dealing with looked nightmarish to me. Heck, there was a time companies would have their own SMTP in house...
  - dumpsterdiver 3y ago
    > won't matter as much if they're about erasing your directories or killing your machine Those sort of attacks have tapered off though, right? Unless you're engaged in a specific feud, or have caught the ire of a social justice warrior, these days we're mostly looking at data exfiltration as the primary goal. That being said, there are a lot of ongoing feuds and active social justice warriors.
    - dns_snek 3y ago
      If we're talking about common, automated attacks, I think the game plan for the past few years has been to probe for systems vulnerable to RCE, install a crypto miner, join the server to a botnet and see how long it takes its owner to notice.
      - dumpsterdiver 3y ago
        Aha, how could I forget the crypto mining botnets. I agree.
  - BirAdam 3y ago
    Layering more abstractions on top of what was already complicated doesn’t fix the complications below, it just hides them. You then have to hope that there’s not a bug somewhere in between those layers. Additionally, each new layer has a cost in performance. More layers means more compute, and eventually mean more money. If you’re Netflix, it’s worth it, if you’re not… what admins dealt with before wasn’t so bad. Now that that job has largely been killed and renamed 20 times, it’s far worse. Admins don’t just have to deal with a server, they have to deal with a server and 60 containers which are just little baby servers.
- TeMPOraL 3y ago
  I think what people are missing is that sure, code "rots", at the very least because of security patches. But since this all happens to everything simultaneously, the more distinct layers and support tools you have in your stack, the more often you have to deal with something breaking in a nontrivial way. In short: the more moving parts you have, the less time you have between major malfunctions. (Manufacturing and hardware world understands it well, which is a big part in why they like integrating things so much.) So e.g. over in the backend-land where I live, it used to be that I had to occasionally update the compiler or one of the few third-party dependencies that I used. Today, I have many more libraries (to the point there's something to update for security reasons roughly once a month, on average), and on top of that, I have CI/CD introducing its own mess, Conan updates which occasionally get messed up, or make some existing recipes incompatible, CMake updates which are done unexpectedly and break stuff, now also Docker is adding more of its own problems, etc. So I have to deal with some kind of tooling breakage every other week now. And always, always, when I think it's all finally OK and I can get on with my actual job, some forgotten or hidden component craps itself out of the blue. Like that time our git precommit hooks broke for me, because someone changed them in a way that doesn't work with my setup. And then me wasting a day on trying to fix it, eventually giving up and degrading my setup to unblock myself. Or another day where, for no apparent reason, some automation that made automated commits to some git repos started losing Change-ID headers in commit messages, making Gerrit very sad, leading to several people wasting a total of several person-days trying to fix it. Etc. There's always something breaking, the frequency of such breakages seems to be increasing, and it's a major source of frustration for me in this job. Which is why I too am often thinking back to "good old days", and am increasingly in favor of keeping the amount of dependencies - both libraries and tooling - to a minimum.
  - shocks 3y ago
    Yea, it’s definitely about trading features for simplicity.
    - TeMPOraL 3y ago
      Right, and it's easy to think about all the winnings when things go right - it's harder to think about increased frequency and cost of failure due to increase in complexity and number of independently moving parts.
- mrweasel 3y ago
  > You have to rebuild every damn container. I can say with great certainty: Almost no one rebuilds their damn container anywhere near as often as the the gray-beard in the basement updates the Debian packages on the the server that runs the container. I had the debate with a client, we did monthly security update, unless something horrible happened. The client was rather upset that we didn't patch more frequently, like weekly or daily. My argumentation is that it doesn't really matter if the Linux kernel or bash is patched, when the only thing running is a container with a beta version of Tomcat that hasn't had security updates applied in three years. Even worse are the people who just pull things from Docker Hub, with no plan as to how and when they'll pull newer versions. But fine, let's just keep running KeyCloak from 2017, and that old Postgresql image which the developer never configured to do backups, I'm sure it's fine.
  - mrighele 3y ago
    > I can say with great certainty: Almost no one rebuilds their damn container anywhere near as often as the the gray-beard in the basement updates the Debian packages on the the server that runs the container. Most probably the gray-beard simply enabled "unattended-upgrades". You can do something similar with a container (track the security fixes for the packages used, force rebuild and deploy when needed), but it is a bit of work and I don't know of any ready solution.
xorcist 3y ago
> In the meantime, your PHP app had 40 major security flaws, no meaningful monitoring, How does Kubernetes, microservices and front end frameworks fix that? They don't. In fact, as someone who works at the more modern end of these things, I'd suspect that "no meaningful monitoring" probably correlates quite well with cloud era microservices thingies. But we can't blame the tools, that space is just younger and by nature contains more immature stuff. I don't think the argument here is for keeping things out of version control and monitoring. It's that you can actually leave stuff alone for several years, come back and find it working just as before, when the layers underneath doesn't constantly change. That can be liberating.
- CoastalCoder 3y ago
  > But we can't blame the tools, that space is just younger and by nature contains more immature stuff. I think the article's point was that the industry is perpetually young and immature because it keeps (unnecessarily) chasing fads.
- Quarrelsome 3y ago
  > How does Kubernetes, microservices and front end frameworks fix that? Doesn't being in a container mean its much easier to rebuild it from zero and thus allowing security updates to be more regular?
  - otabdeveloper4 3y ago
    Not really. A "container" is usually just an instance of a Linux distribution, and the work involved in updating it is roughly the same.
    - edejong 3y ago
      In the 27 minutes after you wrote your comment, our CI/CD rebuild two containers from a latest snapshot. No manual work involved.
      - 3y ago
mrighele 3y ago
> In the meantime, your PHP app had 40 major security flaws I just did a "npx create-react-app text" and I got "74 vulnerabilities (69 moderate, 5 high)" 40 major security flaws in 10 years sounds almost like a bargain.
- Quarrelsome 3y ago
  now read those flaws, compare them to the worst in 2004 and tell me they're the same. They're not the same because the internet was insecure as fuck in 2004 and these days security researchers motivations to receive bounties result in considerably more situational (and significantly less severe) security issues.
- papruapap 3y ago
  create-react-app has been discouraged to use for a couple of years now, not really sure if the React team is still supporting it.
  - firecall 3y ago
    April 2022 was the last maintenance release. Out of curiosity, why is it discouraged? I’m not really in the React world, but have been fixing up a Create React App for a client recently, so learned a bit more about it.
    - rerdavies 3y ago
      So how are you supposed to create static react app now?
      - xdennis 3y ago
        They no longer want you to create a React app by itself[1], but instead use Next.js, Remix, Gatsby, or Expo. [1]: https://react.dev/learn/start-a-new-react-project https://react.dev/learn/start-a-new-react-project
        3y ago
commandlinefan 3y ago
I think the takeaway here is: programming is hard, and we deserve more money.
- xdennis 3y ago
  The sad part is that much of what makes programming hard is the fault of programmers (primarily the endless churn, but others too).
datavirtue 3y ago
Shiney new app has no monitoring, 40 security flaws, and a ultra-modern cloud database that promotes data consistency issues. But all the devs you can hire act like they might know something about this mess. They really don't.

shocks 3y ago

If you forget about a PHP container for a few years it will /also/ have 40 new vulnerabilities. Actually, containers are worse because OS updates of core shared libraries do nothing. You have to rebuild every damn container. Setting up monitoring for your docker containers is also a whole thing. :) I think you’re taking my example a little too literally. My point is not that docker/k8s/whatever is bad; just that the ‘new’ adds features at the cost of simplicity.

makeitdouble 3y ago
The biggest thing with containers is, these 40 vulnerabilities won't matter as much if they're about erasing your directories or killing your machine. It will get rebooted in a pristine state and an attacker would need to stick there killing it at every reboot to have lasting effect. Which is also the point of monitoring a container, which is fairly reliable nowadays, compared to managing your own health check service to ping your box to check if it's alive. All in all, I think k8s is way too complicated for what it does for most people, but administrating servers was also a complex task to begin with, and the things sys admins were dealing with looked nightmarish to me. Heck, there was a time companies would have their own SMTP in house...
- dumpsterdiver 3y ago
  > won't matter as much if they're about erasing your directories or killing your machine Those sort of attacks have tapered off though, right? Unless you're engaged in a specific feud, or have caught the ire of a social justice warrior, these days we're mostly looking at data exfiltration as the primary goal. That being said, there are a lot of ongoing feuds and active social justice warriors.
  - dns_snek 3y ago
    If we're talking about common, automated attacks, I think the game plan for the past few years has been to probe for systems vulnerable to RCE, install a crypto miner, join the server to a botnet and see how long it takes its owner to notice.
    - dumpsterdiver 3y ago
      Aha, how could I forget the crypto mining botnets. I agree.
- BirAdam 3y ago
  Layering more abstractions on top of what was already complicated doesn’t fix the complications below, it just hides them. You then have to hope that there’s not a bug somewhere in between those layers. Additionally, each new layer has a cost in performance. More layers means more compute, and eventually mean more money. If you’re Netflix, it’s worth it, if you’re not… what admins dealt with before wasn’t so bad. Now that that job has largely been killed and renamed 20 times, it’s far worse. Admins don’t just have to deal with a server, they have to deal with a server and 60 containers which are just little baby servers.
TeMPOraL 3y ago
I think what people are missing is that sure, code "rots", at the very least because of security patches. But since this all happens to everything simultaneously, the more distinct layers and support tools you have in your stack, the more often you have to deal with something breaking in a nontrivial way. In short: the more moving parts you have, the less time you have between major malfunctions. (Manufacturing and hardware world understands it well, which is a big part in why they like integrating things so much.) So e.g. over in the backend-land where I live, it used to be that I had to occasionally update the compiler or one of the few third-party dependencies that I used. Today, I have many more libraries (to the point there's something to update for security reasons roughly once a month, on average), and on top of that, I have CI/CD introducing its own mess, Conan updates which occasionally get messed up, or make some existing recipes incompatible, CMake updates which are done unexpectedly and break stuff, now also Docker is adding more of its own problems, etc. So I have to deal with some kind of tooling breakage every other week now. And always, always, when I think it's all finally OK and I can get on with my actual job, some forgotten or hidden component craps itself out of the blue. Like that time our git precommit hooks broke for me, because someone changed them in a way that doesn't work with my setup. And then me wasting a day on trying to fix it, eventually giving up and degrading my setup to unblock myself. Or another day where, for no apparent reason, some automation that made automated commits to some git repos started losing Change-ID headers in commit messages, making Gerrit very sad, leading to several people wasting a total of several person-days trying to fix it. Etc. There's always something breaking, the frequency of such breakages seems to be increasing, and it's a major source of frustration for me in this job. Which is why I too am often thinking back to "good old days", and am increasingly in favor of keeping the amount of dependencies - both libraries and tooling - to a minimum.
- shocks 3y ago
  Yea, it’s definitely about trading features for simplicity.
  - TeMPOraL 3y ago
    Right, and it's easy to think about all the winnings when things go right - it's harder to think about increased frequency and cost of failure due to increase in complexity and number of independently moving parts.
mrweasel 3y ago
> You have to rebuild every damn container. I can say with great certainty: Almost no one rebuilds their damn container anywhere near as often as the the gray-beard in the basement updates the Debian packages on the the server that runs the container. I had the debate with a client, we did monthly security update, unless something horrible happened. The client was rather upset that we didn't patch more frequently, like weekly or daily. My argumentation is that it doesn't really matter if the Linux kernel or bash is patched, when the only thing running is a container with a beta version of Tomcat that hasn't had security updates applied in three years. Even worse are the people who just pull things from Docker Hub, with no plan as to how and when they'll pull newer versions. But fine, let's just keep running KeyCloak from 2017, and that old Postgresql image which the developer never configured to do backups, I'm sure it's fine.
- mrighele 3y ago
  > I can say with great certainty: Almost no one rebuilds their damn container anywhere near as often as the the gray-beard in the basement updates the Debian packages on the the server that runs the container. Most probably the gray-beard simply enabled "unattended-upgrades". You can do something similar with a container (track the security fixes for the packages used, force rebuild and deploy when needed), but it is a bit of work and I don't know of any ready solution.

xorcist 3y ago

> In the meantime, your PHP app had 40 major security flaws, no meaningful monitoring, How does Kubernetes, microservices and front end frameworks fix that? They don't. In fact, as someone who works at the more modern end of these things, I'd suspect that "no meaningful monitoring" probably correlates quite well with cloud era microservices thingies. But we can't blame the tools, that space is just younger and by nature contains more immature stuff. I don't think the argument here is for keeping things out of version control and monitoring. It's that you can actually leave stuff alone for several years, come back and find it working just as before, when the layers underneath doesn't constantly change. That can be liberating.

CoastalCoder 3y ago
> But we can't blame the tools, that space is just younger and by nature contains more immature stuff. I think the article's point was that the industry is perpetually young and immature because it keeps (unnecessarily) chasing fads.
Quarrelsome 3y ago
> How does Kubernetes, microservices and front end frameworks fix that? Doesn't being in a container mean its much easier to rebuild it from zero and thus allowing security updates to be more regular?
- otabdeveloper4 3y ago
  Not really. A "container" is usually just an instance of a Linux distribution, and the work involved in updating it is roughly the same.
  - edejong 3y ago
    In the 27 minutes after you wrote your comment, our CI/CD rebuild two containers from a latest snapshot. No manual work involved.
    - 3y ago

mrighele 3y ago

> In the meantime, your PHP app had 40 major security flaws I just did a "npx create-react-app text" and I got "74 vulnerabilities (69 moderate, 5 high)" 40 major security flaws in 10 years sounds almost like a bargain.

Quarrelsome 3y ago
now read those flaws, compare them to the worst in 2004 and tell me they're the same. They're not the same because the internet was insecure as fuck in 2004 and these days security researchers motivations to receive bounties result in considerably more situational (and significantly less severe) security issues.
papruapap 3y ago
create-react-app has been discouraged to use for a couple of years now, not really sure if the React team is still supporting it.
- firecall 3y ago
  April 2022 was the last maintenance release. Out of curiosity, why is it discouraged? I’m not really in the React world, but have been fixing up a Create React App for a client recently, so learned a bit more about it.
  - rerdavies 3y ago
    So how are you supposed to create static react app now?
    - xdennis 3y ago
      They no longer want you to create a React app by itself[1], but instead use Next.js, Remix, Gatsby, or Expo. [1]: https://react.dev/learn/start-a-new-react-project https://react.dev/learn/start-a-new-react-project
      - 3y ago

commandlinefan 3y ago

I think the takeaway here is: programming is hard, and we deserve more money.

xdennis 3y ago
The sad part is that much of what makes programming hard is the fault of programmers (primarily the endless churn, but others too).

datavirtue 3y ago

Shiney new app has no monitoring, 40 security flaws, and a ultra-modern cloud database that promotes data consistency issues. But all the devs you can hire act like they might know something about this mess. They really don't.