5 ms·

Not so much snarky as a commentary on the diligence. Yes, if there is mission critical software that affects millions of people, do all that stuff. If your pd

Not so much snarky as a commentary on the diligence. Yes, if there is mission critical software that affects millions of people, do all that stuff. If your pdf -> stardust service is more a convenience then don't spend hundreds of thousands on infrastructure.

KronisLV 5y ago
> ...then don't spend hundreds of thousands on infrastructure. I find it curious how we went from doing the basics of software development that would minimize risks and be helpful to almost any project out there to this. To clarify, i agree with the point that you'll need to prioritize different components based on what matters the most, but i don't think that you can't have a common standard set of tools and practices for all of them. Let me address all of the points with examples. > Auth: nobody knows IP address of our server anyway, don't bother with that. And for extra security we have secret port number. Port scanning means that none of your ports are secret. JWT is trivial to implement in most languages. Even basic auth is better than nothing with HTTPS, you don't always need mTLS or the more complicated solutions, but you need something. This should take a few days to a week to implement. Edit: probably a day or less if you have an easily reusable library for this. > Monitoring? Well, we have our clients for that. They'll call us if something happens. This is not viable if you have SLAs or just enjoy sleeping and not getting paged. There are free monitoring solutions out there, such as Zabbix, Nagios, Prometheus & Grafana and others. Preconfigured OS templates also mean that you just need the monitoring appliance and an agent on the node you want to monitor in most cases. This should take close to a week to implement. Edit: probably an hour or less if you already have a server up and just need to add a node. > Deploy? Copy .war file and restart tomcat. Rollback? What's rollback? This is an error prone way to do things, as the experience of Knight Capital showed: https://dougseven.com/2014/04/17/knightmare-a-devops-cautionary-tale https://dougseven.com/2014/04/17/knightmare-a-devops-caution.... In addition, manual configuration changes lead to configuration drift and after a few years you'll have little idea about who changed what and when. In contrast, setting up Ansible and versioning your config, as well as using containers for the actual software releases alongside fully automated CI cycles addresses all of those problems. In regards to rollbacks, if you have automated DB migrations, you might have to spend some time writing reverse migrations for all of the DDL changes. This should take between one to two weeks to implement. Edit: probably a day or less once per project with small fixes here and there. > Outdated packages? We call those stable packages. Log4j might be stable, but it also leads to RCEs. This is not a good argument, at least for as long as the software packages that we use are beyond our ability to control or comprehend. This should be a regular automated process, that alerts you about outdated and/or insecure packages, at least use npm audit or something, or proactive scanning like OpenVAS. This should take close to a week to implement. All of the mentioned software can easily run on a single node with 2 CPU cores and about 8 GB of RAM. I know this, because i did all of the above in a project of mine (mentioned technologies might have been changed, though). Of course, doing that over N projects will probably increase the total time, especially if they have been badly written. In my eyes that's a worthwhile investment, since when you finally have that across all of your apps, you can develop with better certainty that you won't have to stay late and deliver 2 or 3 hotfixes after your release, which goes hand in hand with test coverage and automated testing. That is hardly hundreds of thousands of dollars/euros/whatever on infrastructure. And if the personnel costs are like that, i'd like to work over there, then.
- alisonkisk 5y ago
  I am opposed to sarcasm on the internet, but even I saw the obvious satire of the GP post.
- madsbuch 5y ago
  There are plenty of solutions to authentication. But really, don't implement a user system if it is not needed. There are plenty of other ways to secure on applications, which are way out of scope for this discussion. The main point is, that one should never spend a "a few days to a week" to implement a feature that at best i useless and at worst is detrimental to the service stood up. Implement auth, if it is needed, implement monitoring, CI, CD, dependency monitoring, testing, everything, if it is needed. But don't implement it as dogmatic consequences of doing software development. And regarding the spend: one week worth of work could be USD 8k. So just the initial implementation of your JWT based authentication system is 4% into the "hundreds of thousands of dollars". Then you need to factor in the extra complexity on maintenance and before you know it we do not talk about hundreds of thousands of dollars but millions...
  - hiptobecubic 5y ago
    I feel like "spaghetti garbage now, we'll fix it later" is a big part of why startups fail to execute well. Yeah you saved $8000 by launching your thing in a completely unmaintainable way, but now it's both harder to maintain and more likely to need it. Literally the first time it breaks you will probably lose that cost advantage just because it takes so long to debug. The point you should have made is that dogmatic approaches usually produce a lot of waste, but the example you gave us exactly why teams end up that way. Otherwise people come up with bullshit hacks like you describe and the entire team pays for it.
    - madsbuch 5y ago
      In that case it would be bad to b cut corners, ey? Though these were not the cases I talked about. I talked about implementing features that are not necessary for the product because of dogmatic reasons.

KronisLV 5y ago

> ...then don't spend hundreds of thousands on infrastructure. I find it curious how we went from doing the basics of software development that would minimize risks and be helpful to almost any project out there to this. To clarify, i agree with the point that you'll need to prioritize different components based on what matters the most, but i don't think that you can't have a common standard set of tools and practices for all of them. Let me address all of the points with examples. > Auth: nobody knows IP address of our server anyway, don't bother with that. And for extra security we have secret port number. Port scanning means that none of your ports are secret. JWT is trivial to implement in most languages. Even basic auth is better than nothing with HTTPS, you don't always need mTLS or the more complicated solutions, but you need something. This should take a few days to a week to implement. Edit: probably a day or less if you have an easily reusable library for this. > Monitoring? Well, we have our clients for that. They'll call us if something happens. This is not viable if you have SLAs or just enjoy sleeping and not getting paged. There are free monitoring solutions out there, such as Zabbix, Nagios, Prometheus & Grafana and others. Preconfigured OS templates also mean that you just need the monitoring appliance and an agent on the node you want to monitor in most cases. This should take close to a week to implement. Edit: probably an hour or less if you already have a server up and just need to add a node. > Deploy? Copy .war file and restart tomcat. Rollback? What's rollback? This is an error prone way to do things, as the experience of Knight Capital showed: https://dougseven.com/2014/04/17/knightmare-a-devops-cautionary-tale https://dougseven.com/2014/04/17/knightmare-a-devops-caution.... In addition, manual configuration changes lead to configuration drift and after a few years you'll have little idea about who changed what and when. In contrast, setting up Ansible and versioning your config, as well as using containers for the actual software releases alongside fully automated CI cycles addresses all of those problems. In regards to rollbacks, if you have automated DB migrations, you might have to spend some time writing reverse migrations for all of the DDL changes. This should take between one to two weeks to implement. Edit: probably a day or less once per project with small fixes here and there. > Outdated packages? We call those stable packages. Log4j might be stable, but it also leads to RCEs. This is not a good argument, at least for as long as the software packages that we use are beyond our ability to control or comprehend. This should be a regular automated process, that alerts you about outdated and/or insecure packages, at least use npm audit or something, or proactive scanning like OpenVAS. This should take close to a week to implement. All of the mentioned software can easily run on a single node with 2 CPU cores and about 8 GB of RAM. I know this, because i did all of the above in a project of mine (mentioned technologies might have been changed, though). Of course, doing that over N projects will probably increase the total time, especially if they have been badly written. In my eyes that's a worthwhile investment, since when you finally have that across all of your apps, you can develop with better certainty that you won't have to stay late and deliver 2 or 3 hotfixes after your release, which goes hand in hand with test coverage and automated testing. That is hardly hundreds of thousands of dollars/euros/whatever on infrastructure. And if the personnel costs are like that, i'd like to work over there, then.

alisonkisk 5y ago
I am opposed to sarcasm on the internet, but even I saw the obvious satire of the GP post.
madsbuch 5y ago
There are plenty of solutions to authentication. But really, don't implement a user system if it is not needed. There are plenty of other ways to secure on applications, which are way out of scope for this discussion. The main point is, that one should never spend a "a few days to a week" to implement a feature that at best i useless and at worst is detrimental to the service stood up. Implement auth, if it is needed, implement monitoring, CI, CD, dependency monitoring, testing, everything, if it is needed. But don't implement it as dogmatic consequences of doing software development. And regarding the spend: one week worth of work could be USD 8k. So just the initial implementation of your JWT based authentication system is 4% into the "hundreds of thousands of dollars". Then you need to factor in the extra complexity on maintenance and before you know it we do not talk about hundreds of thousands of dollars but millions...
- hiptobecubic 5y ago
  I feel like "spaghetti garbage now, we'll fix it later" is a big part of why startups fail to execute well. Yeah you saved $8000 by launching your thing in a completely unmaintainable way, but now it's both harder to maintain and more likely to need it. Literally the first time it breaks you will probably lose that cost advantage just because it takes so long to debug. The point you should have made is that dogmatic approaches usually produce a lot of waste, but the example you gave us exactly why teams end up that way. Otherwise people come up with bullshit hacks like you describe and the entire team pays for it.
  - madsbuch 5y ago
    In that case it would be bad to b cut corners, ey? Though these were not the cases I talked about. I talked about implementing features that are not necessary for the product because of dogmatic reasons.