31 ms·

It's often debated but in my mind, you know you're talking to a "senior" dev when nearly every response they have to a question is "well, it depends..." Every

It's often debated but in my mind, you know you're talking to a "senior" dev when nearly every response they have to a question is "well, it depends..." Every software project is different since software is a physical manifestation and codification of human process. And human process is... messy as all hell and doesn't like to be constrained by a lot of rules no matter how much effort and management goes into things. So, I prefer to advise teams build software that can be replaced easily, however that works. Isolate the points of change, make the system components replaceable (because they will be), but keep mind of where things need to be focused on efficiency and less on abstraction. This applies to a single code file all the way up to large scale distributed systems. In larger orgs, this helps since inevitably we all fall into the trap of Conway's law, and re-orgs inevitably lead to refactoring of systems along new ownership/communication lines. So, the right way to do something will always conform to the unique situations in which a system is developed -- "it depends"/

agentultra 3y ago
I think abstraction is key to efficiency. In order to have efficient systems we need to be able to prove properties of the software we write are correct with regards to our specifications. This is what abstractions do for us: they enable us to think in new semantic layers that are precise in the mathematical sense. This isn’t my idea, I just agree wholeheartedly with Conal Eliot. Resiliency is what we get when we want reliable systems but we cut corners with proving the correctness of the software we write: we add process supervisors, memory managers, tooling, exception handlers, etc: things that all cost us some efficiency in order to make our systems reliable enough to be useful. If we want efficiency, at scale, we need tools (languages really) that allow us to write proofs and generate code from these specifications.
- blindhippo 3y ago
  100% This is ideal. The trick for me though is that the foundational assumption on which to build our "proofs" change often and quickly. This creates drag on any software project to keep up, and becomes untenable if the abstractions are wrong (i.e. the interfaces for components aren't adaptable). This is what I meant by designing with the assumption that one component could be replaced. To me it's a about the challenge of designing to reduce the need to "cut corners with proving ... correctness", if that makes sense.
  - agentultra 3y ago
    Totally makes sense. It's an expensive enough process, still, that doing it at the scale that "non-verified" software is written at would be basically impossible. It makes sense that we've invested enough times in resiliency over the years that computer systems work as well as they do, let alone at all. We've squeezed a lot of efficiency out of our systems even without correctness. However I suspect we're beginning to reach a tipping point where it's becoming too expensive to avoid correctness and continue down this path of resiliency. At the scale of data-centers even small gains in efficiency have big effects. It's hard to get those kinds of gains without focusing on correctness. I'm hoping we'll find practical ways to compile dependently-typed programs and build theorem proving tools that scale to modern software practices and teams.
hnfong 3y ago
What does "build software that can be replaced easily" mean, if not "ensure the abstraction for this component makes sense so that people can replace it if needed"? Together with your followup qualification of "keep mind of where things need to be focused on efficiency and less on abstraction" I don't understand what you're trying to say at all. The idea that software needs to be designed to be replaced (as opposed to maintenance/improvement) seems to be very specific to some particular company's organization practices ("re-orgs"). Perhaps my objection to your generic "advice" is that your "it depends" is not "meta" enough to cover the cases where these Conways or whatever re-org crap isn't a recurring thing. For example, a person who's worked on too many failing projects might advise others to just write crap while looking productive by gaming LoC/commit/bugs fixed metrics, because all projects eventually fail anyway, and nobody cares about code quality if they discontinue the project. Another person who's stuck maintaining the crap they wrote themselves 10 years ago might go around telling others to make sure you get the first version right so that you don't end up in a situation like them. I'm not really sure which situation is more prevalent TBH. "It depends" indeed.

agentultra 3y ago

I think abstraction is key to efficiency. In order to have efficient systems we need to be able to prove properties of the software we write are correct with regards to our specifications. This is what abstractions do for us: they enable us to think in new semantic layers that are precise in the mathematical sense. This isn’t my idea, I just agree wholeheartedly with Conal Eliot. Resiliency is what we get when we want reliable systems but we cut corners with proving the correctness of the software we write: we add process supervisors, memory managers, tooling, exception handlers, etc: things that all cost us some efficiency in order to make our systems reliable enough to be useful. If we want efficiency, at scale, we need tools (languages really) that allow us to write proofs and generate code from these specifications.

blindhippo 3y ago
100% This is ideal. The trick for me though is that the foundational assumption on which to build our "proofs" change often and quickly. This creates drag on any software project to keep up, and becomes untenable if the abstractions are wrong (i.e. the interfaces for components aren't adaptable). This is what I meant by designing with the assumption that one component could be replaced. To me it's a about the challenge of designing to reduce the need to "cut corners with proving ... correctness", if that makes sense.
- agentultra 3y ago
  Totally makes sense. It's an expensive enough process, still, that doing it at the scale that "non-verified" software is written at would be basically impossible. It makes sense that we've invested enough times in resiliency over the years that computer systems work as well as they do, let alone at all. We've squeezed a lot of efficiency out of our systems even without correctness. However I suspect we're beginning to reach a tipping point where it's becoming too expensive to avoid correctness and continue down this path of resiliency. At the scale of data-centers even small gains in efficiency have big effects. It's hard to get those kinds of gains without focusing on correctness. I'm hoping we'll find practical ways to compile dependently-typed programs and build theorem proving tools that scale to modern software practices and teams.

hnfong 3y ago

What does "build software that can be replaced easily" mean, if not "ensure the abstraction for this component makes sense so that people can replace it if needed"? Together with your followup qualification of "keep mind of where things need to be focused on efficiency and less on abstraction" I don't understand what you're trying to say at all. The idea that software needs to be designed to be replaced (as opposed to maintenance/improvement) seems to be very specific to some particular company's organization practices ("re-orgs"). Perhaps my objection to your generic "advice" is that your "it depends" is not "meta" enough to cover the cases where these Conways or whatever re-org crap isn't a recurring thing. For example, a person who's worked on too many failing projects might advise others to just write crap while looking productive by gaming LoC/commit/bugs fixed metrics, because all projects eventually fail anyway, and nobody cares about code quality if they discontinue the project. Another person who's stuck maintaining the crap they wrote themselves 10 years ago might go around telling others to make sure you get the first version right so that you don't end up in a situation like them. I'm not really sure which situation is more prevalent TBH. "It depends" indeed.