5 ms·

If you want functioning, robust, maintainable software (or even better, software that doesn't require maintenance), then spend a long time modeling the problem

If you want functioning, robust, maintainable software (or even better, software that doesn't require maintenance), then spend a long time modeling the problem domain. Build it as a system of types, a protocol, perhaps even a language (or at least an AST with semantics). Prove things about this model, particularly some useful things about soundness, consistency and (in)completeness. Learn all the funky symbols people use in the literature, learn about the strange tools you weren't told about in undergrad like dependent typing or higher-order contracts or CRDTs and lattices. Spend a lot of time doing this. Then, when you have determined the essential shape of the domain and nothing more, implement the software. At that point, the code almost writes itself. I submit that if we did that, we would have excellent, elegant, simple software, but following the process would be incredibly hard. So hard, in fact, that it couldn't possibly be distilled into a conference talk.

jcelerier 5y ago
By the time you're 20% in that process your competitor has already overtook the market.
- baryphonic 5y ago
  To quote Thiel, "competition is for losers."
FpUser 5y ago
Domain knowledge is very important. In the real world however by the time you finish this type of process the competition will have had the product out already. It may not be that perfect castle in the sky but it will work and if you have revenue you will have time and means to improve.
- baryphonic 5y ago
  100% agree. It's a trade-off. Get product-market fit first and learn what you can about the domain. Spend enough time on architecture up front so you can easily pivot. That's all the simplicity you should care about at that point. Once you get traction, you can start to afford to have the crazy vision. IMO, at that point it's easily worth the risk. A decent research team will probably discover something, and potentially extremely valuable knowledge. If you were James Clerk Maxwell before he published his equations, how much would they be worth to you, especially if you had paying customers?
- magicalhippo 5y ago
  Our customers don't even want to pay for something that bespoke. They have margins to worry about. So instead we've had to make a system which makes it less painful when bugs occur. For us that means making it trivial to run older major and minor versions our software, and an automated update mechanism which delivers new builds to customers on-premise in less than an hour, updating the DB schema as well.
  - samhw 5y ago
    I don't think this excludes what the GP said, but this is super important as well. I think of it as second-order reliability: design your software not only so that bugs don't occur, but also so that the user can take practical steps to remedy bugs if they do occur. (Also, as one of my past companies enshrined as an engineering axiom: "write software to be debugged". Most programmers write waaay too few logs. You know the print statements you add to your code when it's buggy, to track down what's going wrong? Well, do that all the time, and if there are too many then fix that problem with adequate tooling. If it's running on your customers' computers - whether servers or PCs or phones - then store them locally for N days / N logs and allow them to be submitted when a bug occurs. Stack traces - even good ones - are not nearly enough.)
wpietri 5y ago
What sort of domains do you see as sufficiently well-understood and stable where this process is even achievable? A lot of my career has been in domains where we are exploring problems by building and shipping things to see what really works for users and customers. And other times there's domain volatility driven by changes in technology and competitive landscape. Even for domains that are stable and knowable, I have to wonder what businesses can afford that kind of up-front investment before the first feature ships.
- baryphonic 5y ago
  I've had largely the same experience as you, but I have seen some hints that real simplicity could be possible. If the domain is technology itself, there may be no underlying simplicity. Ultimately, I think we have to make a trade-off between simplicity and easiness. The approach I outlined would be incredibly expensive because the tooling for that approach isn't quite good enough yet, and stakeholders wouldn't even understand it. They wouldn't realize that you were building a pitch for your product not as a PowerPoint deck, but as executable code! A lot of our complexity today is from constructing software itself over layer upon layer of previous complex software (CSS, I'm looking at you), not due to the intrinsic "business cases" our software is meant to solve. Some of that complexity cannot be avoided, and some of it could be but at significant cost. To use an analogy, it's also cheaper to build a traffic light-controlled intersection, but overpasses are simpler. Coincidentally, almost all of the tools I've seen that try to make simplicity cheaper come either from the Scheme/Racket/Lisp world that Hickey himself hails from or from Alan Kay and his sphere of influence. (The two groups have quite a bit of overlap, both in terms of ideas and even people.)
  - jiriro 5y ago
    Could you please elaborate on Hickey’s and Kay’s key ideas and how to try them hands on? I know about Smalltalk (Squeak) so I guess that is the playground for Kay’s. Would just playing with Clojure do the same for the Hickey’s?
  - wpietri 5y ago
    Sorry, I'm still not seeing how/when the approach you're hinting toward is practically valuable. So far it seems to me like you're pursuing one dimension of quality to the exclusion of others. Which is an interesting theoretical exercise, so if that's your jam, have at it. But it sounded to me like you were proposing something people could actually do.
Hercuros 5y ago
Speaking as someone with experience with many of those things (PL theory/formal verification background), I don't think they're even close to being a silver bullet. Coming up with the right abstractions and the right domain model is difficult (especially if you just sit down and try to come up with stuff, you're likely to get it wrong the first time around). Knowing about some of those things could help you come up with better abstractions, but it's neither necessary nor sufficient to ensure that you will. Take dependent types for example. They allow you to express more program invariants or correctness properties in your types. But actually using them requires you to write proofs (at least, if you're using them to their full potential). And I do think that in general System F like type systems hit a nice sweet spot and are generally good enough for the stuff that you might actually want to handle on the type system level. I've also run into similar "proof-like" situations with much simpler type systems like those of Haskell and Rust, where I was structuring my types to "make illegal states unrepresentable", but in the process ended up complicating my program due to having to match the structure of my program to the expected structure of the types. Sometimes it is nice to _not_ to have the type system enforce some of your invariants. (Such things are also doable with dependent types of course, but this is just an example of some of the tradeoffs involved). You can also still have a shitty domain model even if you use all of those fancy tools. They just allow you to be very formal/precise about the domain model (and do perhaps encourage some more uniformity by making it more annoying to express ugly or complicated things).
alatkins 5y ago
Counterpoint is that the Big Design Up-Front utopia didn't win in software, giving rise to Agile (for better or worse).

jcelerier 5y ago

By the time you're 20% in that process your competitor has already overtook the market.

baryphonic 5y ago
To quote Thiel, "competition is for losers."

FpUser 5y ago

Domain knowledge is very important. In the real world however by the time you finish this type of process the competition will have had the product out already. It may not be that perfect castle in the sky but it will work and if you have revenue you will have time and means to improve.

baryphonic 5y ago
100% agree. It's a trade-off. Get product-market fit first and learn what you can about the domain. Spend enough time on architecture up front so you can easily pivot. That's all the simplicity you should care about at that point. Once you get traction, you can start to afford to have the crazy vision. IMO, at that point it's easily worth the risk. A decent research team will probably discover something, and potentially extremely valuable knowledge. If you were James Clerk Maxwell before he published his equations, how much would they be worth to you, especially if you had paying customers?
magicalhippo 5y ago
Our customers don't even want to pay for something that bespoke. They have margins to worry about. So instead we've had to make a system which makes it less painful when bugs occur. For us that means making it trivial to run older major and minor versions our software, and an automated update mechanism which delivers new builds to customers on-premise in less than an hour, updating the DB schema as well.
- samhw 5y ago
  I don't think this excludes what the GP said, but this is super important as well. I think of it as second-order reliability: design your software not only so that bugs don't occur, but also so that the user can take practical steps to remedy bugs if they do occur. (Also, as one of my past companies enshrined as an engineering axiom: "write software to be debugged". Most programmers write waaay too few logs. You know the print statements you add to your code when it's buggy, to track down what's going wrong? Well, do that all the time, and if there are too many then fix that problem with adequate tooling. If it's running on your customers' computers - whether servers or PCs or phones - then store them locally for N days / N logs and allow them to be submitted when a bug occurs. Stack traces - even good ones - are not nearly enough.)

wpietri 5y ago

What sort of domains do you see as sufficiently well-understood and stable where this process is even achievable? A lot of my career has been in domains where we are exploring problems by building and shipping things to see what really works for users and customers. And other times there's domain volatility driven by changes in technology and competitive landscape. Even for domains that are stable and knowable, I have to wonder what businesses can afford that kind of up-front investment before the first feature ships.

baryphonic 5y ago
I've had largely the same experience as you, but I have seen some hints that real simplicity could be possible. If the domain is technology itself, there may be no underlying simplicity. Ultimately, I think we have to make a trade-off between simplicity and easiness. The approach I outlined would be incredibly expensive because the tooling for that approach isn't quite good enough yet, and stakeholders wouldn't even understand it. They wouldn't realize that you were building a pitch for your product not as a PowerPoint deck, but as executable code! A lot of our complexity today is from constructing software itself over layer upon layer of previous complex software (CSS, I'm looking at you), not due to the intrinsic "business cases" our software is meant to solve. Some of that complexity cannot be avoided, and some of it could be but at significant cost. To use an analogy, it's also cheaper to build a traffic light-controlled intersection, but overpasses are simpler. Coincidentally, almost all of the tools I've seen that try to make simplicity cheaper come either from the Scheme/Racket/Lisp world that Hickey himself hails from or from Alan Kay and his sphere of influence. (The two groups have quite a bit of overlap, both in terms of ideas and even people.)
- jiriro 5y ago
  Could you please elaborate on Hickey’s and Kay’s key ideas and how to try them hands on? I know about Smalltalk (Squeak) so I guess that is the playground for Kay’s. Would just playing with Clojure do the same for the Hickey’s?
- wpietri 5y ago
  Sorry, I'm still not seeing how/when the approach you're hinting toward is practically valuable. So far it seems to me like you're pursuing one dimension of quality to the exclusion of others. Which is an interesting theoretical exercise, so if that's your jam, have at it. But it sounded to me like you were proposing something people could actually do.

Hercuros 5y ago

Speaking as someone with experience with many of those things (PL theory/formal verification background), I don't think they're even close to being a silver bullet. Coming up with the right abstractions and the right domain model is difficult (especially if you just sit down and try to come up with stuff, you're likely to get it wrong the first time around). Knowing about some of those things could help you come up with better abstractions, but it's neither necessary nor sufficient to ensure that you will. Take dependent types for example. They allow you to express more program invariants or correctness properties in your types. But actually using them requires you to write proofs (at least, if you're using them to their full potential). And I do think that in general System F like type systems hit a nice sweet spot and are generally good enough for the stuff that you might actually want to handle on the type system level. I've also run into similar "proof-like" situations with much simpler type systems like those of Haskell and Rust, where I was structuring my types to "make illegal states unrepresentable", but in the process ended up complicating my program due to having to match the structure of my program to the expected structure of the types. Sometimes it is nice to _not_ to have the type system enforce some of your invariants. (Such things are also doable with dependent types of course, but this is just an example of some of the tradeoffs involved). You can also still have a shitty domain model even if you use all of those fancy tools. They just allow you to be very formal/precise about the domain model (and do perhaps encourage some more uniformity by making it more annoying to express ugly or complicated things).

alatkins 5y ago

Counterpoint is that the Big Design Up-Front utopia didn't win in software, giving rise to Agile (for better or worse).