4 ms·

I think this is the kind of thing you learn at uni and then potentially unlearn later on. Over the years I became better at using code to explore problem spaces

I think this is the kind of thing you learn at uni and then potentially unlearn later on. Over the years I became better at using code to explore problem spaces and as a design tool. Nowadays I feel that incremental design delivers better results in less time than upfront design.

marcos100 5y ago
I think your incremental design delivers better results because you already know or at least have a hunch of what wouldn't work and avoid that. You have an abstract architecture when starting and change accordingly on the fly, while programming, using your own best practices. Top down and bottom up architecture have their places. Being extreme in favor of one side is usually bad, as almost anything in life.
- skohan 5y ago
  I'm just having trouble understanding what you're talking about. Like what would be a concrete example of how a poor up-front design decision would paint you into an unrecoverable corner?
  - kaba0 5y ago
    My experience says that you might not need much design for a typical CRUD app, but try to write a JVM/compiler/database and you will quickly see that a bad design pretty much aborts the given project and you have to start from almost scratch. There is no incrementel rewrite between different stack/heap handling as those are an absolutely central parts of the design, which are pretty much impossible to try to encapsulate, as opposed to the 34th API endpoint. So what it means is that certain domains have much higher essential complexity and at that point the average encapsulation given by OOP/language tools are not sufficient to contain these parts, complexity will triumph and the whole program has to be viewed as one unified whole. Concurrent applications are a similar can of worms.
    - skohan 5y ago
      Yeah I mean for sure it depends on the domain. But it's interesting you mentioned compilers - I'm in the process of writing one, and I very much used an incremental approach. The first pass I essentially wrote a parser and a component which walked the AST and produced output. At a certain point it was clear that local knowledge of the AST wasn't sufficient to capture non-local details about the program which were required to produce the correct output. I got away for a short time with dirty tricks, but eventually transitioned to a new design: I kept the lexer and parser, and implemented a data driven IR in the style of an ECS system to be built up before emitting output. So I threw out the initial output component, but I learned a ton by starting with an end-to-end compiler, however incomplete. If I hadn't taken that step, and tried first to plan the perfect IR on paper, I am certain I would have reached an inferior result. edit: and even the IR and compiler middleware is loosely coupled. The IR is essentially a set of flat data tables, each of which is built independently by walking the AST. And the compiler is implemented in a series of independent passes: i.e. one pass to build the IR, one pass to derive type information etc. so it's very much grown into a series of independent components, each of which could be independently rewritten without affecting the others very much.
    - CipherThrowaway 5y ago
      All the technologies you mentioned do undergo large component rewrites and refactoring very often. It's true that e.g. the JVM is sometimes hemmed in by decisions from the past. But it is a decades old project and it is not clear that more up front design and deliberation would have future proofed the project for the language and VM conventions of the 2020s. I have applied incremental design to concurrent applications and a compiler + stack VM project that runs in embedded environments. You don't go in blind. You do need domain experience and broad strokes knowledge of the conventions. You make some major architectural decisions up front but these don't involve much planning or design. Contrary to your point about CRUD apps, API design is harder to achieve incrementally since it is an interface and requires cross-team (sometimes cross-organizational) iteration. It's still possible, but your organization needs to be equipped for incremental/agile work.
  - marcos100 5y ago
    If you have infinite time to recover, there is no problem, but you could also design something perfect using that infinite time. A bit of thinking about design and architecture can save you a lot of time. Start with the wrong data structures and maybe you'll have to patch a lot of thing or just redesign everything from scratch. Be an architecture astronaut and you may never release whatever you're suppose to develop. It's all about trade-offs.

marcos100 5y ago

I think your incremental design delivers better results because you already know or at least have a hunch of what wouldn't work and avoid that. You have an abstract architecture when starting and change accordingly on the fly, while programming, using your own best practices. Top down and bottom up architecture have their places. Being extreme in favor of one side is usually bad, as almost anything in life.

skohan 5y ago
I'm just having trouble understanding what you're talking about. Like what would be a concrete example of how a poor up-front design decision would paint you into an unrecoverable corner?
- kaba0 5y ago
  My experience says that you might not need much design for a typical CRUD app, but try to write a JVM/compiler/database and you will quickly see that a bad design pretty much aborts the given project and you have to start from almost scratch. There is no incrementel rewrite between different stack/heap handling as those are an absolutely central parts of the design, which are pretty much impossible to try to encapsulate, as opposed to the 34th API endpoint. So what it means is that certain domains have much higher essential complexity and at that point the average encapsulation given by OOP/language tools are not sufficient to contain these parts, complexity will triumph and the whole program has to be viewed as one unified whole. Concurrent applications are a similar can of worms.
  - skohan 5y ago
    Yeah I mean for sure it depends on the domain. But it's interesting you mentioned compilers - I'm in the process of writing one, and I very much used an incremental approach. The first pass I essentially wrote a parser and a component which walked the AST and produced output. At a certain point it was clear that local knowledge of the AST wasn't sufficient to capture non-local details about the program which were required to produce the correct output. I got away for a short time with dirty tricks, but eventually transitioned to a new design: I kept the lexer and parser, and implemented a data driven IR in the style of an ECS system to be built up before emitting output. So I threw out the initial output component, but I learned a ton by starting with an end-to-end compiler, however incomplete. If I hadn't taken that step, and tried first to plan the perfect IR on paper, I am certain I would have reached an inferior result. edit: and even the IR and compiler middleware is loosely coupled. The IR is essentially a set of flat data tables, each of which is built independently by walking the AST. And the compiler is implemented in a series of independent passes: i.e. one pass to build the IR, one pass to derive type information etc. so it's very much grown into a series of independent components, each of which could be independently rewritten without affecting the others very much.
  - CipherThrowaway 5y ago
    All the technologies you mentioned do undergo large component rewrites and refactoring very often. It's true that e.g. the JVM is sometimes hemmed in by decisions from the past. But it is a decades old project and it is not clear that more up front design and deliberation would have future proofed the project for the language and VM conventions of the 2020s. I have applied incremental design to concurrent applications and a compiler + stack VM project that runs in embedded environments. You don't go in blind. You do need domain experience and broad strokes knowledge of the conventions. You make some major architectural decisions up front but these don't involve much planning or design. Contrary to your point about CRUD apps, API design is harder to achieve incrementally since it is an interface and requires cross-team (sometimes cross-organizational) iteration. It's still possible, but your organization needs to be equipped for incremental/agile work.
- marcos100 5y ago
  If you have infinite time to recover, there is no problem, but you could also design something perfect using that infinite time. A bit of thinking about design and architecture can save you a lot of time. Start with the wrong data structures and maybe you'll have to patch a lot of thing or just redesign everything from scratch. Be an architecture astronaut and you may never release whatever you're suppose to develop. It's all about trade-offs.