6 ms·

So you‘re trying to remove complexity by making it invisible? Sounds great while things work, but doesn‘t that turn debugging into a nightmarish exercise? Becau

by veddox 7y ago

So you‘re trying to remove complexity by making it invisible? Sounds great while things work, but doesn‘t that turn debugging into a nightmarish exercise? Because I never get to see the code that‘s actually executing, the pieces of a function might be scattered over a dozen different layers. Strikes me as the ultimate recipe for self-modifying spaghetti code... Or am I missing something here?

akkartik 7y ago
I'm not sure what you mean by 'invisible'. Notice that I make zero claims of reducing complexity. Layers are only a way to manage and stage complexity. I responded to tooling questions at https://news.ycombinator.com/item?id=21766557#21767393 https://news.ycombinator.com/item?id=21766557#21767393
- magicalhippo 7y ago
  Well studying 010vm.cc, I wouldn't know that 014literal_string.cc alters the to_string function. That information is hidden from me, until I stumble upon it in 014literal_string.cc.
  - akkartik 7y ago
    Most of the time you wouldn't care. The presence of a waypoint is a hint that _something_ is inserting code there. But as long as tests in 010vm.cc continue to pass, it shouldn't matter to it what future layers do. When debugging things, however, you often want to see the big picture of how the layers come together. In those situations I look at the generated C++. It's designed to be clean and easy to read, and it has #line directives so that you can tell what layer each line came from.
    - magicalhippo 7y ago
      But there's no "waypoint" there, it uses the function signature as a marker to insert the code. Guess I'd have to use it for a bit to get a feel for how it is in practice. My initial reaction is that it just moves the complexity around a bit, without really helping much.
      - akkartik 7y ago
        Ah, I see what you're saying. (Links for any others reading: https://akkartik.github.io/mu1/html/010vm.cc.html#L606 https://akkartik.github.io/mu1/html/010vm.cc.html#L606 and https://akkartik.github.io/mu1/html/014literal_string.cc.html#L121 https://akkartik.github.io/mu1/html/014literal_string.cc.htm...) I should have been more hygienic in this old prototype. But honestly this idea of a waypoint being a hint is something I just thought up. Mostly my mindset is that layers should be oblivious to later layers. Ditto for readers; the goal is for you to not have to worry about later layers when you're reading a simple version. > My initial reaction is that it just moves the complexity around a bit, without really helping much. From OP: "We aren't just reordering bits here. There's a new constraint that has no counterpart in current programming practice — remove a feature, and everything before it should build and pass its tests." If this doesn't seem valuable, I probably can't persuade you. In general I've grown a lot less concerned about how code looks and a lot more concerned about having lots of different runnable versions of a single program that a reader can play around with to build up an instinct for it. Each unit test in a program is conceptually a distinct runnable version. Layers push further in that direction.
        magicalhippo 7y ago
        When I read your rationale, it feels a lot like literal programming, as I experienced it when reading PBR[1]. But without the documentation around it, I'm not sure I feel the same way about it. It feels harder to reason about what where why. Also, I'm assuming bugs should be fixed without using this layering. After all, without that layer you got broken code. But often I find there's not a clear-cut distinction between feature and bug fix. What then? Anyway, interesting idea, would have to try it to get a feel like I said. [1]: For example http://www.pbr-book.org/3ed-2018/Sampling_and_Reconstruction/Sampling_Interface.html#BasicSamplerInterface http://www.pbr-book.org/3ed-2018/Sampling_and_Reconstruction...
        akkartik 7y ago
        It's definitely inspired by Literate Programming. My code instructions in OP refer to a directory called wart/literate. > But without the documentation around it, I'm not sure I feel the same way about it. It feels harder to reason about what where why. I don't understand this. Are you saying my approach doesn't permit documentation? I don't see why. Check out the start of the first layer: http://akkartik.github.io/mu/html/000organization.cc.html http://akkartik.github.io/mu/html/000organization.cc.html Do you mean that the documentation doesn't look as nice as Literate systems? I think that's a good thing: http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming. Giving important concepts in your program a good central place is the essential need of documentation. Layers give features a single consolidated place (a https://en.wikipedia.org/wiki/Focal_point_(game_theory) https://en.wikipedia.org/wiki/Focal_point_(game_theory) ) where you can include documentation and be confident that someone interested in the feature will find it. Next to this attribute, I think typography is almost meaningless. I think it's far more important that documentation be accessible from within my programming environment, than that it be typeset well. > I'm assuming bugs should be fixed without using this layering. Yup! From the final paragraph of OP: "Codebases have three kinds of changes: bugfixes, new features and reorganizations. In my new organization bugfixes modify a single layer.."
    - veddox 7y ago
      So am I understanding you correctly that the big advantage of this technique is that it makes the initial reading of the codebase easier? But as soon as you need to do the „real work“ (a.k.a. debugging) you revert to the full source code anyway? I have to admit I don‘t have a whole lot of experience with large codebases, but this seems a bit like premature optimisation. (After all, this is just pushing the work of really understanding the code base away from the initial reading phase and on to the actual debugging/extending - which is the phase most work is done in.) Also, how do you deal with unexpected interactions between layers, especially when you don‘t see all the layers as you implement a new one?
      - akkartik 7y ago
        > am I understanding you correctly that the big advantage of this technique is that it makes the initial reading of the codebase easier? But as soon as you need to do the „real work“ (a.k.a. debugging) you revert to the full source code anyway? No, I've seldom tended to look at the generated C++. It's an escape hatch of last resort. Mostly I live in layers. > how do you deal with unexpected interactions between layers, especially when you don‘t see all the layers as you implement a new one? If you're constantly having to think about interactions between layers, you probably are trying to over-use layers. Did you ever find yourself creating git commits that tend to change one line in some place and add a function someplace else? That's a sort of classic use case for layers. But I'm not sure I can convey this any more clearly through prose. Perhaps what would work is for you to try to read a codebase built in this way, to get a feel for what I use layers for. Candidate codebases: https://github.com/akkartik/mu#readme https://github.com/akkartik/mu#readme https://github.com/akkartik/mu1#readme https://github.com/akkartik/mu1#readme (an earlier prototype that uses very different mechanisms, and entirely different code)

by veddox 7y ago

akkartik 7y ago

I'm not sure what you mean by 'invisible'. Notice that I make zero claims of reducing complexity. Layers are only a way to manage and stage complexity. I responded to tooling questions at https://news.ycombinator.com/item?id=21766557#21767393 https://news.ycombinator.com/item?id=21766557#21767393

magicalhippo 7y ago
Well studying 010vm.cc, I wouldn't know that 014literal_string.cc alters the to_string function. That information is hidden from me, until I stumble upon it in 014literal_string.cc.
- akkartik 7y ago
  Most of the time you wouldn't care. The presence of a waypoint is a hint that _something_ is inserting code there. But as long as tests in 010vm.cc continue to pass, it shouldn't matter to it what future layers do. When debugging things, however, you often want to see the big picture of how the layers come together. In those situations I look at the generated C++. It's designed to be clean and easy to read, and it has #line directives so that you can tell what layer each line came from.
  - magicalhippo 7y ago
    But there's no "waypoint" there, it uses the function signature as a marker to insert the code. Guess I'd have to use it for a bit to get a feel for how it is in practice. My initial reaction is that it just moves the complexity around a bit, without really helping much.
    - akkartik 7y ago
      Ah, I see what you're saying. (Links for any others reading: https://akkartik.github.io/mu1/html/010vm.cc.html#L606 https://akkartik.github.io/mu1/html/010vm.cc.html#L606 and https://akkartik.github.io/mu1/html/014literal_string.cc.html#L121 https://akkartik.github.io/mu1/html/014literal_string.cc.htm...) I should have been more hygienic in this old prototype. But honestly this idea of a waypoint being a hint is something I just thought up. Mostly my mindset is that layers should be oblivious to later layers. Ditto for readers; the goal is for you to not have to worry about later layers when you're reading a simple version. > My initial reaction is that it just moves the complexity around a bit, without really helping much. From OP: "We aren't just reordering bits here. There's a new constraint that has no counterpart in current programming practice — remove a feature, and everything before it should build and pass its tests." If this doesn't seem valuable, I probably can't persuade you. In general I've grown a lot less concerned about how code looks and a lot more concerned about having lots of different runnable versions of a single program that a reader can play around with to build up an instinct for it. Each unit test in a program is conceptually a distinct runnable version. Layers push further in that direction.
      - magicalhippo 7y ago
        When I read your rationale, it feels a lot like literal programming, as I experienced it when reading PBR[1]. But without the documentation around it, I'm not sure I feel the same way about it. It feels harder to reason about what where why. Also, I'm assuming bugs should be fixed without using this layering. After all, without that layer you got broken code. But often I find there's not a clear-cut distinction between feature and bug fix. What then? Anyway, interesting idea, would have to try it to get a feel like I said. [1]: For example http://www.pbr-book.org/3ed-2018/Sampling_and_Reconstruction/Sampling_Interface.html#BasicSamplerInterface http://www.pbr-book.org/3ed-2018/Sampling_and_Reconstruction...
        akkartik 7y ago
        It's definitely inspired by Literate Programming. My code instructions in OP refer to a directory called wart/literate. > But without the documentation around it, I'm not sure I feel the same way about it. It feels harder to reason about what where why. I don't understand this. Are you saying my approach doesn't permit documentation? I don't see why. Check out the start of the first layer: http://akkartik.github.io/mu/html/000organization.cc.html http://akkartik.github.io/mu/html/000organization.cc.html Do you mean that the documentation doesn't look as nice as Literate systems? I think that's a good thing: http://akkartik.name/post/literate-programming http://akkartik.name/post/literate-programming. Giving important concepts in your program a good central place is the essential need of documentation. Layers give features a single consolidated place (a https://en.wikipedia.org/wiki/Focal_point_(game_theory) https://en.wikipedia.org/wiki/Focal_point_(game_theory) ) where you can include documentation and be confident that someone interested in the feature will find it. Next to this attribute, I think typography is almost meaningless. I think it's far more important that documentation be accessible from within my programming environment, than that it be typeset well. > I'm assuming bugs should be fixed without using this layering. Yup! From the final paragraph of OP: "Codebases have three kinds of changes: bugfixes, new features and reorganizations. In my new organization bugfixes modify a single layer.."
  - veddox 7y ago
    So am I understanding you correctly that the big advantage of this technique is that it makes the initial reading of the codebase easier? But as soon as you need to do the „real work“ (a.k.a. debugging) you revert to the full source code anyway? I have to admit I don‘t have a whole lot of experience with large codebases, but this seems a bit like premature optimisation. (After all, this is just pushing the work of really understanding the code base away from the initial reading phase and on to the actual debugging/extending - which is the phase most work is done in.) Also, how do you deal with unexpected interactions between layers, especially when you don‘t see all the layers as you implement a new one?
    - akkartik 7y ago
      > am I understanding you correctly that the big advantage of this technique is that it makes the initial reading of the codebase easier? But as soon as you need to do the „real work“ (a.k.a. debugging) you revert to the full source code anyway? No, I've seldom tended to look at the generated C++. It's an escape hatch of last resort. Mostly I live in layers. > how do you deal with unexpected interactions between layers, especially when you don‘t see all the layers as you implement a new one? If you're constantly having to think about interactions between layers, you probably are trying to over-use layers. Did you ever find yourself creating git commits that tend to change one line in some place and add a function someplace else? That's a sort of classic use case for layers. But I'm not sure I can convey this any more clearly through prose. Perhaps what would work is for you to try to read a codebase built in this way, to get a feel for what I use layers for. Candidate codebases: https://github.com/akkartik/mu#readme https://github.com/akkartik/mu#readme https://github.com/akkartik/mu1#readme https://github.com/akkartik/mu1#readme (an earlier prototype that uses very different mechanisms, and entirely different code)