9 ms·

I have found myself tending more and more towards a style where I break even some relatively small modules up into fairly small pieces, and I have very, very cl

by jerf 7y ago

I have found myself tending more and more towards a style where I break even some relatively small modules up into fairly small pieces, and I have very, very clearly specified definitions for "what this module consumes" and "what this module provides". (I have not quite reached "very, very clearly". At the moment I'm still resisting the sheer amount of keyboard typing it takes to be clear. But I can see I'm trending this way.) In essence, take the idea of a dependency-injected function that uses no globals, and bring the same organization up to the module level. Nominally, languages support this, but a lot of it is still implicit. For instance, do you have a command you can point at a module of your code and get a complete report of A: what libraries this code uses and B: exactly which subset of calls from those libraries this code uses? There's so many languages and environments and IDEs and such out there I imagine the answer may be yes for a few of you, but probably not that many, and even fewer of you use it. The primary reason I find myself moving this way is to try to make it so you can read a piece of code and the cognitive overhead is minimized, because there's a clear flow: 1. Here are my assumptions. 2. Here is my environment. 3. Here is what I do in that environment. 4. Here are the test cases that show that the thing I wanted to do is in fact done. In current languages, these things are not exactly "all mixed up", but they are not exactly cleanly separated, either. I realize this may sound vacuous and obvious, but, err, if that's so, a lot more of us could stand to actually do it, to put it in politic terms. I think the languages and environments work against us in a lot of ways by making it very easy to add dependencies without much thought and weave all those concerns together into one big undifferentiated mass. In the last few weeks, I've had a language coalescing in my head, which I'm not particularly happy about since I have no chance of being able to implement it, and one of the things it does is to encourage this sort of thing by making it easy. Basically, whenever importing a library, it would automatically add a layer of abstraction between your code and that library that allows you to override that library wholesale for testing purposes or something. I do this manually in a lot of languages I work in, but it involves writing a tedious layer that just takes calls to "A" in one side and routes calls to "A" out another. There's an idea of a "context" that you can pass to a module that would do this override. Basically it would be a statically-typed ability to monkeypatch, safely, and in a way where by and large, the compiler could optimize access to the "default" implementation such that you should generally not be paying an abstraction penalty. Then there would be a report that you could use the runtime tooling to generate that would tell you exactly what external functionality you're using, and you could use that to guide you in your override so you only implement what you need. (I have a mental image of a cell, which being biologic, doesn't really do anything cleanly, but taking it metaphorically, you can see cell walls declaring the things they will allow to pass through, and with only a bit more work we can clearly declare what comes out.) There are, of course, bits and pieces of this scattered all over the language landscape, but I'm not aware of anything that quite has everything I want in one place. (Perhaps surprisingly, Perl's "local" keyword is the closest single thing I know, albeit not written in a way that can support threading well which I'd want to fix. You can use local to override arbitrary function/method symbols, and it will be scoped within that local only, giving you that "dynamic language" monkeypatching while preventing it from being global state.) Part of the idea of that report too is that it would be part of the documentation for a module, further increasing the ability to pick up any arbitrary module in a program, and cleanly cut away "look, this is exactly what this particular module does. You may not necessarily know what the other modules this communicates with is doing with that stuff, but at least you know what this module is doing." Again, that may sound like it's a thing that all languages already do when you bring up simplified mental model of a codebase, but think about this next time you're hip deep in code you've never seen before and you hit the thirtieth line of code and suddenly, oh crap, it just referenced another module I've never heard of.... this is when your cognitive load goes through the roof.

JamesBarney 7y ago
> I have found myself tending more and more towards a style where I break even some relatively small modules up into fairly small pieces, and I have very, very clearly specified definitions for "what this module consumes" and "what this module provides". (I have not quite reached "very, very clearly". At the moment I'm still resisting the sheer amount of keyboard typing it takes to be clear. But I can see I'm trending this way.) In essence, take the idea of a dependency-injected function that uses no globals, and bring the same organization up to the module level. I'm not quite sure how this is different from classes. Visual Studio build out reports on call and data flow so you can easily see what calls what and what dependencies you have. You can also do neat things like see what code is dependent on a given library. > In the last few weeks, I've had a language coalescing in my head, which I'm not particularly happy about since I have no chance of being able to implement it, and one of the things it does is to encourage this sort of thing by making it easy. Basically, whenever importing a library, it would automatically add a layer of abstraction between your code and that library that allows you to override that library wholesale for testing purposes or something. I do this manually in a lot of languages I work in, but it involves writing a tedious layer that just takes calls to "A" in one side and routes calls to "A" out another. There's an idea of a "context" that you can pass to a module that would do this override. Basically it would be a statically-typed ability to monkeypatch, safely, and in a way where by and large, the compiler could optimize access to the "default" implementation such that you should generally not be paying an abstraction penalty. Then there would be a report that you could use the runtime tooling to generate that would tell you exactly what external functionality you're using, and you could use that to guide you in your override so you only implement what you need. C# has this feature and calls it "Microsoft Fakes" but they got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make. But boy is it useful for unit testing legacy code and reducing boiler plate required for testing. Honestly everything you mention is available on the .NET stack.
- jerf 7y ago
  "I'm not quite sure how this is different from classes." Well, depending on your definition of classes, I suppose. "A struct bundled together with methods for operating on it + some sort of polymorphism" on its own doesn't say anything about dependency management. But get 10 programmers together and ask for a definition of "class" and you can easily get 15 answers. "Visual Studio build out reports on call and data flow so you can easily see what calls what and what dependencies you have." Does it really have a report of exactly what APIs you use out of a module? I'd love to see an example of that if it's not too difficult, not because I disbelieve you, but because I'd love to see it. I know I've seen plenty of flow diagrams for what "libraries" or "modules" you use (whatever word is appropriate to your language), but knowing that you use "the AWS S3 module" is much less informative than knowing "you only use GetObject", to be specific about an example. "C# has this feature and calls it "Microsoft Fakes" but they got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make." I am not intimately familiar with this feature, so please do correct me if I am wrong. But I observe that according to this page: https://docs.microsoft.com/en-us/visualstudio/test/isolating-code-under-test-with-microsoft-fakes?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/test/isolating... that for stubs you have to manually create an interface, and for shims you're in a situation where the code is being rewritten dynamically. I would propose making it so that all library usage is automatically an interface type created by your usage. And as it would be implemented in the language spec, rather than being done by instruction-rewrite very late in the process, it would also be something that could be built on by other things. To be clear A: I'm well aware that I, just like everybody else, do not have any totally unique and new ideas literally never considered by anyone ever, so I am well aware that there are things like this in various bits and pieces elsewhere and B: I'm not exactly "criticizing" the .NET stack, especially with my vaporware-beyond-vaporware ideas. I'm just observing that there is an engineering difference between the things that run as instruction-level rewrites very late in the process and things integrated at the beginning. That's one of the ways Microsoft and Oracle/Java punch above the "weight" of what I'd otherwise expect from the languages in question merely on their features, and a legitimate advantage of being on their stacks, but even for something the size of Microsoft, that's where you have to stop advancing. You can't really build on that sort of tech because you can't stack that many such features together before the complexity exceeds what even those entities can deal with. (There's also some other features in my highly vaporous vaporware language that this integrates with in some other ways, which is why I'm concerned about needing to be able to build on these features more officially than last-minute assembly rewrites.)

by jerf 7y ago

JamesBarney 7y ago

> I have found myself tending more and more towards a style where I break even some relatively small modules up into fairly small pieces, and I have very, very clearly specified definitions for "what this module consumes" and "what this module provides". (I have not quite reached "very, very clearly". At the moment I'm still resisting the sheer amount of keyboard typing it takes to be clear. But I can see I'm trending this way.) In essence, take the idea of a dependency-injected function that uses no globals, and bring the same organization up to the module level. I'm not quite sure how this is different from classes. Visual Studio build out reports on call and data flow so you can easily see what calls what and what dependencies you have. You can also do neat things like see what code is dependent on a given library. > In the last few weeks, I've had a language coalescing in my head, which I'm not particularly happy about since I have no chance of being able to implement it, and one of the things it does is to encourage this sort of thing by making it easy. Basically, whenever importing a library, it would automatically add a layer of abstraction between your code and that library that allows you to override that library wholesale for testing purposes or something. I do this manually in a lot of languages I work in, but it involves writing a tedious layer that just takes calls to "A" in one side and routes calls to "A" out another. There's an idea of a "context" that you can pass to a module that would do this override. Basically it would be a statically-typed ability to monkeypatch, safely, and in a way where by and large, the compiler could optimize access to the "default" implementation such that you should generally not be paying an abstraction penalty. Then there would be a report that you could use the runtime tooling to generate that would tell you exactly what external functionality you're using, and you could use that to guide you in your override so you only implement what you need. C# has this feature and calls it "Microsoft Fakes" but they got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make. But boy is it useful for unit testing legacy code and reducing boiler plate required for testing. Honestly everything you mention is available on the .NET stack.

jerf 7y ago
"I'm not quite sure how this is different from classes." Well, depending on your definition of classes, I suppose. "A struct bundled together with methods for operating on it + some sort of polymorphism" on its own doesn't say anything about dependency management. But get 10 programmers together and ask for a definition of "class" and you can easily get 15 answers. "Visual Studio build out reports on call and data flow so you can easily see what calls what and what dependencies you have." Does it really have a report of exactly what APIs you use out of a module? I'd love to see an example of that if it's not too difficult, not because I disbelieve you, but because I'd love to see it. I know I've seen plenty of flow diagrams for what "libraries" or "modules" you use (whatever word is appropriate to your language), but knowing that you use "the AWS S3 module" is much less informative than knowing "you only use GetObject", to be specific about an example. "C# has this feature and calls it "Microsoft Fakes" but they got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make." I am not intimately familiar with this feature, so please do correct me if I am wrong. But I observe that according to this page: https://docs.microsoft.com/en-us/visualstudio/test/isolating-code-under-test-with-microsoft-fakes?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/test/isolating... that for stubs you have to manually create an interface, and for shims you're in a situation where the code is being rewritten dynamically. I would propose making it so that all library usage is automatically an interface type created by your usage. And as it would be implemented in the language spec, rather than being done by instruction-rewrite very late in the process, it would also be something that could be built on by other things. To be clear A: I'm well aware that I, just like everybody else, do not have any totally unique and new ideas literally never considered by anyone ever, so I am well aware that there are things like this in various bits and pieces elsewhere and B: I'm not exactly "criticizing" the .NET stack, especially with my vaporware-beyond-vaporware ideas. I'm just observing that there is an engineering difference between the things that run as instruction-level rewrites very late in the process and things integrated at the beginning. That's one of the ways Microsoft and Oracle/Java punch above the "weight" of what I'd otherwise expect from the languages in question merely on their features, and a legitimate advantage of being on their stacks, but even for something the size of Microsoft, that's where you have to stop advancing. You can't really build on that sort of tech because you can't stack that many such features together before the complexity exceeds what even those entities can deal with. (There's also some other features in my highly vaporous vaporware language that this integrates with in some other ways, which is why I'm concerned about needing to be able to build on these features more officially than last-minute assembly rewrites.)