4 ms·

> I have found myself tending more and more towards a style where I break even some relatively small modules up into fairly small pieces, and I have very, very

> I have found myself tending more and more towards a style where I break even some relatively small modules up into fairly small pieces, and I have very, very clearly specified definitions for "what this module consumes" and "what this module provides". (I have not quite reached "very, very clearly". At the moment I'm still resisting the sheer amount of keyboard typing it takes to be clear. But I can see I'm trending this way.) In essence, take the idea of a dependency-injected function that uses no globals, and bring the same organization up to the module level. I'm not quite sure how this is different from classes. Visual Studio build out reports on call and data flow so you can easily see what calls what and what dependencies you have. You can also do neat things like see what code is dependent on a given library. > In the last few weeks, I've had a language coalescing in my head, which I'm not particularly happy about since I have no chance of being able to implement it, and one of the things it does is to encourage this sort of thing by making it easy. Basically, whenever importing a library, it would automatically add a layer of abstraction between your code and that library that allows you to override that library wholesale for testing purposes or something. I do this manually in a lot of languages I work in, but it involves writing a tedious layer that just takes calls to "A" in one side and routes calls to "A" out another. There's an idea of a "context" that you can pass to a module that would do this override. Basically it would be a statically-typed ability to monkeypatch, safely, and in a way where by and large, the compiler could optimize access to the "default" implementation such that you should generally not be paying an abstraction penalty. Then there would be a report that you could use the runtime tooling to generate that would tell you exactly what external functionality you're using, and you could use that to guide you in your override so you only implement what you need. C# has this feature and calls it "Microsoft Fakes" but they got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make. But boy is it useful for unit testing legacy code and reducing boiler plate required for testing. Honestly everything you mention is available on the .NET stack.

jerf 7y ago
"I'm not quite sure how this is different from classes." Well, depending on your definition of classes, I suppose. "A struct bundled together with methods for operating on it + some sort of polymorphism" on its own doesn't say anything about dependency management. But get 10 programmers together and ask for a definition of "class" and you can easily get 15 answers. "Visual Studio build out reports on call and data flow so you can easily see what calls what and what dependencies you have." Does it really have a report of exactly what APIs you use out of a module? I'd love to see an example of that if it's not too difficult, not because I disbelieve you, but because I'd love to see it. I know I've seen plenty of flow diagrams for what "libraries" or "modules" you use (whatever word is appropriate to your language), but knowing that you use "the AWS S3 module" is much less informative than knowing "you only use GetObject", to be specific about an example. "C# has this feature and calls it "Microsoft Fakes" but they got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make." I am not intimately familiar with this feature, so please do correct me if I am wrong. But I observe that according to this page: https://docs.microsoft.com/en-us/visualstudio/test/isolating-code-under-test-with-microsoft-fakes?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/test/isolating... that for stubs you have to manually create an interface, and for shims you're in a situation where the code is being rewritten dynamically. I would propose making it so that all library usage is automatically an interface type created by your usage. And as it would be implemented in the language spec, rather than being done by instruction-rewrite very late in the process, it would also be something that could be built on by other things. To be clear A: I'm well aware that I, just like everybody else, do not have any totally unique and new ideas literally never considered by anyone ever, so I am well aware that there are things like this in various bits and pieces elsewhere and B: I'm not exactly "criticizing" the .NET stack, especially with my vaporware-beyond-vaporware ideas. I'm just observing that there is an engineering difference between the things that run as instruction-level rewrites very late in the process and things integrated at the beginning. That's one of the ways Microsoft and Oracle/Java punch above the "weight" of what I'd otherwise expect from the languages in question merely on their features, and a legitimate advantage of being on their stacks, but even for something the size of Microsoft, that's where you have to stop advancing. You can't really build on that sort of tech because you can't stack that many such features together before the complexity exceeds what even those entities can deal with. (There's also some other features in my highly vaporous vaporware language that this integrates with in some other ways, which is why I'm concerned about needing to be able to build on these features more officially than last-minute assembly rewrites.)
- nemaar 7y ago
  I'm interested in your language ideas. Could you please share them with as much detail as you can/want?
  - jerf 7y ago
    Well, I've stayed up late one night already trying to get this crap out of my head. As anticipated, it has only made it worse in the short term, but I expect once I'm "done" it'll go away. It'll show up at jerf.org/iri, but I warn you that A: it may be a while yet and B: I'm not 100% sure it'll ever be publishable, but we'll see.
- JamesBarney 7y ago
  I didn't mean to say "look your idea's not unique", hust a hey if your interested in those features I happen to know of some similar features on my preferred stack. But here's an example of a dependency graph from ndepend(a plugin) https://www.ndepend.com/docs/visual-studio-dependency-graph https://www.ndepend.com/docs/visual-studio-dependency-graph You can break it up by namespace or class(I believe). > and for shims you're in a situation where the code is being rewritten dynamically. I would propose making it so that all library usage is automatically an interface type created by your usage. And as it would be implemented in the language spec, rather than being done by instruction-rewrite very late in the process, it would also be something that could be built on by other things. Mind explaining the advantage of auto-generating interfaces for library usage over shims? And a couple of other questions, is an interface automatically generated on compile for every exposed class? How does this work for static class usage?
  - jerf 7y ago
    Thank you for the example. "Mind explaining the advantage of auto-generating interfaces for library usage over shims?" For the particular definition of "shim" used by Microsoft, the fact that it's always implicitly there, rather than something you have to generate and add to the code. For example, if you import the "strings" library and use only "TrimSpace" and "IndexOf", there would be a way to A: have the tooling system directly feed you a listing of "this is what you use from this library/object/etc.", possibly even pregenerating the manifested interface and an initial stub object for you and B: there's a way to coordinate passing your new interface as the implementation of the "string library" that is going to be used for a particular run time context. The compiler can also statically test that your implementation completely covers the module; if you add a call to string.ToLower(), the compiler can complain that your shim is now incomplete. For static usage, I expect to say that the common use case is that you use the default implementation of the strings library, and in that case, it can be statically compiled as usual. I observe that in practice there's almost always a "real" implementation that is used everywhere except tests. If you're willing to pay the price for dynamic resolution, you'd be able swap out whatever you like during runtime though. In the common case, I expect it would look a lot like what a modern language does; you say "import string" (somehow), and you just get on with using the string functions. If you don't want to override that, you don't see anything strange. Languages that already have some similar features would include Python, where a module comes in to your namespace as just another object. In Python it's pretty easy to "duck type" up something that looks enough like a module that you could replace one if you want. I don't see it done often, probably because it's a global variable modification, but it's an example of how you can conceive of a library or module as something other than a completely static import of a static name. (Hypothetically, you could even get some dynamic overrides to be compiled statically in some cases. There are languages like Rust that successfully do that sort of thing a lot. However, that has a lot of preconditions for it to work, and requires a great deal of sophistication in the compiler. Initially I'd punt; performance is not my #1 goal, and if the implementation just fell back to dynamic resolution it wouldn't be the end of the world. Plenty of languages do just fine with having vtables.)
GordonS 7y ago
> got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make Hah, adding a matching interface for every..single..class is something that drives me nuts! Many C# devs seem to have fallen into this trap, because "I need it for mocking". Sure, but do you actually need to mock out the dependency? IMO, overuse of mocks makes for brittle tests that are painful to maintain. I prefer only mocking "complex" dependencies. In any case, hopefully C# 8's default interface methods will help alleviate this madness.

jerf 7y ago

"I'm not quite sure how this is different from classes." Well, depending on your definition of classes, I suppose. "A struct bundled together with methods for operating on it + some sort of polymorphism" on its own doesn't say anything about dependency management. But get 10 programmers together and ask for a definition of "class" and you can easily get 15 answers. "Visual Studio build out reports on call and data flow so you can easily see what calls what and what dependencies you have." Does it really have a report of exactly what APIs you use out of a module? I'd love to see an example of that if it's not too difficult, not because I disbelieve you, but because I'd love to see it. I know I've seen plenty of flow diagrams for what "libraries" or "modules" you use (whatever word is appropriate to your language), but knowing that you use "the AWS S3 module" is much less informative than knowing "you only use GetObject", to be specific about an example. "C# has this feature and calls it "Microsoft Fakes" but they got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make." I am not intimately familiar with this feature, so please do correct me if I am wrong. But I observe that according to this page: https://docs.microsoft.com/en-us/visualstudio/test/isolating-code-under-test-with-microsoft-fakes?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/test/isolating... that for stubs you have to manually create an interface, and for shims you're in a situation where the code is being rewritten dynamically. I would propose making it so that all library usage is automatically an interface type created by your usage. And as it would be implemented in the language spec, rather than being done by instruction-rewrite very late in the process, it would also be something that could be built on by other things. To be clear A: I'm well aware that I, just like everybody else, do not have any totally unique and new ideas literally never considered by anyone ever, so I am well aware that there are things like this in various bits and pieces elsewhere and B: I'm not exactly "criticizing" the .NET stack, especially with my vaporware-beyond-vaporware ideas. I'm just observing that there is an engineering difference between the things that run as instruction-level rewrites very late in the process and things integrated at the beginning. That's one of the ways Microsoft and Oracle/Java punch above the "weight" of what I'd otherwise expect from the languages in question merely on their features, and a legitimate advantage of being on their stacks, but even for something the size of Microsoft, that's where you have to stop advancing. You can't really build on that sort of tech because you can't stack that many such features together before the complexity exceeds what even those entities can deal with. (There's also some other features in my highly vaporous vaporware language that this integrates with in some other ways, which is why I'm concerned about needing to be able to build on these features more officially than last-minute assembly rewrites.)

nemaar 7y ago
I'm interested in your language ideas. Could you please share them with as much detail as you can/want?
- jerf 7y ago
  Well, I've stayed up late one night already trying to get this crap out of my head. As anticipated, it has only made it worse in the short term, but I expect once I'm "done" it'll go away. It'll show up at jerf.org/iri, but I warn you that A: it may be a while yet and B: I'm not 100% sure it'll ever be publishable, but we'll see.
JamesBarney 7y ago
I didn't mean to say "look your idea's not unique", hust a hey if your interested in those features I happen to know of some similar features on my preferred stack. But here's an example of a dependency graph from ndepend(a plugin) https://www.ndepend.com/docs/visual-studio-dependency-graph https://www.ndepend.com/docs/visual-studio-dependency-graph You can break it up by namespace or class(I believe). > and for shims you're in a situation where the code is being rewritten dynamically. I would propose making it so that all library usage is automatically an interface type created by your usage. And as it would be implemented in the language spec, rather than being done by instruction-rewrite very late in the process, it would also be something that could be built on by other things. Mind explaining the advantage of auto-generating interfaces for library usage over shims? And a couple of other questions, is an interface automatically generated on compile for every exposed class? How does this work for static class usage?
- jerf 7y ago
  Thank you for the example. "Mind explaining the advantage of auto-generating interfaces for library usage over shims?" For the particular definition of "shim" used by Microsoft, the fact that it's always implicitly there, rather than something you have to generate and add to the code. For example, if you import the "strings" library and use only "TrimSpace" and "IndexOf", there would be a way to A: have the tooling system directly feed you a listing of "this is what you use from this library/object/etc.", possibly even pregenerating the manifested interface and an initial stub object for you and B: there's a way to coordinate passing your new interface as the implementation of the "string library" that is going to be used for a particular run time context. The compiler can also statically test that your implementation completely covers the module; if you add a call to string.ToLower(), the compiler can complain that your shim is now incomplete. For static usage, I expect to say that the common use case is that you use the default implementation of the strings library, and in that case, it can be statically compiled as usual. I observe that in practice there's almost always a "real" implementation that is used everywhere except tests. If you're willing to pay the price for dynamic resolution, you'd be able swap out whatever you like during runtime though. In the common case, I expect it would look a lot like what a modern language does; you say "import string" (somehow), and you just get on with using the string functions. If you don't want to override that, you don't see anything strange. Languages that already have some similar features would include Python, where a module comes in to your namespace as just another object. In Python it's pretty easy to "duck type" up something that looks enough like a module that you could replace one if you want. I don't see it done often, probably because it's a global variable modification, but it's an example of how you can conceive of a library or module as something other than a completely static import of a static name. (Hypothetically, you could even get some dynamic overrides to be compiled statically in some cases. There are languages like Rust that successfully do that sort of thing a lot. However, that has a lot of preconditions for it to work, and requires a great deal of sophistication in the compiler. Initially I'd punt; performance is not my #1 goal, and if the implementation just fell back to dynamic resolution it wouldn't be the end of the world. Plenty of languages do just fine with having vtables.)

GordonS 7y ago

> got a lot of vocal pushback from the TDD community who see great value in adding dependency injection and an interface to every library call you make Hah, adding a matching interface for every..single..class is something that drives me nuts! Many C# devs seem to have fallen into this trap, because "I need it for mocking". Sure, but do you actually need to mock out the dependency? IMO, overuse of mocks makes for brittle tests that are painful to maintain. I prefer only mocking "complex" dependencies. In any case, hopefully C# 8's default interface methods will help alleviate this madness.