3 ms·

Yea, this is an issue rather near and dear to my heart (due to pain). I very much appreciate strong and safe types, but it tends to mean enormous pain when mak

by Groxx 2y ago

Yea, this is an issue rather near and dear to my heart (due to pain). I very much appreciate strong and safe types, but it tends to mean enormous pain when making small obvious fixes to past mistakes, and you really don't want to block those. It just makes everything harder in the long term. As an up-front caveat for below: I don't know Elixir in detail, so I'm approaching this as a general type/lang-design issue. And this is a bit stream-of-consciousness-y and I'm not really seeking any goal beyond maybe discussion. --- Structural sub-typing with inference (or similar, e.g. perhaps something fancier with dependent types) seems like kinda the only real option, as you need to be able to adapt to whatever bowl of Hyrum Slaw[1] has been created when you weren't looking. Allowing code that is still provably correct to continue to work without modification seems like a natural fit, and for changes I've made it would fairly often mean >90% of users would do absolutely nothing and simply get better safety and better behavior for free. It might even be an ideal end state. I kinda like the ergonomics of `revision 2` here, it's clear what it's doing and can provide tooling hints in a rather important and complicated situation... but tbh I'm just not sure how much this offers vs actual structural typing, e.g. just having an implicit revision per field. With explicit revisions you can bundle interrelated changes (which is quite good, and doing this with types alone ~always requires some annoying ceremony), but it seems like you'll also be forcing code to accept all of v2..N-1 to get the change in vN because they're not independent. The "you must accept all intermediate changes" part is in some ways natural, but you'll also be (potentially) forcing it on your users, and/or writing a lot of transitioning code to avoid constraining them. I'm guessing this is mostly due to Elixir's type system, and explicit versions are a pragmatic tradeoff? A linear rather than combinatoric growth of generated types? >It is unlikely - or shall we say, not advisable to - for a given application to depend on several revisions over a long period of time. They are meant to be transitory. An application, yes - applications should migrate when they are able, and because they are usage-leaf-nodes they can do that without backwards compatibility concerns. But any library that uses other libraries generally benefits from supporting as many versions as possible, to constrain the parent-library's users as little as possible. It's exactly the same situation as you see in normal SAT-like dependency management: applications should pin versions for stability, libraries should try to allow as broad of a range as possible to avoid conflicts. >Would downcasting actually be useful in practice? That is yet to be seen. I would pretty-strongly assume both "yes" and "it's complicated". For end-users directly touching those fields: yes absolutely, pick your level of risk and live with it! This kind of thing is great for isolated workarounds and "trust me, it's fine" scenarios, code has varying risk/goals and that's good. Those happen all the time, even if nobody really likes them afterward. But if those choices apply to all libraries using it in a project... well then it gets complicated. Unless you know how all of them use it, and they all agree, you can't safely make that decision. Ruby has refinements which can at least somewhat deal with this, by restricting when those decisions apply, and Lisps with continuations have another kind of tool, but most popular languages do not... and I have no idea how possible either would be in Elixir. --- All that probably summarizes as: if we could boil the ocean, would this be meaningfully different than structural typing with type inference, and no versioning? It sounds like this might be a reasonable middle-ground for Elixir, but what about in general, when trying to apply this strategy to other languages? And viewed through that lens, are there other structural typing tools worth looking at? [1] https://en.wiktionary.org/wiki/Hyrum%27s_law https://en.wiktionary.org/wiki/Hyrum%27s_law

josevalim 2y ago
Thank you for the comments. Your questions at the end resonate a lot with what I have been asking myself! > Structural sub-typing with inference Can we have structural sub-typing with inference that is relatively fast and will generate reasonable error reports? We have been bulking up the amount of inference for dynamic code in our system and sometimes the inferred types get quite large, which can make trouble shooting daunting. In any case, better inference is a win even without taking data evolution into account. > but tbh I'm just not sure how much this offers vs actual structural typing The bulk of the work is definitely achieved by structural typing. The revisions help generate automated type signatures that guarantee you have not widened the output for old versions. If all you have is inference, you could accidentally introduce breaking changes? I guess there may be some automated way where we could check old inferred types against new ones but I am not sure how it could be done without annotating that _something_ has changed? > The "you must accept all intermediate changes" part is in some ways natural, but you'll also be (potentially) forcing it on your users, and/or writing a lot of transitioning code to avoid constraining them. Theoretically, you do not need to support all revisions, only support more than one revision at once. A library that provides r1-r2-r3 can be supported downstream through the pairs `r1-r2` and then `r2-r3` and that should hopefully provide a smoother upgrade experience to everyone compared to `r1`, `r2`, and `r3` being part of distinct major versions.
Vampiero 2y ago
> It just makes everything harder in the long term. Opinions. I'd gladly refactor a C# legacy monster because I can follow the compiler errors and reason about the shape of the types and how they interact in an abstract, codeless way. I can modify 200+ files and not lose track of what I'm doing. I can apply the latest language features to make the code more concise and expressive while preserving the semantics. I can use my IDE to modify every single reference to some type or object across the entire solution in a couple of keystrokes without worrying that my search and replace regex will match something it shouldn't. I can simplify and generalize redundancies. I can transform everything in my head and make sure it's internally consistent. I can even use reflection to do advanced metaprogramming and make sure that it, too, is consistent (with a couple of tests). If it compiles, chances are it's going to work. The more static typing, the better, because that means that most of the logic is baked in the abstractions and I don't have to run the debugger. I would never refactor a legacy Javascript application because I'm not a masochist. There are many juniors who don't yet know what a pain it is to work on large dynamic projects so I'll let them have all the fun while I use my time productively. I haven't done a null check in years (in my personal projects; at my workplace they don't even know what an option type is). But I'll take Prolog. It's dynamic, but unification is basically type theory anyway. You can declare all your constraints and reason about them clearly. As God intended for us applied mathematicians. So your claim that static typing makes things harder in the long term is alien to me. The real issue is that companies are afraid of refactoring because the client doesn't see the immediate benefit, so they simply don't do it, and then 10 years later they just redo the project from scratch in the latest shiny language (while reinventing the whole domain mapping layer because the old engineers were probably idiots since they wrote such unmaintanable code -- let's ignore the fact that they, too, never had time to refactor and dogfood).

by Groxx 2y ago

josevalim 2y ago

Thank you for the comments. Your questions at the end resonate a lot with what I have been asking myself! > Structural sub-typing with inference Can we have structural sub-typing with inference that is relatively fast and will generate reasonable error reports? We have been bulking up the amount of inference for dynamic code in our system and sometimes the inferred types get quite large, which can make trouble shooting daunting. In any case, better inference is a win even without taking data evolution into account. > but tbh I'm just not sure how much this offers vs actual structural typing The bulk of the work is definitely achieved by structural typing. The revisions help generate automated type signatures that guarantee you have not widened the output for old versions. If all you have is inference, you could accidentally introduce breaking changes? I guess there may be some automated way where we could check old inferred types against new ones but I am not sure how it could be done without annotating that _something_ has changed? > The "you must accept all intermediate changes" part is in some ways natural, but you'll also be (potentially) forcing it on your users, and/or writing a lot of transitioning code to avoid constraining them. Theoretically, you do not need to support all revisions, only support more than one revision at once. A library that provides r1-r2-r3 can be supported downstream through the pairs `r1-r2` and then `r2-r3` and that should hopefully provide a smoother upgrade experience to everyone compared to `r1`, `r2`, and `r3` being part of distinct major versions.

Vampiero 2y ago

> It just makes everything harder in the long term. Opinions. I'd gladly refactor a C# legacy monster because I can follow the compiler errors and reason about the shape of the types and how they interact in an abstract, codeless way. I can modify 200+ files and not lose track of what I'm doing. I can apply the latest language features to make the code more concise and expressive while preserving the semantics. I can use my IDE to modify every single reference to some type or object across the entire solution in a couple of keystrokes without worrying that my search and replace regex will match something it shouldn't. I can simplify and generalize redundancies. I can transform everything in my head and make sure it's internally consistent. I can even use reflection to do advanced metaprogramming and make sure that it, too, is consistent (with a couple of tests). If it compiles, chances are it's going to work. The more static typing, the better, because that means that most of the logic is baked in the abstractions and I don't have to run the debugger. I would never refactor a legacy Javascript application because I'm not a masochist. There are many juniors who don't yet know what a pain it is to work on large dynamic projects so I'll let them have all the fun while I use my time productively. I haven't done a null check in years (in my personal projects; at my workplace they don't even know what an option type is). But I'll take Prolog. It's dynamic, but unification is basically type theory anyway. You can declare all your constraints and reason about them clearly. As God intended for us applied mathematicians. So your claim that static typing makes things harder in the long term is alien to me. The real issue is that companies are afraid of refactoring because the client doesn't see the immediate benefit, so they simply don't do it, and then 10 years later they just redo the project from scratch in the latest shiny language (while reinventing the whole domain mapping layer because the old engineers were probably idiots since they wrote such unmaintanable code -- let's ignore the fact that they, too, never had time to refactor and dogfood).