4 ms·

Working on a large scala codebase, I quickly learnt that 'compile time' in one language does not always happen before 'run time' in another language. What matt

Working on a large scala codebase, I quickly learnt that 'compile time' in one language does not always happen before 'run time' in another language. What matters is not the phase in which the bug is found but how close in absolute time finding the big is to writing it.

tluyben2 6y ago
Agreed and we have to continue improving in every way; dynamic/static/hybrid, just saying that I have not seen this dynamic enlightenment in larger projects. I have only seen the pain of runtime errors that other (static language) teams never had. Sure, if you would-have-written a test for it, you wouldn't have had it either, but types rather force you to think about it while writing. So sure, you are 'done faster', but the fall-out, and again ofcourse YMMV, of having statically preventable bugs popping up in Sentry at 3 am in the morning with things you would've prevented (not necessarily directly by the types but you would've thought about it more because you had to define the types, which is I think what the parent poster here misses too; I just tend to think less and try more without types which, again for me, is a bad state, but ymmv) is not great. But sure, I am biased as my experience with statically typed langs has been good since I moved from asm/basic in the 80s to pascal/c (they were an improvement over asm/basic and my first experience with types, not saying you should use them now, or not).
- kybernetikos 6y ago
  The view I've heard expressed is that deep thought on a piece of code reduces bugs. Whether that takes the shape of religious TDD, rigorous proofs or detailed type design doesn't make such a lot of difference. I used to be fully bought into types, but I've since realised that they have a number of downsides that in many cases more than offset their benefits: 1. Ergonomic typesystems require a lot of work to happen at compile time and slow down the iteration time (one of the more important things for programming in my view). In my view, saving the source and seeing the result almost immediately in a browser is one of the big advantages of web development. 2. Types are almost always written in a second, much less powerful DSL and then sprinkled distractingly through the code that actually does the work. I prefer the way Haskell does this- separate the type signature out onto at least a separate line rather than mashing the two different languages together. 3. Higher levels of abstraction tend to become very hairy in many type systems (although not all). This ends up just meaning that people who like types often restrict themselves (unconciously) to less abstract programming. They spot the time they're saving by avoiding some kinds of bugs, but they don't see the time they're wasting by being unable to talk at a higher level of abstraction. Another way this shows itself is that types are very rarely first class objects in strongly typed languages, making it very difficult to create code that operates on types, or understands types. 4. Type systems open up opportunities for type driven architecture astronauting, which is just yet another way you can go down an unproductive rabbit hole. There was an interesting study done on different teams solving problems with different languages. The differences of different teams within the same language was much bigger than between languages, but the team that made slowest progress (and without particularly having an unusually low number of bugs) was the team that leant the hardest into encoding everything in the type system. 5. Type systems encourage code generation build pipelines, which again slows iteration time and makes everyones life miserable. 6. Type systems reflect a incorrect model of the world - user input, network input, file system data is not typed. The misery that I've had with some web server frameworks that refuse to acknowlege that they don't know every possible thing that the web client might send them and are able to slot it into a predefined type. I think this is the same kind of error that we made with OO systems - thinking that we could fit the world into a predefined inheritance hierarchy. 7. Type systems encourage a static view of the world. The types of things can change under you, dynamically, (e.g. the structure of a table in a database), but in most typed languages you can't cope with that correctly without shutting down and deploying entirely new code. 8. Related to that, it's hard to imagine using a strongly typed language with the live image approach of smalltalk or sometimes used by lisp systems. This means that the popuarity of strongly typed languages is killing valuable and interesting approaches to building complex systems that emphasise observability, interaction and iteration as a way of understanding them. There are genuine advantages to typed languages, but many of the advantages touted as being unique to typed languages can be provided by advanced linting and IDEs (intellij was surprisingly capable on plain JS + jsdoc). You can also ameliorate some of the disadvantages of untyped languages while keeping the benefits by deliberately programming in a fail-fast way. I'm sure that type systems have their place. The research I've come across on empirical studies suggests that while there may be positive effects they are small, which does not at all mesh with the extreme partisanship I generally observe. Yes, type systems gain you something, but there seems much less awareness of what you lose.
  - tluyben2 6y ago
    I am positive about a lot of these points for the future. Especially the performance points; that's going forward fast. But yes, that's often pretty slow; not that bothered by it for my work though. Also, linters work well for statically typed languages too; I usually don't have to compile for 100s of lines of code. If the editor does not complain, it'll probably all work fine. Like I said; do what works for you , but I think at least a good mix will get more benefits. 6. People mention this more often, but I just don't see how that works; you cannot program without knowing what data you are getting. Sure the world is not typed, but at the moment you are going to use the data, it is typed; be it in your logic, head or actual types. Any webserver can go lower level and give you a ByteStream, but when you finish parsing that, you still have types. You might not know them upfront, so you use ByteStream for a bit, but once you know, you bake types and the world is nicer. Imho :) Not sure why that's a difference? 7. This is an issue where? I know it's Erlang domain, but microservices/docker/k8s/ci/cd/lambda/functions/.../all modern crap do this (redeploy, killing the previous instance(s)) with any code, always, including dynamically typed code. So sounds like a niche? 8. Agree with this; we should experiment and research these things and continue building them. I work with Lisp/Clojure as well and like it, I just miss types often. I never suggested it's all crap; I'm just looking where benefit comes from.
    - steveklabnik 6y ago
      A good post on point six: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-type-systems-are-not-inherently-more-open/ https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...
    - kybernetikos 6y ago
      > you cannot program without knowing what data you are getting Types almost always overconstrain. Each part of your code relies on some very specific properties of your data, yet most type systems end up restricting your function to only work with data that meets a whole bunch of other properties that your code doesn't actually care about.
      - 6y ago

tluyben2 6y ago

Agreed and we have to continue improving in every way; dynamic/static/hybrid, just saying that I have not seen this dynamic enlightenment in larger projects. I have only seen the pain of runtime errors that other (static language) teams never had. Sure, if you would-have-written a test for it, you wouldn't have had it either, but types rather force you to think about it while writing. So sure, you are 'done faster', but the fall-out, and again ofcourse YMMV, of having statically preventable bugs popping up in Sentry at 3 am in the morning with things you would've prevented (not necessarily directly by the types but you would've thought about it more because you had to define the types, which is I think what the parent poster here misses too; I just tend to think less and try more without types which, again for me, is a bad state, but ymmv) is not great. But sure, I am biased as my experience with statically typed langs has been good since I moved from asm/basic in the 80s to pascal/c (they were an improvement over asm/basic and my first experience with types, not saying you should use them now, or not).

kybernetikos 6y ago
The view I've heard expressed is that deep thought on a piece of code reduces bugs. Whether that takes the shape of religious TDD, rigorous proofs or detailed type design doesn't make such a lot of difference. I used to be fully bought into types, but I've since realised that they have a number of downsides that in many cases more than offset their benefits: 1. Ergonomic typesystems require a lot of work to happen at compile time and slow down the iteration time (one of the more important things for programming in my view). In my view, saving the source and seeing the result almost immediately in a browser is one of the big advantages of web development. 2. Types are almost always written in a second, much less powerful DSL and then sprinkled distractingly through the code that actually does the work. I prefer the way Haskell does this- separate the type signature out onto at least a separate line rather than mashing the two different languages together. 3. Higher levels of abstraction tend to become very hairy in many type systems (although not all). This ends up just meaning that people who like types often restrict themselves (unconciously) to less abstract programming. They spot the time they're saving by avoiding some kinds of bugs, but they don't see the time they're wasting by being unable to talk at a higher level of abstraction. Another way this shows itself is that types are very rarely first class objects in strongly typed languages, making it very difficult to create code that operates on types, or understands types. 4. Type systems open up opportunities for type driven architecture astronauting, which is just yet another way you can go down an unproductive rabbit hole. There was an interesting study done on different teams solving problems with different languages. The differences of different teams within the same language was much bigger than between languages, but the team that made slowest progress (and without particularly having an unusually low number of bugs) was the team that leant the hardest into encoding everything in the type system. 5. Type systems encourage code generation build pipelines, which again slows iteration time and makes everyones life miserable. 6. Type systems reflect a incorrect model of the world - user input, network input, file system data is not typed. The misery that I've had with some web server frameworks that refuse to acknowlege that they don't know every possible thing that the web client might send them and are able to slot it into a predefined type. I think this is the same kind of error that we made with OO systems - thinking that we could fit the world into a predefined inheritance hierarchy. 7. Type systems encourage a static view of the world. The types of things can change under you, dynamically, (e.g. the structure of a table in a database), but in most typed languages you can't cope with that correctly without shutting down and deploying entirely new code. 8. Related to that, it's hard to imagine using a strongly typed language with the live image approach of smalltalk or sometimes used by lisp systems. This means that the popuarity of strongly typed languages is killing valuable and interesting approaches to building complex systems that emphasise observability, interaction and iteration as a way of understanding them. There are genuine advantages to typed languages, but many of the advantages touted as being unique to typed languages can be provided by advanced linting and IDEs (intellij was surprisingly capable on plain JS + jsdoc). You can also ameliorate some of the disadvantages of untyped languages while keeping the benefits by deliberately programming in a fail-fast way. I'm sure that type systems have their place. The research I've come across on empirical studies suggests that while there may be positive effects they are small, which does not at all mesh with the extreme partisanship I generally observe. Yes, type systems gain you something, but there seems much less awareness of what you lose.
- tluyben2 6y ago
  I am positive about a lot of these points for the future. Especially the performance points; that's going forward fast. But yes, that's often pretty slow; not that bothered by it for my work though. Also, linters work well for statically typed languages too; I usually don't have to compile for 100s of lines of code. If the editor does not complain, it'll probably all work fine. Like I said; do what works for you , but I think at least a good mix will get more benefits. 6. People mention this more often, but I just don't see how that works; you cannot program without knowing what data you are getting. Sure the world is not typed, but at the moment you are going to use the data, it is typed; be it in your logic, head or actual types. Any webserver can go lower level and give you a ByteStream, but when you finish parsing that, you still have types. You might not know them upfront, so you use ByteStream for a bit, but once you know, you bake types and the world is nicer. Imho :) Not sure why that's a difference? 7. This is an issue where? I know it's Erlang domain, but microservices/docker/k8s/ci/cd/lambda/functions/.../all modern crap do this (redeploy, killing the previous instance(s)) with any code, always, including dynamically typed code. So sounds like a niche? 8. Agree with this; we should experiment and research these things and continue building them. I work with Lisp/Clojure as well and like it, I just miss types often. I never suggested it's all crap; I'm just looking where benefit comes from.
  - steveklabnik 6y ago
    A good post on point six: https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-type-systems-are-not-inherently-more-open/ https://lexi-lambda.github.io/blog/2020/01/19/no-dynamic-typ...
  - kybernetikos 6y ago
    > you cannot program without knowing what data you are getting Types almost always overconstrain. Each part of your code relies on some very specific properties of your data, yet most type systems end up restricting your function to only work with data that meets a whole bunch of other properties that your code doesn't actually care about.
    - 6y ago