6 ms·

Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. Like Smalltalk, they're forced to do this to catch the errors tha

Ruby's community has an obsessive culture towards unit testing, that could impact bug levels. Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. So then, could the argument be made that less stringent type annotation hurts code quality by removing effort from unit testing?

robbrit 7y ago
> they're forced to do this Figuratively speaking. None of the languages they test enforce any unit testing. What I'm suggesting is that since the Ruby community tends to be more obsessive about testing, that might lead to an improvement in code quality when comparing against other similar languages like PHP or JS which are less obsessive about testing. It's something you might have to control for to see the impact. Type systems are a completely different factor; you'd have to control for that variable too to determine the impact of it. My experience with both systems makes me predict that static typing would improve code quality, but if you want to actually be scientific you'd want to add that factor to the statistical analysis to see if the data support that hypothesis. Anecdotes like personal experience don't count.
- stcredzero 7y ago
  Figuratively speaking. None of the languages they test enforce any unit testing. The trivial type-mistake "Unhandled Exceptions" in Smalltalk would leak out more often than was comfortable, if the project didn't have a good means of testing to catch those. I suspect that has something to do with the Ruby community being more obsessive about testing.
  - igouy 7y ago
    > "Unhandled Exceptions" in Smalltalk ? MessageNotUnderstood
    - stcredzero 7y ago
      Right. This, and many others, would result in an "Unhandled Exception" notifier dialog in many versions of Smalltalk. I used to work for the company acting as the vendor of 2 (3?) commercial Smalltalks, also doing consulting work for others.
gnaritas 7y ago
No, the behavioral test [1] one writes cannot be replaced by type checks nor does a type system reduce the need to write tests if you're doing testing correctly. [1] https://twitter.com/unclebobmartin/status/1135894370165710848 https://twitter.com/unclebobmartin/status/113589437016571084...
Zak 7y ago
Python is a language fundamentally similar to Ruby, that has no static type checking, that's used in similar application domains, etc.... It does not have as strong a culture of unit testing as Ruby, and falls above the line (by a statistically insignificant amount) in figure 5 while Ruby is below (by a statistically significant amount). It does not appear that the paper took automated testing into account at all. Whether greater use of automated testing in Ruby is responsible for the lower defect rate observed here relative to Python would definitely be an interesting topic for another paper. Clojure is also dynamically typed by default and below the line. It does not have Ruby's strong unit testing culture, but it does have a design that encourages functional programming, which proponents argue reduces sources of error. It also has a culture of REPL-driven development that arguably fills some of the role of automated testing.
dragonwriter 7y ago
> Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. Very few of the unit tests I've written for Ruby/Python catch errors that would be caught by typical type system use in C/Java/etc. (Haskell, maybe, but...) The main thing I miss in dynamic languages vs. static is the more powerful automatic testing possible when test specs can leverage type information (e.g., QuickCheck/ScalaCheck), not an absence of need for testing.
- joshlemer 7y ago
  I've only ever done property based testing in Scala, but I thought it was pretty popular in dynamic languages like clojure and erlang. What would prevent it from working in Ruby/Python?
  - dragonwriter 7y ago
    Without type information already in the code (for parameters, etc.) there would be a lot more overhead in setting it up; it's possible, but higher marginal friction in the ideal case. Plus, AFAIK, no one has written the libraries, making it even higher marginal friction in practice even more than than is true in the ideal case.
    - ramchip 7y ago
      Type information generally isn’t used when doing property testing in Elixir, and Python has a very mature library too (hypothesis). If there’s a difference in popularity, it could just be culture. The Erlang/Elixir community is rather fanatical about reliability.
      - dragonwriter 7y ago
        Hypothesis uses the same type annotations as Python's optional typechecker as a tool to select appropriate input data generators without additional test-specific description. That's a fairly common feature of property-based testing tools; I don't know for sure, but I wouldn't be surprised to find that hypothesis and mypy tend to be used in the same segments of the Python community.

robbrit 7y ago

> they're forced to do this Figuratively speaking. None of the languages they test enforce any unit testing. What I'm suggesting is that since the Ruby community tends to be more obsessive about testing, that might lead to an improvement in code quality when comparing against other similar languages like PHP or JS which are less obsessive about testing. It's something you might have to control for to see the impact. Type systems are a completely different factor; you'd have to control for that variable too to determine the impact of it. My experience with both systems makes me predict that static typing would improve code quality, but if you want to actually be scientific you'd want to add that factor to the statistical analysis to see if the data support that hypothesis. Anecdotes like personal experience don't count.

stcredzero 7y ago
Figuratively speaking. None of the languages they test enforce any unit testing. The trivial type-mistake "Unhandled Exceptions" in Smalltalk would leak out more often than was comfortable, if the project didn't have a good means of testing to catch those. I suspect that has something to do with the Ruby community being more obsessive about testing.
- igouy 7y ago
  > "Unhandled Exceptions" in Smalltalk ? MessageNotUnderstood
  - stcredzero 7y ago
    Right. This, and many others, would result in an "Unhandled Exception" notifier dialog in many versions of Smalltalk. I used to work for the company acting as the vendor of 2 (3?) commercial Smalltalks, also doing consulting work for others.

gnaritas 7y ago

No, the behavioral test [1] one writes cannot be replaced by type checks nor does a type system reduce the need to write tests if you're doing testing correctly. [1] https://twitter.com/unclebobmartin/status/1135894370165710848 https://twitter.com/unclebobmartin/status/113589437016571084...

Zak 7y ago

Python is a language fundamentally similar to Ruby, that has no static type checking, that's used in similar application domains, etc.... It does not have as strong a culture of unit testing as Ruby, and falls above the line (by a statistically insignificant amount) in figure 5 while Ruby is below (by a statistically significant amount). It does not appear that the paper took automated testing into account at all. Whether greater use of automated testing in Ruby is responsible for the lower defect rate observed here relative to Python would definitely be an interesting topic for another paper. Clojure is also dynamically typed by default and below the line. It does not have Ruby's strong unit testing culture, but it does have a design that encourages functional programming, which proponents argue reduces sources of error. It also has a culture of REPL-driven development that arguably fills some of the role of automated testing.

dragonwriter 7y ago

> Like Smalltalk, they're forced to do this to catch the errors that would've been caught by type annotation. Very few of the unit tests I've written for Ruby/Python catch errors that would be caught by typical type system use in C/Java/etc. (Haskell, maybe, but...) The main thing I miss in dynamic languages vs. static is the more powerful automatic testing possible when test specs can leverage type information (e.g., QuickCheck/ScalaCheck), not an absence of need for testing.

joshlemer 7y ago
I've only ever done property based testing in Scala, but I thought it was pretty popular in dynamic languages like clojure and erlang. What would prevent it from working in Ruby/Python?
- dragonwriter 7y ago
  Without type information already in the code (for parameters, etc.) there would be a lot more overhead in setting it up; it's possible, but higher marginal friction in the ideal case. Plus, AFAIK, no one has written the libraries, making it even higher marginal friction in practice even more than than is true in the ideal case.
  - ramchip 7y ago
    Type information generally isn’t used when doing property testing in Elixir, and Python has a very mature library too (hypothesis). If there’s a difference in popularity, it could just be culture. The Erlang/Elixir community is rather fanatical about reliability.
    - dragonwriter 7y ago
      Hypothesis uses the same type annotations as Python's optional typechecker as a tool to select appropriate input data generators without additional test-specific description. That's a fairly common feature of property-based testing tools; I don't know for sure, but I wouldn't be surprised to find that hypothesis and mypy tend to be used in the same segments of the Python community.