4 ms·

Do you think it's possible for well used types to displace the need for some tests?

JohnBooty 5y ago
I'm not the person you're responding to, but I would say: "yeah, but it'd probably be bad practice." What would you be testing for, exactly? That `SomeClass#some_method` raises a `NoMethodError` when you pass it the wrong thing? You could test for that, but I don't think that would be a good use of code or your time. If A is passing the wrong things to B, your specs will fail anyway on that `NoMethodError` once B tries to actually do something with those improper argument(s). I suppose it could be useful in cases where you'd stubbed out B in your specs, but ehhhh.
- whakim 5y ago
  In my experience this here (usually some method unexpectedly getting passed `nil`) is the single biggest class of bugs in production Ruby applications. Also, what happens when you don't get a NoMethodError? Duck typing is an extremely common practice in Ruby, which means that you can easily run into situations where code "runs" but the output is nonsensical.
  - JohnBooty 5y ago
    In my experience this here (usually some method unexpectedly getting passed `nil`) is the single biggest class of bugs in production Ruby applications. Right, but how do tests help you here? Your specs aren't passing those unexpected nils, and if they were, your specs would fail on the resulting NoMethodError. Static analysis can find these potential problems in a statically typed language - I miss the days when Resharper would let me know about potential nulls/nils. That's a strong argument for static typing. But, this particular comment thread is about addressing this class of bug in Ruby via tests, and my vote would be generally no. Duck typing is an extremely common practice in Ruby, which means that you can easily run into situations where code "runs" but the output is nonsensical. I've been working with Ruby full time for years (admittedly, mostly boring CRUD Rails apps) and I've just never found this to be a problem. It obviously can happen, but I just don't see those name collisions ever happening.
    - whakim 5y ago
      > But, this particular comment thread is about addressing this class of bug in Ruby via tests, and my vote would be generally no. I'm pretty sure I started the comment thread ;). I think there's a couple of scenarios when a function gets some rogue input (e.g., a string instead of an integer) which then triggers a NoMethodError. I agree that tests don't really help you here - what are you testing, that your method correctly throws a NoMethodError? That being said, to combat this issue you often see dynamically typed codebases littered with scattershot validation, i.e. fail gracefully if you somehow wind up with an unexpected input. And then you often write tests for that. Static typing often obviates these tests, because presumably the rogue input has already been handled further up the call stack. Stubbing is the other issue; I think it happens much more frequently than you suggest that A is tested (stubbing B) and B is tested in isolation; or perhaps there's only one test of A which doesn't stub B, and that test doesn't happen to trigger the NoMethodError. So I think we're mostly agreeing, but I do think you get to write fewer tests overall. > It obviously can happen, but I just don't see those name collisions ever happening. It has happened to me (I think in the Rails context less often) but I agree it's not exactly common. It is deadly when it does, though.
- emodendroket 5y ago
  > I suppose it could be useful in cases where you'd stubbed out B in your specs, but ehhhh. You're talking like that's rare and not something people constantly do when writing tests in Ruby.
  - JohnBooty 5y ago
    It is of course super common. Unit tests should be stubbing out as much as possible/reasonable. The presumption is that we're talking about a well rounded test suite with both unit tests and integration tests, and the integration tests would indeed be catching the fact that A is passing entirely the wrong thing to B. Of course, this does mean we're leaning heavily on the test suite here. But, in any nontrivial application I hope that we would have a robust test suite, right? Otherwise we're going to have some problems whether we're statically or dynamically typed.
    - emodendroket 5y ago
      Over time I have come to view mock- and stub-heavy tests as next to worthless.
- abstract_put 5y ago
  My experience with a medium-size but long timespan project was the two things you mentioned a bit dismissively are actually pretty massive time sucks. People are terrible at stubbing, and refactoring without types is painful. The vast majority of my Ruby experience is within Rails, so maybe that's a contributing factor? Having types not line up at various boundaries (DB/API/reading from a file) is already a pain, but Ruby made it worse by having that bad data pass through many layers until it actually blows up somewhere far removed from the issue. I worked on very real bugs where e.g. a corner case lead to a date being deserialized as a string and then because the last few characters were numeric interpreted as a number so when treated as a date resolved as millis since epoch (or something similarly crazy). It took ~1 of those bugs for me to be convinced that I had no interest in dealing with those kinds of problems, and adding a 'Date' type means it fails in exactly the right place immediately and is a 2 second fix. I agree with you though - it'd be a silly test to write, so how do you get the correctness/robustness without either types or tests that look like they're effectively validating types?
  - joelbluminator 5y ago
    > Having types not line up at various boundaries (DB/API/reading from a file) is already a pain That's not really accurate - Postgres (for instance) types ARE mapped to Ruby types whenever you read something from the database, just like they are being mapped to Java types or any other language. I guess you mean you can have inadvertent type coercion where a Ruby string is saved into a PG numeric column or something? Anyway for super sensitive code (let's say payments/prices) you can work with DRY types or Sorbet or any other solution. As a rule it's quite rare that I actually see these problems. Can they occur? Sure. Do they happen so often I wish I was using C++? No. In fact I can't even say it happens more than once a year that I see this type of bug.
BurningFrog 5y ago
I came the other way. As in, tests displaced types. Did Java for 13 years. Then moved to Ruby and lost the type system I had leaned so hard on. After an adjustment period I got into the new groove of just testing all code, and I don't miss my hard typed days at all.

I'm not the person you're responding to, but I would say: "yeah, but it'd probably be bad practice." What would you be testing for, exactly? That `SomeClass#some_method` raises a `NoMethodError` when you pass it the wrong thing? You could test for that, but I don't think that would be a good use of code or your time. If A is passing the wrong things to B, your specs will fail anyway on that `NoMethodError` once B tries to actually do something with those improper argument(s). I suppose it could be useful in cases where you'd stubbed out B in your specs, but ehhhh.

whakim 5y ago
In my experience this here (usually some method unexpectedly getting passed `nil`) is the single biggest class of bugs in production Ruby applications. Also, what happens when you don't get a NoMethodError? Duck typing is an extremely common practice in Ruby, which means that you can easily run into situations where code "runs" but the output is nonsensical.
- JohnBooty 5y ago
  In my experience this here (usually some method unexpectedly getting passed `nil`) is the single biggest class of bugs in production Ruby applications. Right, but how do tests help you here? Your specs aren't passing those unexpected nils, and if they were, your specs would fail on the resulting NoMethodError. Static analysis can find these potential problems in a statically typed language - I miss the days when Resharper would let me know about potential nulls/nils. That's a strong argument for static typing. But, this particular comment thread is about addressing this class of bug in Ruby via tests, and my vote would be generally no. Duck typing is an extremely common practice in Ruby, which means that you can easily run into situations where code "runs" but the output is nonsensical. I've been working with Ruby full time for years (admittedly, mostly boring CRUD Rails apps) and I've just never found this to be a problem. It obviously can happen, but I just don't see those name collisions ever happening.
  - whakim 5y ago
    > But, this particular comment thread is about addressing this class of bug in Ruby via tests, and my vote would be generally no. I'm pretty sure I started the comment thread ;). I think there's a couple of scenarios when a function gets some rogue input (e.g., a string instead of an integer) which then triggers a NoMethodError. I agree that tests don't really help you here - what are you testing, that your method correctly throws a NoMethodError? That being said, to combat this issue you often see dynamically typed codebases littered with scattershot validation, i.e. fail gracefully if you somehow wind up with an unexpected input. And then you often write tests for that. Static typing often obviates these tests, because presumably the rogue input has already been handled further up the call stack. Stubbing is the other issue; I think it happens much more frequently than you suggest that A is tested (stubbing B) and B is tested in isolation; or perhaps there's only one test of A which doesn't stub B, and that test doesn't happen to trigger the NoMethodError. So I think we're mostly agreeing, but I do think you get to write fewer tests overall. > It obviously can happen, but I just don't see those name collisions ever happening. It has happened to me (I think in the Rails context less often) but I agree it's not exactly common. It is deadly when it does, though.
emodendroket 5y ago
> I suppose it could be useful in cases where you'd stubbed out B in your specs, but ehhhh. You're talking like that's rare and not something people constantly do when writing tests in Ruby.
- JohnBooty 5y ago
  It is of course super common. Unit tests should be stubbing out as much as possible/reasonable. The presumption is that we're talking about a well rounded test suite with both unit tests and integration tests, and the integration tests would indeed be catching the fact that A is passing entirely the wrong thing to B. Of course, this does mean we're leaning heavily on the test suite here. But, in any nontrivial application I hope that we would have a robust test suite, right? Otherwise we're going to have some problems whether we're statically or dynamically typed.
  - emodendroket 5y ago
    Over time I have come to view mock- and stub-heavy tests as next to worthless.
abstract_put 5y ago
My experience with a medium-size but long timespan project was the two things you mentioned a bit dismissively are actually pretty massive time sucks. People are terrible at stubbing, and refactoring without types is painful. The vast majority of my Ruby experience is within Rails, so maybe that's a contributing factor? Having types not line up at various boundaries (DB/API/reading from a file) is already a pain, but Ruby made it worse by having that bad data pass through many layers until it actually blows up somewhere far removed from the issue. I worked on very real bugs where e.g. a corner case lead to a date being deserialized as a string and then because the last few characters were numeric interpreted as a number so when treated as a date resolved as millis since epoch (or something similarly crazy). It took ~1 of those bugs for me to be convinced that I had no interest in dealing with those kinds of problems, and adding a 'Date' type means it fails in exactly the right place immediately and is a 2 second fix. I agree with you though - it'd be a silly test to write, so how do you get the correctness/robustness without either types or tests that look like they're effectively validating types?
- joelbluminator 5y ago
  > Having types not line up at various boundaries (DB/API/reading from a file) is already a pain That's not really accurate - Postgres (for instance) types ARE mapped to Ruby types whenever you read something from the database, just like they are being mapped to Java types or any other language. I guess you mean you can have inadvertent type coercion where a Ruby string is saved into a PG numeric column or something? Anyway for super sensitive code (let's say payments/prices) you can work with DRY types or Sorbet or any other solution. As a rule it's quite rare that I actually see these problems. Can they occur? Sure. Do they happen so often I wish I was using C++? No. In fact I can't even say it happens more than once a year that I see this type of bug.

BurningFrog 5y ago

I came the other way. As in, tests displaced types. Did Java for 13 years. Then moved to Ruby and lost the type system I had leaned so hard on. After an adjustment period I got into the new groove of just testing all code, and I don't miss my hard typed days at all.