4 ms·
I think the biggest problem with testing in Rails is the strange ideology about it. Most of the Rails apps I've code reviewed have lots of tests of business lo
by grandalf 11y ago
I think the biggest problem with testing in Rails is the strange ideology about it.
Most of the Rails apps I've code reviewed have lots of tests of business logic that are tightly coupled to ActiveRecord and are (as a result) very slow.
One way to improve upon this might be to move from a test-oriented approach to a contracts approach where domain objects obey contracts.
It seems that this is the actual intuition behind the "declarative" associations in ActiveRecord, yet so many tests simply test what has been declared, as if it can't entirely be believed.... Does this model actually have a working input validation? Does it have working associations? If this is in doubt there are bigger problems.
Also, most apps use relational, hierarchical data (which is actually just a graph), which we declare via AR associations... models contain a mix of logic and data plumbing, and thus persistence layer concerns tremendously complicate testing simple graph oriented intuitions about our data, which could also be stated declaratively and enforced contractually instead of via the typical "callback hell" approach. So we stub callbacks and mock adjacent graph nodes and tell ourselves falsely that our "model" is well tested.
The complexity in testing most apps lies in setting up app state to test edge cases in the algorithms. There have been lots of workarounds to try to skip this or make it faster, but in the end the abstraction is the problem.
I want to be able to create simple data, such as a few hash literals, and test the flow of data through the business logic (algorithms) of my code without having to wait 60 seconds for the app to boot.
Ruby is not all that slow a language. I think a reasonable test suite for a large app should run in under 2-3 seconds. But Rails apps end up testing ActiveRecord, template rendering, and all sorts of other code that is already well tested and should be assumed to work by the test suite, but generally can't be because of the tight coupling mentioned above.
One other example: Templates are essentially functions that output HTML, but since they have ill-defined input requirements, there is not a simple way to throw valid input at them and verify correct output other than simulating the entire server request/response, yet most programmer errors will be in the implementation of the template, not in the boilerplate controller code. Yet millions of hours of CPU time (and programmer time) are burned testing these the hard way.
- ahuth 11y agoRegarding the testing of views: If using RSpec, there are "view" tests that test rendering the html in isolation. You specify the instance variables (AKA input) that the template is expecting, and you test the output. No server request/response is involved. See https://www.relishapp.com/rspec/rspec-rails/v/2-0/docs/view-specs/view-spec https://www.relishapp.com/rspec/rspec-rails/v/2-0/docs/view-...
- grandalf 11y agoGlad to see that exists... very cool.
- deleted 11y ago[deleted]
- eldavido 11y agoAgree re: methodology. There are really two problems in the model layer of a database-based app: - Moving data into and out of the database, the "Data Access Layer" - How the model objects interact and behave once moved into memory, the "Domain Model"; this is pure OO and is amenable to fast, in-memory-only testing. Rails treats these two as a single layer. On a project I'm working on now, we separated them into two distinct layers, which allows 1-2ms test execution in the model layer. We do test persistence, and it takes some time, but we can be a lot sparser with these slow, heavy tests. In the end I think the choice of approach depends on how complex the domain model is. My domain has almost 100 objects arranged in various ways, putting us over the threshold where persistence should be treated as its own concern (vs. bundling it into the model). I suspect a lot of apps are better done via the Rails approach as it's much simpler, if slower to test.
- grandalf 11y ago> I suspect a lot of apps are better done via the Rails approach as it's much simpler, if slower to test. I have been inclined to agree with this, but lately have begun to wonder. I think the relational model (and document model) lead to the need for lots of manual traversal/management of the graph/hierarchy of a domain model. And, in an OOP sense, models have behavior (and of course state). But behavior involves mutating the state of one domain object and others, via their public API (behaviors). Yet most of the so-called behaviors are actually just adding/changing nodes/edges to the graph, which makes the model's main behavior data validation. So we have this elaborate, tightly-coupled system that pretty much only does data validation, but via the elaborate mechanism of object behavior and internal state. The typical test suite tests the propagation of this data through the system and verifies that the proper object state has been mutated/persisted, since the proper functioning of the system is easy to doubt.
- eldavido 11y agoEmail me if you want to discuss this further. My email's in my profile. I came from using Rails to doing DDD-style C#/.NET development. It feels very "enterprise" but then, there are some very good lessons I've learned, one being how to write fast code-only unit tests.
- hinkley 11y agoWe used factory girl to alleviate some of this on the rails project I participated in. It helped but we still had a problem every time a new required field was added to the data model. In practice because of the issue of fixing all the existing tests, the cost of adding a new field went up logarithmically. Honestly if you can't eliminate the persistence layer, I'd rather see DB vendors put more time and effort into making in memory tables work. The Java people (hibernate) swap out the store for an in-memory one to speed things up, but I'd rather see a Postgres instance with heavy write-back caching for testing.
- MichaelGG 11y agoPut the data files on a RAM disk?
- grandalf 11y agoIn the code I've reviewed, it's not the speed of the db (mysql vs sqlite in-memory, for example), it's the overhead of ActiveRecord itself... tons of ruby object creation, GC, etc...
- hinkley 11y agoI think we were down to about 80ms per test on that one. I've since had projects with 9ms per test and it's hard to go back to thinking 80ms is 'good'.