4 ms·
Overall agree with the OP, but wanted to provide a minor objection, or maybe clarification. I've been writing software professionally since '98, so I'm a littl
by lbayes 5y ago
Overall agree with the OP, but wanted to provide a minor objection, or maybe clarification.
I've been writing software professionally since '98, so I'm a little greener than @tbray. During my career, I've almost entirely been focused on building UI's, though I have done quite a lot of full-stack and embedded work.
I got infected by testing in ~2003-ish when I started reading Kent Beck and Martin Fowler.
FWIW - I had a 4 year stint at Google (10 years ago), where I led the team that built YouTube on TV.
Our product on the TV team certainly had it's own set of challenges, but quality wasn't one of them. By the time I left, we had over 2,400 unit/component tests that would run in ~30 seconds. This single application ran on many hundreds of SKUs (PS3, PS4, XBox 360, XBox One, Wii U, Smart-ish TV's, Set to boxes, Over the top boxes, Cable boxes, Roku, Chromecast, Blue Ray Disk Players, crazy, crazy stuff).
A given engineer could set up a file system watcher, that would run a much faster subset of tests in under a second. We deployed this application to production every week (which was revolutionary back then), and one of the years I was there, we had a single P0 production incident for the entire year.
We changed out the underlying UI framework 3 times over 3 years. We did this incrementally with side-by-side running frameworks and while delivering to production every week.
It would have been utterly impossible for us to accomplish this work without the testing infrastructure that we had. And by testing infrastructure, I mean:
1) Karma (or Mocha or Jasmine) for unit / component tests
2) Manual testing on problematic machines where automation was prohibitively expensive
The OP-linked twitter graphics are also triggering me.
Every time I've seen "Integration Tests" in a production environment, they've had the following characteristics:
* Slow: 10-40 minutes
* Brittle: Break mysteriously while working (if you can even run them from your workstation)
* Time-consuming: These tests chew a lot of time to author, and much, much more time to maintain
* Flaky: This is the deal-killer for me. They fail randomly in CI, and chew hours of the team's time trying to troubleshoot. You'll know you're here when you see random timeouts in your tests.
The main insight I came here to share is this:
There's nothing magical about UI. It's just software. It's probably (IMO) one of the more complicated areas of software engineering, but that's because of what UI is generally made of:
UI tool kits usually represent the user interface as a long-lived, mutable, wide and deep tree of interdependent state with a multitude of message passing schemes and needless hooks to global references everywhere.
Despite this, it's almost always possible to:
1) Disconnect and/or minimize the hooks to global state
2) Clarify message passing rules/practices
3) Break down the tree into manageable chunks
Once these ideas are in place, we can incrementally test UI as it's being built.
As an example for why we don't need "Integration Tests", I've often written "component" tests that instantiate a relatively complex node of my UI tree, poke the data, and verify some substructure was updated. This can and should be done without random timeouts and global nonsense.
We may not always be able to test our root node or main loop, but jeebus people, make those two things tiny and test everything else.
It's not magic, and it's super valuable.
It does admittedly get me worked up when people notice that UI is rarely tested, and then make claims that it probably shouldn't be.
It's not even mainly about the maintenance. Writing tests first(ish?) dramatically improves the design of my otherwise, much less clean code. Testing as part of development is an extremely powerful design tool that has ancillary benefits for maintenance and reliability.
As the OP said, these tests have to run incredibly fast (< 1 second at author time) and additionally, the unit under test cannot have a bunch of tentacles reaching out to manipulate global state.
Yes, almost all UI tool kits as they exist today are horrible to work with, but that's no excuse to just abandon the primary practice that almost makes our work bearable.
[Update: formatting]