4 ms·
My pet reason: Your framework doesn't solve any of the genuinely hard problems that annoy us when we're trying to test stuff. Lots of people have written varia
by jarrett 15y ago
My pet reason: Your framework doesn't solve any of the genuinely hard problems that annoy us when we're trying to test stuff.
Lots of people have written variations on assertions and mocks. But where's the full-stack test harness for a Rails app with heavy JS usage, that runs at a reasonable speed, gives us useful traces on the client and server, and doesn't break when I slightly rework my HTML?
Or, where's the tool for testing integration with REST services outside your control, that doesn't depend on the real service most of the time, but can be told to run against the real service as a sanity check when you want to? (Like Ruby's VCR, but better.)
Or, how about a test framework that accurately emulates mobile browsers and automatically?
I'm not getting on anyone's case for not having written this stuff. Because hey, I'm just as guilty of not solving these problems as is the rest of the community. But if you want to make a testing tool that people will absolutely love, solve some of these problems really elegantly. (I know, easier said than done. And yes, some tools have solved some of these problems to some extent. But I'd consider each of my examples to be an ongoing pain point for developers.)
- Fluxx 15y ago> But where's the full-stack test harness for a Rails app with heavy JS usage, that runs at a reasonable speed, gives us useful traces on the client and server, and doesn't break when I slightly rework my HTML? If you're talking Python, Lettuce does this is Python, there also is Splinter. You can run selenium with a headless firefox quite easily with Xvfb. If you're talking rails, Capybara has drivers for Webkit (QI), selenium and even PhantomJS. Their speed is variable, but it's not too bad. On the "HTML changes break my tests," I'd argue that A) you should write your front end tests not tied to the HTML if at all possible. Refer to elements by semantic names, not XPath or CSS selectors. And be smart about your steps. Make your submit_form() method actually look for a submit button in more than one way. B) you shouldn't have more than a handful of full stack frond end tests. They're hard to write, slow, and don't give much help telling you WHAT went wrong. Most of the time the ROI is only there for verifying a few user paths, but not much more than that.
- benjiweber 15y agoThe Page Object Pattern can help reduce the brittleness of automated UI tests. http://code.google.com/p/selenium/wiki/PageObjects http://code.google.com/p/selenium/wiki/PageObjects
- saurik 15y agoSo, while I agree that it isn't right to "get on anyone's case for not having written this stuff", I do think it is reasonable to get on peoples' cases when they do release things if they didn't solve the "real problems": my message to people about to release a test framework would be "do you solve a real problem? if not, just keep it to yourself: please do not pollute the keyword space"; the corollary to this is that if you spent a weekend on it, it almost certainly does not solve a "real problem" ;P.
- aaronblohowiak 15y agoLet a hundred flowers bloom.
- saurik 15y agoIf only we could then still find the rare ones. :( This same attitude is what leads to hundreds of "articles" that are just "I read an article on some other website and summarized it on my website", making it impossible to find any of the actually useful or new content on any new subject. :(