4 ms·

Here are 4 real-world bug reports, in their entirety. Anybody care to rewrite them as automated test cases for me? * "Try it Now" button sometimes drops bel

• diN0bot 17y ago
  different programming domains have different problems... wrt this list: * if GUIs were easier to write tests for, then you wouldn't have problems writing tests for the first 3 on your list. * timing issues and edge cases are one of the reasons i value tests, assertions and strong design so much: debugging is hard. * the last one sounds like a feature that hasn't been written, not a bug.
  • joe_the_user 17y ago
    timing issues and edge cases are one of the reasons i value tests, assertions and strong design so much: debugging is hard. It always seems like when you scratch the TDD surface, "writing tests first lets you write bug-free software" becomes "tests are an important tool along with good design, assertions, etc". I agree with the later but I don't see how it's different earlier methodologies - and earlier methodologies certainly didn't let you write bug-free code.
• lallysingh 17y ago
  Check out Selenium.
• vlisivka 17y ago
  > * "Try it Now" button sometimes drops below the "Learn More" button in FireFox Use JavaScript or Selenium for that Test case. Get coordinates of buttons (getBoundingClientRect()) and compare them. Execute this test case on tons of different configurations. > * Capitalization inconsistent in top nav Easy to test. Also add spell checking along the way. > Popup blockers occasionally stop the App window from opening properly Selenium? > Wierd timeout errors clumped around 1:00 am every few nights You always can shift clock automatically (on Linux). > Accounting needs to be able to call up weekly income reports If it is missing feature, then no test case is need until you will start work on it. PS. If you want to automate your manual testing, you will look for ways to do that. If not, you will look for problems, which are prevents you from doing that. I wrote example how to execute HTMLUnit tests at build stage because I want that. Almost everybody else says that this is not possible. ( http://vlisivka.pp.ua/en/integration_testing_with_maven_testng_htmlunit_embeded_tomcat6 http://vlisivka.pp.ua/en/integration_testing_with_maven_test... )
  • InclinedPlane 17y ago
    > Wierd timeout errors clumped around 1:00 am every few nights You always can shift clock automatically (on Linux). Obviously a problem of this sort is not due to the mere numerical value of the time but rather some daily scheduled process that runs around 1am (perhaps a database backup) which interferes with the web app in some way. This sort of thing can be simulated but it's a non-trivial task and it's probably better to apply those efforts in other ways (by diagnosing the problem, putting in place a performance monitoring system, buying more hardware, etc.)
    • vlisivka 17y ago
      Typical QA cycle: * find problem; * identify root cause of the problem; * write automated, semi-automated or manual test case, which will highlight problem; * fix problem. If you will not use test cases at development and staging stages, then you will just use your production environment for testing purposes. If your downtime cost you nothing - don't bother with test cases. In our case, we had similar problem, so I created very simple test case in shell in less than 2 hours. We spent more than year until it was fully fixed.
• fexl 17y ago
  Yes, bugs still need to be reported, and not all of them yield to automated tests. Nevertheless I do concur with the spirit of the article. I write very demanding test cases, often before anything goes wrong but certainly after it does, even for non-deterministic code (e.g. I'll fork a thousand processes in an automated test and invoke random busy loops here and there to encourage contention.) I don't do a lot of GUI though, other than HTML interfaces, which are fairly amenable to some automated testing. One time I broke a sign-up form due to a misnamed field, so now I have an automated test for that. For things like "Capitalization inconsistent in top nav", that's a prime example of why you still need to keep track of bugs. Even if you're working alone, just keep a "TODO" list. You can keep the TODOs in a note-taking program such as Tomboy, but better yet, put them right in the code. Then don't release until "grep -r TODO" comes up clean. (If you want to postpone a TODO change it to LATER.) It might be difficult to create a regression test for that one, and it probably wouldn't be cost effective, but I suppose could have a snippet of code that iterated over the list of navigation items and tested the case. Again, probably not worth it. Better just to stomp that TODO and move on. I can't say I have the silver bullet for the examples you list, but the important thing is to keep manufacturing better bullets with higher silver content. I figure I gotta try at least.
  • deleted 17y ago
    [deleted]
• jerf 17y ago
  Whenever the unit testing comes up, people always cite GUI issues as untestable. But it should be pointed out that in Fred Brook's usage of the terms, this is an accidental problem, not an essential one. GUIs are hard to test not because GUIs are hard to test, but because your GUI doesn't contain provisions for being tested. Nothing stops a GUI from being very queryable. Nothing stops you from being able to query the locations of two widgets and making assertions based on those locations. Nothing stops you from getting text back out, or verifying fonts, or verifying the lack of overlap, or lack of popups... except that GUIs are written to be opaque monoliths, graveyards of data. This is correctable, but not by the end programmer. So it is true that GUIs are untestable, but this doesn't have to be true. But it's going to take a conscious effort by QT or GTK or someone to make their GUIs testable before anything will change. (I acknowledge your other issues; don't agree with all of them but the GUI issue is what I'm passionate about.)
  • Silhouette 17y ago
    > So it is true that GUIs are untestable, but this doesn't have to be true. I respectfully disagree. You can't codify aesthetics or usability, and even if you could, the return on investment would be abysmal.
    • jerf 17y ago
      "Aesthetics" or "usability" as a whole? Sure, of course not. No more than you can codify "works correctly" as a whole. Codify huge swathes of it, though? Absolutely. Assert that correct fonts and font sizes are used. Assert that color schemes are correctly followed, or that certain widgets always have attached desirable behaviors, assert that all widgets have keyboard shortcuts, assert that those keyboard shortcuts are coherent (CTRL-F is not assigned to three actions), assert that using the keyboard shortcut has the same behavior as whatever it is the shortcut to, assert that automated validation of appropriate accessibility guidelines hold (ie, for web interfaces assert that a Bobby run over your interface holds no surprises), and so on. Sure, you can't "automate usability" but if you just give up at that point you'll miss out on a lot of things you can automate... or could automate if GUIs let you, getting back to my original point.