4 ms·

Shippable quality to me is a full regression test. That's awful hard to do in the few days a 2 week sprint allows.

nradov 8y ago
Your full regression test suite needs to be 100% automated and fast enough to execute within a couple days. If you don't have that then you have a bigger problem that goes beyond any particular development methodology. A proper Scrum definition of done requires full test automation for every user story. You can't merge code to the release branch or accept the user story until all the automated tests (unit, functional, integration, system, performance, etc) are working.
- Clubber 8y ago
  I assume you don't mean unit tests, which should never be a replacement for full regression testing. Ya since that is essentially a deal breaker, I wonder why all the scrum consultants fail to mention that. Automated full regression is very expensive to develop and maintain. I think a lot of people who implement scrum don't realize that they are sacrificing significant levels of testing when they implement it without a testing automation team. To me, that's just another reason to ditch scrum.
  - nradov 8y ago
    I mean all tests — unit, functional, performance, everything — necessary to verify that a particular product build meets your customer's quality standards. Automated full regression testing is far cheaper than manual testing. If it seems too expensive then you're probably doing it wrong or failing to account for full lifecycle costs. This is absolutely not a reason to ditch Scrum. In fact Scrum is conducive to good quality practices.
- tootie 8y ago
  Scrum does not require automation at all. Generally speaking, it should, but full automation is folly and dependence on automation is an anti-pattern. You simply can't test everything without eyes. Usually when you automate as soon as something is built, you tend to automate what it does instead of what it's supposed to do. You're also trying to hit a moving target and will end up spending a lot of time maintaining your tests instead of actually testing. There needs to be a balance of automation and manual testing.
  - nradov 8y ago
    Full automation is not at all folly. Manual testing doesn't scale and can't practically be done on large, complex products if you want to release on a rapid cadence while keeping costs reasonable. The only two types of testing you can't really automate are exploratory testing and usability (UX). Exploratory testing should be done concurrent with product coding, and then the tests that seem useful can be scripted as a permanent part of the test automation suite. Usability testing should be done either on a separate branch, or on the release branch with a feature toggle. It's essentially part of requirements analysis and not part of qualifying a particular build for release to customers. While Scrum doesn't call for any particular testing approach, in practice it's almost impossible to do Scrum effectively on a large product with multiple agile teams unless you have a very high level of automation. Otherwise development just bogs down and you end up with nonsense like "hardening sprints" (basically waterfall in disguise).
tootie 8y ago
It's really not possible to take stories from dev to QA, to remediation to release within a sprint without having idle cycles of either developers or testers. It's what make Kanban more appealing. The sprint boundaries can be useful guidelines, but I'd never hold anyone's feet to the fire over sprint commitments. I don't care about carryover so long as we have good rolling velocity.
- nradov 8y ago
  It's certainly possible to take stories all the way through in a single cycle. I've done it many times and seen others do it as well. There's nothing wrong with having some idle cycles. In order to maximize economic value you should focus on the cost of delay rather than insisting on 100% productivity.

Your full regression test suite needs to be 100% automated and fast enough to execute within a couple days. If you don't have that then you have a bigger problem that goes beyond any particular development methodology. A proper Scrum definition of done requires full test automation for every user story. You can't merge code to the release branch or accept the user story until all the automated tests (unit, functional, integration, system, performance, etc) are working.

Clubber 8y ago
I assume you don't mean unit tests, which should never be a replacement for full regression testing. Ya since that is essentially a deal breaker, I wonder why all the scrum consultants fail to mention that. Automated full regression is very expensive to develop and maintain. I think a lot of people who implement scrum don't realize that they are sacrificing significant levels of testing when they implement it without a testing automation team. To me, that's just another reason to ditch scrum.
- nradov 8y ago
  I mean all tests — unit, functional, performance, everything — necessary to verify that a particular product build meets your customer's quality standards. Automated full regression testing is far cheaper than manual testing. If it seems too expensive then you're probably doing it wrong or failing to account for full lifecycle costs. This is absolutely not a reason to ditch Scrum. In fact Scrum is conducive to good quality practices.
tootie 8y ago
Scrum does not require automation at all. Generally speaking, it should, but full automation is folly and dependence on automation is an anti-pattern. You simply can't test everything without eyes. Usually when you automate as soon as something is built, you tend to automate what it does instead of what it's supposed to do. You're also trying to hit a moving target and will end up spending a lot of time maintaining your tests instead of actually testing. There needs to be a balance of automation and manual testing.
- nradov 8y ago
  Full automation is not at all folly. Manual testing doesn't scale and can't practically be done on large, complex products if you want to release on a rapid cadence while keeping costs reasonable. The only two types of testing you can't really automate are exploratory testing and usability (UX). Exploratory testing should be done concurrent with product coding, and then the tests that seem useful can be scripted as a permanent part of the test automation suite. Usability testing should be done either on a separate branch, or on the release branch with a feature toggle. It's essentially part of requirements analysis and not part of qualifying a particular build for release to customers. While Scrum doesn't call for any particular testing approach, in practice it's almost impossible to do Scrum effectively on a large product with multiple agile teams unless you have a very high level of automation. Otherwise development just bogs down and you end up with nonsense like "hardening sprints" (basically waterfall in disguise).

tootie 8y ago

It's really not possible to take stories from dev to QA, to remediation to release within a sprint without having idle cycles of either developers or testers. It's what make Kanban more appealing. The sprint boundaries can be useful guidelines, but I'd never hold anyone's feet to the fire over sprint commitments. I don't care about carryover so long as we have good rolling velocity.

nradov 8y ago
It's certainly possible to take stories all the way through in a single cycle. I've done it many times and seen others do it as well. There's nothing wrong with having some idle cycles. In order to maximize economic value you should focus on the cost of delay rather than insisting on 100% productivity.