4 ms·

Speaking from experience, the promise of completing tests in 1 minute or less is very appealing. Indeed. There is a qualitative difference between tests comple

Speaking from experience, the promise of completing tests in 1 minute or less is very appealing. Indeed. There is a qualitative difference between tests completing fast enough that a dev is OK with just waiting a few seconds for the results and tests becoming a "go do something else for a while" background activity. There is also a qualitative difference between tests running fast enough that if something does go wrong the code can often be backed out before it interferes with anyone else and tests that run slowly enough that the CI system can't keep up with merges from the team and changes start piling up that then need sorting out somehow every time a set of tests does fail.

punkpeye 4y ago
Another thing to consider is that most engineers do not run all integration tests locally before committing them, simply because they take long time, i.e. A typical workflow is: commit changes -> raise a PR -> wait for all tests to complete -> [if tests fails, go back to step 1] This hugely impacts developer ability to switch context, because they need to stash code, switch branches, etc. If all integration tests could complete in pre-push commit hook (which is reasonable if they take less than a minute), then we would dramatically reduce the context switching involved in day to day engineering.
- Silhouette 4y ago
  I agree. That's another reasonable point. If I were in your position I would absolutely be emphasizing these kinds of qualitative benefits in the marketing even if your pricing model is actually very quantitative in the end. Anyone who runs the kind of engineering group you're currently aiming at won't have to think very hard to get an idea of the quantitative scale of the benefits they might enjoy as a result. Ideally you want authorising a few (or more than a few) thousand bucks a month to be a quick and obvious decision for them. "We provide a simple tool that enables radically better development processes with likely ROI between 10x and 100x" is a decent elevator pitch to people like VP Engineering and CFO who are going to end up making a purchasing decision.

punkpeye 4y ago

Another thing to consider is that most engineers do not run all integration tests locally before committing them, simply because they take long time, i.e. A typical workflow is: commit changes -> raise a PR -> wait for all tests to complete -> [if tests fails, go back to step 1] This hugely impacts developer ability to switch context, because they need to stash code, switch branches, etc. If all integration tests could complete in pre-push commit hook (which is reasonable if they take less than a minute), then we would dramatically reduce the context switching involved in day to day engineering.

Silhouette 4y ago
I agree. That's another reasonable point. If I were in your position I would absolutely be emphasizing these kinds of qualitative benefits in the marketing even if your pricing model is actually very quantitative in the end. Anyone who runs the kind of engineering group you're currently aiming at won't have to think very hard to get an idea of the quantitative scale of the benefits they might enjoy as a result. Ideally you want authorising a few (or more than a few) thousand bucks a month to be a quick and obvious decision for them. "We provide a simple tool that enables radically better development processes with likely ROI between 10x and 100x" is a decent elevator pitch to people like VP Engineering and CFO who are going to end up making a purchasing decision.