6 ms·

How One Jira Ticket Made My Employer $1MM/Month: 7 Metrics That Actually Matter

66 points by stickhandle 9y ago

foobaw 9y ago
Interesting article - most of these metrics should be learned on the job for a lot of product managers.
lloydde 9y ago
2016
tyfon 9y ago
At first I thought the ticket was going to be "stop using JIRA/Confluence". There is so much time wasted on this system where I work it's amazing. Some people sit all day updating JIRA with all kind of crap instead of actually doing their job. At this point I'm starting to get the same feeling towards it as towards SAP.
- bdcravens 9y ago
  I think Jira's knobs and settings can get in the way, but I've found it to be fairly effective when used in a focused way (and recent updates seem to have optimized for that)
  - taeric 9y ago
    Like any tool for managing the health of something, it all depends on the quality of the data in the system. What I have not seen, is evidence that it helps improve the data. So, if you have someone that is good at keeping the data healthy, they can make the tool look amazing. The tool doesn't really help people get there, though. I don't think there is a universal answer here, yet. :( I would love to be proven wrong.
  - foobarian 9y ago
    I find that it's very effective at providing a "collective memory". People come and go, but as long as they document what they did in a ticket that information is easy to search and can save a lot of effort. And by "document" I don't even mean a fancy design doc and pages of spec. Just a few sentences on why something was done, or a couple of copy-pasted command lines used for testing or reproducing a bug can be a big help.
  - tetha 9y ago
    Our JIRA is kinda our external memory and our input for external teams. For team members, it's easier and more effective to bob a post-it onto the right white board and explain it during standup. Hosted JIRA is just infuriatingly slow for that workflow, tbh. For everyone else, yeah. Put in a JIRA ticket, one of us will put in clarifications there. If it's something weird, we'll document steps there and convert them into a runbook later. It's good enough for that.
- rootusrootus 9y ago
  The most irritating thing I run into with JIRA is that every time our ALM team rolls out a new point release, things subtly change; behavior is a little different, controls moved around, etc. Normally I'm not one to get all bent out of shape when someone moves the cheese, but dang, every. single. release. things just move around for no apparent reason and I have to relearn how to do the exact same task. Confluence is also evil. Not because it's Atlassian, but because wikis have many downsides to go along with the upsides. Everyone can create their own pages, so ... they do. And they don't keep maintaining it. So the amount of "documentation" we have is ever increasing and out of date. And like you say, a few people seem to do nothing but confluence work all day long.
  - jonathonf 9y ago
    > So the amount of "documentation" we have is ever increasing and out of date Has anyone found a _good_ way to maintain documentation?
    - toomuchtodo 9y ago
      Hire people to work on documentation.
justherefortart 9y ago
The problem is in most businesses you can explain this until you're blue in the face, to no avail.
aaronbrethorst 9y ago
tl;dr: be sure to usability test your products.
ukulele 9y ago
> Tip: Your marketing and sales teams should never be allowed to discuss features “in the pipeline” Probably unpopular here, but this is a misguided, engineer-centric view of a business. Sales & marketing should absolutely have some leeway on unreleased features, as they have very close relationships with customers and can get early feedback in the realest way possible: "Will the customer pay for it?" Cutting off this avenue is popular for engineers (myself included) but not necessarily good for the long term health of a business.
- dkyc 9y ago
  I think this is meant in another way: sales and marketing should never discuss future features with potential customers. Meaning, they should not try to win deals by promising features not yet rolled out, rather than not being involved in internal priorization discussions. I happen to agree with the author to an extent. Selling future things rather than what's presently available makes sales a lot easier (which is why reps are often eager to do it), but the true value is created when sales sells what is already there. My father always put it this way: You have to sell what you have.
  - daurnimator 9y ago
    Contrarily, large customers often want to know what your roadmap is like: they want to be assured that your company/product is moving in the same direction that they are. Sales and marketing solve this by mentioning things in the pipeline. Without knowledge of what's coming up you'll fail to close a huge number of deals.
  - ukulele 9y ago
    This approach is cutting off a major strategic artery. Sales & marketing can do much more than just pushing what they're given; they should be a valuable pulse on the customer, and the truest feedback you can get for an upcoming feature is whether or not someone will pay you for it. Particularly for larger or more advanced features, it makes much more sense to run engineering partly in parallel with sales & marketing (with feedback between the two) rather than in serial. Product and innovation cycles in the company are much tighter that way, and more accurate.
mful 9y ago
> Do I want my team to value the number of tickets closed, or do I want them to value our critical business KPIs? How does this look in practice, in large organizations? Is everyone empowered to freelance on whatever they think will improve said KPIs? At some point, someone needs to have the power to decide what is being built (hopefully with an eye towards KPI-impact), and others need to fall in line and close some tickets.
- foobarian 9y ago
  Well, presumably the tickets you have are there to build KPI-improving features. The people who open the tickets open them for that purpose! At our company even people who "freelance" stuff still open tickets for that work, because it's useful to document stuff and help others learn.
tnolet 9y ago
This post has nothing to do with tickets, Jira, Agile or actual in pocket cash. It just says "do a usability test" some time before shipping.
tarr11 9y ago
After a while, I ran the same analysis I did before. The shopping cart abandonment rate was reduced by double digits: A difference worth more than $1 million dollars per month. This is not a statistically valid way to test the effectiveness of your change. For example, if you sell toys, and you run an analysis on the week before Christmas, then you make your code change, and then check again on the week after Christmas, you are going to have differences in customer behavior that have nothing to do with your change.
- minimaxir 9y ago
  It also depends on how long "a while" is.
deleted 9y ago
[deleted]
radley 9y ago
Sounds like the team lacked a UX designer, probably as well an experienced software product manager. Testing conversion rates is UX 101.

66 points by stickhandle 9y ago

foobaw 9y ago

Interesting article - most of these metrics should be learned on the job for a lot of product managers.

lloydde 9y ago

2016

tyfon 9y ago

At first I thought the ticket was going to be "stop using JIRA/Confluence". There is so much time wasted on this system where I work it's amazing. Some people sit all day updating JIRA with all kind of crap instead of actually doing their job. At this point I'm starting to get the same feeling towards it as towards SAP.

bdcravens 9y ago
I think Jira's knobs and settings can get in the way, but I've found it to be fairly effective when used in a focused way (and recent updates seem to have optimized for that)
- taeric 9y ago
  Like any tool for managing the health of something, it all depends on the quality of the data in the system. What I have not seen, is evidence that it helps improve the data. So, if you have someone that is good at keeping the data healthy, they can make the tool look amazing. The tool doesn't really help people get there, though. I don't think there is a universal answer here, yet. :( I would love to be proven wrong.
- foobarian 9y ago
  I find that it's very effective at providing a "collective memory". People come and go, but as long as they document what they did in a ticket that information is easy to search and can save a lot of effort. And by "document" I don't even mean a fancy design doc and pages of spec. Just a few sentences on why something was done, or a couple of copy-pasted command lines used for testing or reproducing a bug can be a big help.
- tetha 9y ago
  Our JIRA is kinda our external memory and our input for external teams. For team members, it's easier and more effective to bob a post-it onto the right white board and explain it during standup. Hosted JIRA is just infuriatingly slow for that workflow, tbh. For everyone else, yeah. Put in a JIRA ticket, one of us will put in clarifications there. If it's something weird, we'll document steps there and convert them into a runbook later. It's good enough for that.
rootusrootus 9y ago
The most irritating thing I run into with JIRA is that every time our ALM team rolls out a new point release, things subtly change; behavior is a little different, controls moved around, etc. Normally I'm not one to get all bent out of shape when someone moves the cheese, but dang, every. single. release. things just move around for no apparent reason and I have to relearn how to do the exact same task. Confluence is also evil. Not because it's Atlassian, but because wikis have many downsides to go along with the upsides. Everyone can create their own pages, so ... they do. And they don't keep maintaining it. So the amount of "documentation" we have is ever increasing and out of date. And like you say, a few people seem to do nothing but confluence work all day long.
- jonathonf 9y ago
  > So the amount of "documentation" we have is ever increasing and out of date Has anyone found a _good_ way to maintain documentation?
  - toomuchtodo 9y ago
    Hire people to work on documentation.

justherefortart 9y ago

The problem is in most businesses you can explain this until you're blue in the face, to no avail.

aaronbrethorst 9y ago

tl;dr: be sure to usability test your products.

ukulele 9y ago

> Tip: Your marketing and sales teams should never be allowed to discuss features “in the pipeline” Probably unpopular here, but this is a misguided, engineer-centric view of a business. Sales & marketing should absolutely have some leeway on unreleased features, as they have very close relationships with customers and can get early feedback in the realest way possible: "Will the customer pay for it?" Cutting off this avenue is popular for engineers (myself included) but not necessarily good for the long term health of a business.

dkyc 9y ago
I think this is meant in another way: sales and marketing should never discuss future features with potential customers. Meaning, they should not try to win deals by promising features not yet rolled out, rather than not being involved in internal priorization discussions. I happen to agree with the author to an extent. Selling future things rather than what's presently available makes sales a lot easier (which is why reps are often eager to do it), but the true value is created when sales sells what is already there. My father always put it this way: You have to sell what you have.
- daurnimator 9y ago
  Contrarily, large customers often want to know what your roadmap is like: they want to be assured that your company/product is moving in the same direction that they are. Sales and marketing solve this by mentioning things in the pipeline. Without knowledge of what's coming up you'll fail to close a huge number of deals.
- ukulele 9y ago
  This approach is cutting off a major strategic artery. Sales & marketing can do much more than just pushing what they're given; they should be a valuable pulse on the customer, and the truest feedback you can get for an upcoming feature is whether or not someone will pay you for it. Particularly for larger or more advanced features, it makes much more sense to run engineering partly in parallel with sales & marketing (with feedback between the two) rather than in serial. Product and innovation cycles in the company are much tighter that way, and more accurate.

mful 9y ago

> Do I want my team to value the number of tickets closed, or do I want them to value our critical business KPIs? How does this look in practice, in large organizations? Is everyone empowered to freelance on whatever they think will improve said KPIs? At some point, someone needs to have the power to decide what is being built (hopefully with an eye towards KPI-impact), and others need to fall in line and close some tickets.

foobarian 9y ago
Well, presumably the tickets you have are there to build KPI-improving features. The people who open the tickets open them for that purpose! At our company even people who "freelance" stuff still open tickets for that work, because it's useful to document stuff and help others learn.

tnolet 9y ago

This post has nothing to do with tickets, Jira, Agile or actual in pocket cash. It just says "do a usability test" some time before shipping.

tarr11 9y ago

After a while, I ran the same analysis I did before. The shopping cart abandonment rate was reduced by double digits: A difference worth more than $1 million dollars per month. This is not a statistically valid way to test the effectiveness of your change. For example, if you sell toys, and you run an analysis on the week before Christmas, then you make your code change, and then check again on the week after Christmas, you are going to have differences in customer behavior that have nothing to do with your change.

minimaxir 9y ago
It also depends on how long "a while" is.

deleted 9y ago

[deleted]

radley 9y ago

Sounds like the team lacked a UX designer, probably as well an experienced software product manager. Testing conversion rates is UX 101.