4 ms·

While this sounds very cool, and my inner geek would love this dashboard on the wall of the office - is it actually useful or is it a distraction? What actions

While this sounds very cool, and my inner geek would love this dashboard on the wall of the office - is it actually useful or is it a distraction? What actions or decisions would you make within minutes of seeing the results? If product changes take days or weeks, daily analytics is just as useful, and stops people wasting time on looking at the data more than once per day.

rossjudson 14y ago
This is precisely the right point. We look at an analytic display to determine if action should be taken, and what those actions should be. The vast majority of "big data" is noise, more than effectively summarized with basic statistics.
sardonicbryan 14y ago
There are three major classes of decisions that get made from this. In descending order: 1) We generally have pretty tight revenue targets, so one thing I and my teams do now is forecast out expected revenue on a daily basis for the next 30 days, as revenue is highly dependent on a stream of new content, features and what we call live ops. Live ops are basically a combination of promotions, contests, events and tournaments. We have decent tools, so we can generally spin up a live op in a matter of minutes to a couple hours without doing an engineering push. We also have content we can release with a few hours of work, so if today is not going to go well, it gives us enough time to prep something for tomorrow. It's highly useful to know if a Live Op or new feature/content release is "working" or not. With the real time dashboard and experience, I can know with high confidence whether it's working within 30 minutes of release, which gives me more than enough time to spin something else up so that we don't lose the day. 2) On many days, we actually do multiple releases or live ops in a single game -- the pace of releases is fast even compared to other web companies. As unscientific as this is, it's often easier for me to evaluate/get a feel for the impact of a release/live op based on how much it moves the projection in the first hour after release, and before the next thing comes along. Obviously with stuff like that, you validate after the fact against more full data, but I've found after looking at this lot for over a year, I have a strong "feel" for what's working and what's not. 3) Related to 1) and 2), but I've found that using the tool gives me a much tighter feedback loop. When I make a feature release or live op and I'm sitting on the dashboard watching the release move or not move the projection, I have a much tighter feel on the impact. Most of my peers who don't have something like this often are not as rigorous in understanding and evaluating the different levers they have to impact revenue.

rossjudson 14y ago

This is precisely the right point. We look at an analytic display to determine if action should be taken, and what those actions should be. The vast majority of "big data" is noise, more than effectively summarized with basic statistics.

sardonicbryan 14y ago

There are three major classes of decisions that get made from this. In descending order: 1) We generally have pretty tight revenue targets, so one thing I and my teams do now is forecast out expected revenue on a daily basis for the next 30 days, as revenue is highly dependent on a stream of new content, features and what we call live ops. Live ops are basically a combination of promotions, contests, events and tournaments. We have decent tools, so we can generally spin up a live op in a matter of minutes to a couple hours without doing an engineering push. We also have content we can release with a few hours of work, so if today is not going to go well, it gives us enough time to prep something for tomorrow. It's highly useful to know if a Live Op or new feature/content release is "working" or not. With the real time dashboard and experience, I can know with high confidence whether it's working within 30 minutes of release, which gives me more than enough time to spin something else up so that we don't lose the day. 2) On many days, we actually do multiple releases or live ops in a single game -- the pace of releases is fast even compared to other web companies. As unscientific as this is, it's often easier for me to evaluate/get a feel for the impact of a release/live op based on how much it moves the projection in the first hour after release, and before the next thing comes along. Obviously with stuff like that, you validate after the fact against more full data, but I've found after looking at this lot for over a year, I have a strong "feel" for what's working and what's not. 3) Related to 1) and 2), but I've found that using the tool gives me a much tighter feedback loop. When I make a feature release or live op and I'm sitting on the dashboard watching the release move or not move the projection, I have a much tighter feel on the impact. Most of my peers who don't have something like this often are not as rigorous in understanding and evaluating the different levers they have to impact revenue.