4 ms·

I wonder what metric Google uses as a proxy for "conversion" in their A/B testing-- page views? Length of time on site? Ads clicked?

stingraycharles 17y ago
IMHO there would be only one thing that makes sense: money earned.
- michael_dorfman 17y ago
  And how would they A/B test for that, exactly? Obviously, that's what they are interested in, in the long run-- but to test design changes, they need a more specific (and easily measurable) metric. And, no doubt, they have one. I just don't know what it might be.
  - inerte 17y ago
    They have several. To answer your question, if I was to pick up only one, the first that came to my mind was "pageviews". So, everything that increases pageviews is good. Well, not quite, because so is CTR, and in fact doubling CTR allows you to halve pageviews. Anyway, whatever they think of, the ultimate step is money earned. No single path leads to this, but several, like pageviews, CTR, click value, searches per day, other services integration, etc... Edited: You're thinking about A/B testing but I don't know if you're aware there's also multivariate testing, which Google does use (its optimizer easily creates A/B and multivariate).
- gmurphy 17y ago
  This is the thought process and slippery slope that leads to sites like experts-exchange - with a large userbase, it's easy to create and demonstrate changes that result in bags of money in the medium term, yet destroy the long-term value and usage of your site.
moultano 17y ago
Pretty much every statistic you can imagine gets tracked. The interesting thing is how they are used. Time on site is a negative signal for instance.
timcederman 17y ago
Position of result clicked (including ads) Number of result clicks per session (including ads) % of sessions with 0 results % of sessions with no click-through etc.

4 ms·

I wonder what metric Google uses as a proxy for "conversion" in their A/B testing-- page views? Length of time on site? Ads clicked?

by michael_dorfman 17y ago

stingraycharles 17y ago
IMHO there would be only one thing that makes sense: money earned.
- michael_dorfman 17y ago
  And how would they A/B test for that, exactly? Obviously, that's what they are interested in, in the long run-- but to test design changes, they need a more specific (and easily measurable) metric. And, no doubt, they have one. I just don't know what it might be.
  - inerte 17y ago
    They have several. To answer your question, if I was to pick up only one, the first that came to my mind was "pageviews". So, everything that increases pageviews is good. Well, not quite, because so is CTR, and in fact doubling CTR allows you to halve pageviews. Anyway, whatever they think of, the ultimate step is money earned. No single path leads to this, but several, like pageviews, CTR, click value, searches per day, other services integration, etc... Edited: You're thinking about A/B testing but I don't know if you're aware there's also multivariate testing, which Google does use (its optimizer easily creates A/B and multivariate).
- gmurphy 17y ago
  This is the thought process and slippery slope that leads to sites like experts-exchange - with a large userbase, it's easy to create and demonstrate changes that result in bags of money in the medium term, yet destroy the long-term value and usage of your site.
moultano 17y ago
Pretty much every statistic you can imagine gets tracked. The interesting thing is how they are used. Time on site is a negative signal for instance.
timcederman 17y ago
Position of result clicked (including ads) Number of result clicks per session (including ads) % of sessions with 0 results % of sessions with no click-through etc.