4 ms·

It's a 1% impact to conversion rate which, if you have over $1B in annual revenue like Trivago, is quite a large amount of money. And even if you don't, taking

It's a 1% impact to conversion rate which, if you have over $1B in annual revenue like Trivago, is quite a large amount of money. And even if you don't, taking a 1% hit to your conversion rate and hurting user experience just because you want to use a fancy new image format is probably not a good idea.

dahart 8y ago
Agreed, with all of that. It doesn’t answer my question, though, right? If the IE11 problem is temporary, and the IE11 perf gets improved next week in response to this article, then Trivago might have to do a 180, no? Even if we wait a while, IE11 usage will dwindle. Couldn’t the mere passage of time easily turn the story of JPEG-XR into a 1% or more net positive for Trivago, especially if the format is already a win on Edge? Every new technology fails to stack up to decades old standards by some metrics, yet we improve on and build new things anyway because we can see the future potential. This article isn’t acknowledging future potential. My question is what justification is there for declaring the format permanently unusable?
- jonas21 8y ago
  I don't think the author is saying not to use JPEG-XR for all time. They're just saying not to use it right now on production web sites.
jbreckmckye 8y ago
Without a breakdown of the stats and methodology, we have no way to know if this result was statistically meaningful, nor whether Trivago ran any null hypothesis tests. We don't even know that there were no confounding factors - just none the author thought of. The may be a spurious figure.
- jonas21 8y ago
  Except that the author claims it's a statistically significant result immediately before reporting the 1% number. The full phrase (which was truncated in the other comment) is: "we were able to statistically verify a -1% negative impact on core business metrics including conversions"
  - deleted 8y ago
    [deleted]
  - jbreckmckye 8y ago
    Even if so: the relationship between page conversion and revenue is not direct. You don't know whether the lost leads were high quality, nor whether they were reacquired through other means. Ultimately though the issue is more fundamental: you can't make assertions about performance without publishing actual figures. All we have to go on is the author's assertion that the test methodology was sound and the results were out of the bounds of chance. That's not really enough. I have worked with many, many teams who earnestly believed they were running "scientific" MVTs, with solid statistical analysis free of bias. They weren't.

dahart 8y ago

Agreed, with all of that. It doesn’t answer my question, though, right? If the IE11 problem is temporary, and the IE11 perf gets improved next week in response to this article, then Trivago might have to do a 180, no? Even if we wait a while, IE11 usage will dwindle. Couldn’t the mere passage of time easily turn the story of JPEG-XR into a 1% or more net positive for Trivago, especially if the format is already a win on Edge? Every new technology fails to stack up to decades old standards by some metrics, yet we improve on and build new things anyway because we can see the future potential. This article isn’t acknowledging future potential. My question is what justification is there for declaring the format permanently unusable?

jonas21 8y ago
I don't think the author is saying not to use JPEG-XR for all time. They're just saying not to use it right now on production web sites.

jbreckmckye 8y ago

Without a breakdown of the stats and methodology, we have no way to know if this result was statistically meaningful, nor whether Trivago ran any null hypothesis tests. We don't even know that there were no confounding factors - just none the author thought of. The may be a spurious figure.

jonas21 8y ago
Except that the author claims it's a statistically significant result immediately before reporting the 1% number. The full phrase (which was truncated in the other comment) is: "we were able to statistically verify a -1% negative impact on core business metrics including conversions"
- deleted 8y ago
  [deleted]
- jbreckmckye 8y ago
  Even if so: the relationship between page conversion and revenue is not direct. You don't know whether the lost leads were high quality, nor whether they were reacquired through other means. Ultimately though the issue is more fundamental: you can't make assertions about performance without publishing actual figures. All we have to go on is the author's assertion that the test methodology was sound and the results were out of the bounds of chance. That's not really enough. I have worked with many, many teams who earnestly believed they were running "scientific" MVTs, with solid statistical analysis free of bias. They weren't.