4 ms·

Multivariate (i.e. A/B) testing tools. They change the page client-side, which introduces a problem they call “flicker”. It just means that for a moment you se

Multivariate (i.e. A/B) testing tools. They change the page client-side, which introduces a problem they call “flicker”. It just means that for a moment you see the server-side rendered page as-is, then the tool hacks it into something else, depending on which group you’re in. This can obviously be jarring. The “flicker” is overcome by a page-hiding snippet that shows the page after the tool has done its dirty business or after a timeout (typically 3s). Often, these tools aren’t used for multivariate testing at all. They’re used because the marketing team doesn’t want to go through the web team, so they get an exec to force the web team to install the tool (along with 50 other pieces of marketing crapware) so marketing can hack its changes into the website after it’s loaded. It’s not uncommon for these tools to completely break the website, because the web team makes a change that breaks assumptions relied upon by the marketing team’s hacks. Often, the marketing team will have these changes made by a developer who they hire directly. They are unqualified to interview developers and this is one of many ways incompetent developers find employment.

Normal_gaussian 7y ago
on the flip side many websites run completely without an in house dev team, using a corporate hosting provider and retainer based web companies. The ability to use an injection tool and hire devs for small changes on quick turnarounds can be massive in enabling the marketing department to move forwards. And yet another side shows sites deliberately engineered this way to allow agile behaviour by non-devs. I've seen a few where they track an internal metric so that a dev somes and consolidates it all before it gets too bad.
- steve_taylor 7y ago
  The problem is it doesn’t work reliably because a lot of people use blockers. Instead of seeing the intended content, they see a blank page for a few seconds, followed by the original content that had already loaded a few seconds ago. Some websites don’t have a timeout, causing many users to see just a blank page unless they disable their blockers. Also, it’s terrible for SEO.
Causality1 7y ago
I'm not sure I've ever seen A/B testing in web articles that wasn't a case of A being an accurate headline and B being the most outrageous thing you could possibly title it without outright lying, and B always wins.
- colejohnson66 7y ago
  AreTechnica is guilty of this. They do A/B testing of headlines for 10 minutes, but their headlines aren’t the worst. I have seen some outrageous headlines regarding Republicans doing something they don’t like (Ars is a left leaning site) that are clearly designed to get clicks. It’s quite annoying.
  - Causality1 7y ago
    That's precisely the site I was thinking of, actually. Some of their writers like Jim Salter and Megan Guess are absolutely fantastic and others you pre-cringe before clicking the story.

Normal_gaussian 7y ago

on the flip side many websites run completely without an in house dev team, using a corporate hosting provider and retainer based web companies. The ability to use an injection tool and hire devs for small changes on quick turnarounds can be massive in enabling the marketing department to move forwards. And yet another side shows sites deliberately engineered this way to allow agile behaviour by non-devs. I've seen a few where they track an internal metric so that a dev somes and consolidates it all before it gets too bad.

steve_taylor 7y ago
The problem is it doesn’t work reliably because a lot of people use blockers. Instead of seeing the intended content, they see a blank page for a few seconds, followed by the original content that had already loaded a few seconds ago. Some websites don’t have a timeout, causing many users to see just a blank page unless they disable their blockers. Also, it’s terrible for SEO.

Causality1 7y ago

I'm not sure I've ever seen A/B testing in web articles that wasn't a case of A being an accurate headline and B being the most outrageous thing you could possibly title it without outright lying, and B always wins.

colejohnson66 7y ago
AreTechnica is guilty of this. They do A/B testing of headlines for 10 minutes, but their headlines aren’t the worst. I have seen some outrageous headlines regarding Republicans doing something they don’t like (Ars is a left leaning site) that are clearly designed to get clicks. It’s quite annoying.
- Causality1 7y ago
  That's precisely the site I was thinking of, actually. Some of their writers like Jim Salter and Megan Guess are absolutely fantastic and others you pre-cringe before clicking the story.