3 ms·

If they were to actually test "chat bubble blindness", you have to replace the bubble icon with the text at roughly the same position and compare. They didn't d

If they were to actually test "chat bubble blindness", you have to replace the bubble icon with the text at roughly the same position and compare. They didn't do that. Rather, they moved the chat to more prominent position - at the top of the page near the center, with bigger space, etc. So of course it increases the engagement. More important question is if they are getting spurious extra engagement (i.e. more chat traffic eventually costs more money), or they are getting the engagement they want.

steve-benjamins 6y ago
Good luck AB testing things in the real world :p
- drivebycomment 6y ago
  Indeed. I'm not saying they should do a perfect A/B test. Rather, in this case, their conclusion (of "chat bubble blindness") is not a logical conclusion you can draw from the experiment they performed.
  - renewiltord 6y ago
    No, that's not the conclusion. That's the portion that is explicitly speculation. There are tells in the language: > The navigation link led to a 62% increase in live chat conversations...This felt like a win— I like live chat but I hate chat bubbles...In any case, don't expect to see a chat bubble on Atlist ever again. That's the conclusion: "For this page, losing the bubble is okay. We're going to lose the bubble." > And if I hate them, maybe users hate them too? Maybe users have developed a chat bubble blindness— just like banner ad blindness. That's the speculation. You can tell because of the 'maybe's in there. This is actually quite well-written. The conclusions are written in clear language indicating fact. There is a potential "future work" description that speculates on possibilities while expressing the uncertainty. IMHO very well expressed, good text. But what you've said has made me think about why people sometimes read scientific papers the way they do. They'll say "They claim that this will make fusion power possible" or something like that when those parts are in the "Future Work" section and are therefore unsubstantiated and intended to be a "here are some distributaries of this river of fact". Interesting. I think the cues are adequate, but clearly they are not for many. I still think they should write the way they do and just accept that some in the audience aren't going to make it there.

steve-benjamins 6y ago

Good luck AB testing things in the real world :p

drivebycomment 6y ago
Indeed. I'm not saying they should do a perfect A/B test. Rather, in this case, their conclusion (of "chat bubble blindness") is not a logical conclusion you can draw from the experiment they performed.
- renewiltord 6y ago
  No, that's not the conclusion. That's the portion that is explicitly speculation. There are tells in the language: > The navigation link led to a 62% increase in live chat conversations...This felt like a win— I like live chat but I hate chat bubbles...In any case, don't expect to see a chat bubble on Atlist ever again. That's the conclusion: "For this page, losing the bubble is okay. We're going to lose the bubble." > And if I hate them, maybe users hate them too? Maybe users have developed a chat bubble blindness— just like banner ad blindness. That's the speculation. You can tell because of the 'maybe's in there. This is actually quite well-written. The conclusions are written in clear language indicating fact. There is a potential "future work" description that speculates on possibilities while expressing the uncertainty. IMHO very well expressed, good text. But what you've said has made me think about why people sometimes read scientific papers the way they do. They'll say "They claim that this will make fusion power possible" or something like that when those parts are in the "Future Work" section and are therefore unsubstantiated and intended to be a "here are some distributaries of this river of fact". Interesting. I think the cues are adequate, but clearly they are not for many. I still think they should write the way they do and just accept that some in the audience aren't going to make it there.