3 ms·

The test appears to have been implemented as a JavaScript button with a timer. How many people would have simply set a timer for 10 minutes, then wait for a bi

by smokel 2y ago

The test appears to have been implemented as a JavaScript button with a timer. How many people would have simply set a timer for 10 minutes, then wait for a bit and return to the page to see if they had won a prize?

throwanem 2y ago
The Page Visibility API has been around for quite a while: https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibi... That said, as far as I can tell they aren't actually using it, although it is pretty hard to tell - I don't find any mention of it (as the string "visibilitychange", or "hidden" as an object prop, in the minified code) in the page or the linked sources. The timer did disappear after I had had the tab backgrounded for a little while, but clicking "I Quit" only pops a premium upsell sheet, so who knows. In entire fairness, though, I don't think they're really treating it as a particularly rigorous experiment in the first place. Assuming good faith on participants' part in the general case seems reasonable, especially given that it is paywalled and thus not too susceptible to widespread gaming of the 4chan stripe. And they are using Sentry and DD RUM on the experiment page, so if it really meant that much to them, they probably could filter out at least some confounders. (edit: On reflection, they'd have done well to support the fullscreen API here, which in its modern iteration I believe can be used with any block-scoped element, or maybe just any element. But it looks like the UI for this had to be put together in a hurry and maybe mostly outside the confines of their CMS, so I don't suppose I can really blame whoever wrote it for the omission.)
scotty79 2y ago
or you could stand up and go grab a coffee ... or look at your phone or other device instead

3 ms·

The test appears to have been implemented as a JavaScript button with a timer. How many people would have simply set a timer for 10 minutes, then wait for a bi

by smokel 2y ago

throwanem 2y ago
The Page Visibility API has been around for quite a while: https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibi... That said, as far as I can tell they aren't actually using it, although it is pretty hard to tell - I don't find any mention of it (as the string "visibilitychange", or "hidden" as an object prop, in the minified code) in the page or the linked sources. The timer did disappear after I had had the tab backgrounded for a little while, but clicking "I Quit" only pops a premium upsell sheet, so who knows. In entire fairness, though, I don't think they're really treating it as a particularly rigorous experiment in the first place. Assuming good faith on participants' part in the general case seems reasonable, especially given that it is paywalled and thus not too susceptible to widespread gaming of the 4chan stripe. And they are using Sentry and DD RUM on the experiment page, so if it really meant that much to them, they probably could filter out at least some confounders. (edit: On reflection, they'd have done well to support the fullscreen API here, which in its modern iteration I believe can be used with any block-scoped element, or maybe just any element. But it looks like the UI for this had to be put together in a hurry and maybe mostly outside the confines of their CMS, so I don't suppose I can really blame whoever wrote it for the omission.)
scotty79 2y ago
or you could stand up and go grab a coffee ... or look at your phone or other device instead