3 ms·

A couple things: 1) I’d like to see additional measures beyond dB. A 50dB consistent white noise does not bother me (and may even enhance my sleep) but an inte

by bumby 1y ago

A couple things: 1) I’d like to see additional measures beyond dB. A 50dB consistent white noise does not bother me (and may even enhance my sleep) but an intermittent off/on sound of the same magnitude has a very different effect 2) what is the accuracy of the consumer metrics compared to medical benchmarks? In other words, how reliable are the sleep measures of consumer devices? Anecdotally, I’ve noticed my watch thinks I’m sleeping sometimes when I watch a movie, and I’ve heard MDs say the metrics aren’t accurate enough to make strong conclusions.

wkat4242 1y ago
Yes i always sleep with a nature sound generator. It's comforting.
thesuitonym 1y ago
It would also be interesting to see the effects of intermittent noise when there is ambient background noise.
willsmith72 1y ago
yes, that would be nice to have. but surely the data we have already would suggest for the benefit of our entire society, especially children (with the link between good sleep and mental development), we should incentivise thicker windows, better sills and other quick wins now
- bumby 1y ago
  To the point above, we should be careful about making strong conclusions if there is a lot of uncertainty in the underlying data.
  - willsmith72 1y ago
    On the contrary, we gain more by making faster decisions than we do by being 100% precise
    - bumby 1y ago
      Nobody claimed we need 100% precision (or, more to the point, 100% certainty). But I would argue the level of our convictions should be proportional to the risk and level of certainty. To the point being made, for most Americans the cost of updating your building envelope and fenestration is relatively high. Yet if we look at the accuracy of the data (compared to the medical device standard), it’s moderate accuracy. So we’re taking a high risk for something that we’re only moderately confident will work. That’s not a good tradeoff. There may be interventions that are lower risk that make for a more balanced approach. “Move fast and break things” as a general philosophy is a risk-blind approach.
      - willsmith72 1y ago
        This isn't the first study on noise and sleep, in fact it's barely scientific compared to the rest of the research. There is no scientific disagreement as to the correlation, it's just "how big" and "what specifically". Compare the downside of "smaller impact than expected" with the cost of delay and it's a no-brainer
        bumby 1y ago
        >smaller impact than expected Yes, it’s about both effect size and uncertainty. That’s science. But your original post also brought policy into play. Unless you actually know that uncertainty, effect size, and those costs (including opportunity costs), you’re just making stuff up because “it just makes sense so it must be true.” That’s neither good science, nor good policy. Advocating for big policy changes (like changing building codes) without understanding those aspects is like jumping into a pool headfirst before knowing the depth.
        willsmith72 1y ago
        and this is how societal progress slowly crawls to a halt.
        deleted 1y ago
brandonb 1y ago
#1 is a great idea for follow-up -- looking at variance, 90th percentile vs median, and so on. There's probably some very salient measure of sleep-affecting noise that we can derive from the data. On the accuracy of sleep metrics, when Apple Watch makes an error, it tends to misclassify deep or REM sleep as core (light) sleep, or waking as light sleep. It's relatively rare to misclassify deep or REM sleep as awake, or deep as REM (and vice versa). That's partly why we focus on those specific sleep metrics. You can see a full confusion matrix here: https://www.empirical.health/metrics/deep-sleep-percent#accuracy-of-apple-watchs-deep-sleep-estimate https://www.empirical.health/metrics/deep-sleep-percent#accu...
- bumby 1y ago
  Thanks! That’s great information. Do you have any data on how that performance compares to medical-grade devices?
  - brandonb 1y ago
    The confusion matrix linked above compares the Apple Watch to polysomnography equipment (which is the medical device used in a sleep lab). It doesn't _quite_ answer your question, since we're assuming the polysomnography equipment is ground truth, but it at least gives an idea of the discordance between a consumer device and a medical device.
garyfirestorm 1y ago
There is a field dedicated to this called as psychoacoustics. Sound pressure level (SPL) alone won’t work for this experiment like you said. There are other metrics like steady state and transient noise, masking from different frequencies having different effects on humans. It is definitely doable but far more complicated than the tweets seem to make it sound.

by bumby 1y ago

wkat4242 1y ago

Yes i always sleep with a nature sound generator. It's comforting.

thesuitonym 1y ago

It would also be interesting to see the effects of intermittent noise when there is ambient background noise.

willsmith72 1y ago

yes, that would be nice to have. but surely the data we have already would suggest for the benefit of our entire society, especially children (with the link between good sleep and mental development), we should incentivise thicker windows, better sills and other quick wins now

bumby 1y ago
To the point above, we should be careful about making strong conclusions if there is a lot of uncertainty in the underlying data.
- willsmith72 1y ago
  On the contrary, we gain more by making faster decisions than we do by being 100% precise
  - bumby 1y ago
    Nobody claimed we need 100% precision (or, more to the point, 100% certainty). But I would argue the level of our convictions should be proportional to the risk and level of certainty. To the point being made, for most Americans the cost of updating your building envelope and fenestration is relatively high. Yet if we look at the accuracy of the data (compared to the medical device standard), it’s moderate accuracy. So we’re taking a high risk for something that we’re only moderately confident will work. That’s not a good tradeoff. There may be interventions that are lower risk that make for a more balanced approach. “Move fast and break things” as a general philosophy is a risk-blind approach.
    - willsmith72 1y ago
      This isn't the first study on noise and sleep, in fact it's barely scientific compared to the rest of the research. There is no scientific disagreement as to the correlation, it's just "how big" and "what specifically". Compare the downside of "smaller impact than expected" with the cost of delay and it's a no-brainer
      - bumby 1y ago
        >smaller impact than expected Yes, it’s about both effect size and uncertainty. That’s science. But your original post also brought policy into play. Unless you actually know that uncertainty, effect size, and those costs (including opportunity costs), you’re just making stuff up because “it just makes sense so it must be true.” That’s neither good science, nor good policy. Advocating for big policy changes (like changing building codes) without understanding those aspects is like jumping into a pool headfirst before knowing the depth.
        willsmith72 1y ago
        and this is how societal progress slowly crawls to a halt.
        deleted 1y ago

brandonb 1y ago

#1 is a great idea for follow-up -- looking at variance, 90th percentile vs median, and so on. There's probably some very salient measure of sleep-affecting noise that we can derive from the data. On the accuracy of sleep metrics, when Apple Watch makes an error, it tends to misclassify deep or REM sleep as core (light) sleep, or waking as light sleep. It's relatively rare to misclassify deep or REM sleep as awake, or deep as REM (and vice versa). That's partly why we focus on those specific sleep metrics. You can see a full confusion matrix here: https://www.empirical.health/metrics/deep-sleep-percent#accuracy-of-apple-watchs-deep-sleep-estimate https://www.empirical.health/metrics/deep-sleep-percent#accu...

bumby 1y ago
Thanks! That’s great information. Do you have any data on how that performance compares to medical-grade devices?
- brandonb 1y ago
  The confusion matrix linked above compares the Apple Watch to polysomnography equipment (which is the medical device used in a sleep lab). It doesn't _quite_ answer your question, since we're assuming the polysomnography equipment is ground truth, but it at least gives an idea of the discordance between a consumer device and a medical device.

garyfirestorm 1y ago

There is a field dedicated to this called as psychoacoustics. Sound pressure level (SPL) alone won’t work for this experiment like you said. There are other metrics like steady state and transient noise, masking from different frequencies having different effects on humans. It is definitely doable but far more complicated than the tweets seem to make it sound.