5 ms·

I'd take all of this with a giant pinch of salt. If the data is anything like the sleep data generated by my Garmin vivosmart HR+ it's not much better than garb

I'd take all of this with a giant pinch of salt. If the data is anything like the sleep data generated by my Garmin vivosmart HR+ it's not much better than garbage. It frequently counts me as sleeping when I sit down to watch TV in the evening, or read a book in bed. Sometimes I stay up until 1am reading and it always tracks me as deep sleep from 10pm when I got into bed. I usually take it off for my morning shower (It is waterproof, it just annoys me when washing) and it nearly always counts the walking from bed to shower as a few minutes of restless sleep and the shower itself as deep sleep. I'm not sure how it makes an assesment of my sleep level. I don't know if it's just based on movement or if it takes heart rate data into account but it does have a heart rate sensor. It seems to usually be wrong. In theroy it would be possible to know about these limitations and adjust your use of it to make it more accurate, but unless the participents of the study are aware of that and have been manually correcting the data then that won't be the case. (Not a general critisim on the device, I like it for everything else, but it's sleep data is clearly rather inaccurate)

ClayM 9y ago
Mine is actually pretty good - it doesn't pick up when I sit down, but will occasionally turn on sleep mode while I'm lying in bed reading. However it marks me as awake during that sleep time, so I guess it's accurate?
OrwellianChild 9y ago
These are all worthy considerations for these trackers... I can only speak for the Fitbits, specifically the Charge 2 (which has a HR monitor too). Fitbits have 2 measurement modes for sleep - normal and sensitive. Normal pretty much tracks me as sleeping like the dead for 8 hours each night (however long I'm immobile in bed). Sensitive, on the other hand, tracks much more detail and can tell when I'm restless, sleeping deeply, and even when I get up in the night and move around. I highly recommend the sensitive sleep setting for Fitbit users - it gives you more data to play with! Fitbit recently re-worked the way they track sleep in its HR-monitor-equipped devices (Charge 2, Aria HR, etc.). It introduced more details (it was just tracking restless/deep before) and this upgrade greatly improved the sleep tracking for me. Before Sleep Stages, it would track only 3-4 hours of my sleep each night. Now, it catches the full 6-9 hours. Now, all of this is to be taken with a grain of salt in terms of accuracy, but to compare among Fitbit users and to yourself over time, this information is incredibly useful! The strongest example of the usefulness of this for me so far has been the effect of moderate alcohol consumption on the quality of my sleep. I am far more restless and get much poorer sleep after drinking just 1-2 beers on a full stomach. This has led me to cut back almost completely on alcohol and the quality of my rest has dramatically improved!
- iClaudiusX 9y ago
  How do you know if it's "catching" anything other than noise? Have you done any kind of comparison against a gold standard method like a sleep study? From what I can tell, none of these accelerometer-based methods work at all and the companies pushing these as sleep trackers are making false claims.
  - OrwellianChild 9y ago
    Like I said, accuracy isn't so much the goal... The readouts I get show pretty consistent measurements over time, which gives me enough confidence to compare them to other Fitbit users among my friends/family as well as to myself over time (see the alcohol anecdote above). Setting the sensitivity to "normal" just raises the noise floor in their filter so high that it obscures the data.On sensitive, I've never had issues with it identifying "sitting" as "sleeping", yet it still effectively tracks my sleep (including periods of wakefulness immediately pre- and post- slumber in bed as well as mid-night disruptions that wake me up). This suggests that I'm getting pretty good data (though not equivalent to a sleep study with ECG/EKG equipment). In this case, "good" is good enough for my uses.
  - SEJeff 9y ago
    I've done a sleep study at a hospital to see if I was an actual insomniac or had sleep apnea. It turns out I slept like absolute crap at the study mainly because the thing they put up my nose tickled my nose hairs and the thing on my head was uncomfortable, so I just was restless the entire night.
- 0xffff2 9y ago
  >The strongest example of the usefulness of this for me so far has been the effect of moderate alcohol consumption on the quality of my sleep. I am far more restless and get much poorer sleep after drinking just 1-2 beers on a full stomach. I'm curious, did you not notice this before? I have an MS Band, but the sleep tracking seems similar to the new Fitbit tracking. It did give me some concrete data on how drinking affects my sleep, but that data wasn't really valuable because I already knew that drinking too much or too late would have a negative impact on my sleep because I felt tired the next morning.
Mahn 9y ago
> It frequently counts me as sleeping when I sit down to watch TV in the evening, or read a book in bed. Sometimes I stay up until 1am reading and it always tracks me as deep sleep from 10pm when I got into bed. I usually take it off for my morning shower (It is waterproof, it just annoys me when washing) and it nearly always counts the walking from bed to shower as a few minutes of restless sleep and the shower itself as deep sleep. Speaking as a Fitbit Charge 2 user, none of that happens with my device. I don't know how accurate the rest of the data is, but it does track sleep only when I'm actually sleeping.
- KWD 9y ago
  I agree. The Charge 2 is pretty accurate with the sleep start/end times, and when I'm awake during the night.
- rconti 9y ago
  Ditto. I had zero of those issues when my Charge HR was working. (I just threw it out yesterday as it finally gave up the ghost). It would accurately identify when I was asleep vs just sitting still, and lest you think it was biased towards nights, it accurately counted naps on the rare occasions I took one. The only thing it was BAD at was managing the data when changing timezones. I can hardly blame it. Who really knows what "time" it was when I was half-sleeping on the plane to Australia, and the clock is wrong anyway until I sync it two days later.
rodgerd 9y ago
As someone who migrated from Fitbit to Garmin, the sleep tracking is an awful regression - it's much, much worse with the Garmin.
devbent 9y ago
> It frequently counts me as sleeping when I sit down to watch TV in the evening, or read a book in bed. This is why I pushed for sleep on v1 of Microsoft Band to be manually triggered. Eventually cloud detected sleep was lit up, but we benefited from lots of gathered data at that point, but even so, it is a hard problem to solve. The best solution would be per user training, allowing users to correct auto detection and use that to improve future tracking, but AFAIK no one in industry is doing that. > I'm not sure how it makes an assesment of my sleep level. I don't know if it's just based on movement or if it takes heart rate data into account This is an incredibly hard problem. For sleep studies, multiple doctors look over all the vitals gathered (and for a full sleep clinic study, subjects are very wired up!) And to determine when the subject is in a given phase of sleep, they vote, and it is not always a consensus decision![0] Trying to then make a machine algorithm out of that is difficult, especially working with consumer level sensors. That said, across most users, data tend to be mostly correct, which is what these consumer devices target. Being able to say someone got 7.5hrs of sleep is good enough, the 10 minute bathroom break doesn't impact the overall numbers too much. Being able to give a good summary of weeks and months is huge, and being able to spot large regressions and improvements is key to user satisfaction. The ultimate goal is to gather a lot of data, noise or not, run it through machine learning, and start being able to give people actionable advice that they will see real results from. [0] this was relayed to me by the team who doing sleep classification, other sleep studies may happen differently.
deleted 9y ago
[deleted]
smartbit 9y ago
This review [1] compared the results of 6 sleep trackers with sleeping laboratorium measurements. It concludes that the trackers were able to track the lenght of the sleep quite accurately. OTOH all trackers displayed large errors in measuring the type of sleep e.g. deep sleep, light sleep & REM. They conclude that sleep trackers are good for creating awareness of your sleepquota. [1] https://www.heise.de/ct/ausgabe/2015-25-Schlaftracker-vs-Schlaflabor-2910003.html https://www.heise.de/ct/ausgabe/2015-25-Schlaftracker-vs-Sch...
- brlewis 9y ago
  Right. That's why this article is describing a breakthrough even though Fitbit isn't the first to claim to track sleep stages.
js2 9y ago
I have a Garmin 935 and had a 235 before it and for me the sleep tracking seems to be quite accurate. I can't vouch for the deep/light sleep, but the time of sleep/wake is almost always correct. The 935 also has "Move IQ" events where it auto-detects certain patterns. It captures when I walk my dog, and even captured some laps I did in the pool the other day. The optical heart rate tracking also looks pretty good.
agumonkey 9y ago
Is it possible to cross correct the data using heart rate ? I believe it's lower when actually sleeping rather than just being in bed.
- brlewis 9y ago
  Absolute heart rate, no, but heart rate variability, yes. It took a lot of research to make it accurate. https://help.fitbit.com/articles/en_US/Help_article/2163#AutomaticDetection https://help.fitbit.com/articles/en_US/Help_article/2163#Aut...
dmichulke 9y ago
I have a Xiaomi band 2 which I bought for some 30 EUR and it also considers me asleep when playing games before I go to sleep and sometimes when watching TV as well. Other than that it's pretty accurate, it even detects the occasional 15 steps to the toilet and back as awake time and I'm generally happy with it. It has a heart rate sensor but AFAIK you need to put some other firmware on it to use that during the night at predefined intervals. Regarding tracking sleep quality (it only has light, deep and no sleep as categories), I feel it tracks it well enough and I treat the duration of deep sleep as an upper bound (usually around 2h30). So if the band shows me little deep sleep it's almost always correct (i.e. I don't feel I slept well). If it shows lots of deep sleep I can sometime confirm and sometimes not. Of course, all this is subjective, so "sample size = 1" holds ;)
antirez 9y ago
I had a Fitbit and now the Garmin you have. The latter is a better device from almost any POV but sleep tracking. Fitbit sleep tracking is much more reliable and actually almost never reports non true things.

ClayM 9y ago

Mine is actually pretty good - it doesn't pick up when I sit down, but will occasionally turn on sleep mode while I'm lying in bed reading. However it marks me as awake during that sleep time, so I guess it's accurate?

OrwellianChild 9y ago

These are all worthy considerations for these trackers... I can only speak for the Fitbits, specifically the Charge 2 (which has a HR monitor too). Fitbits have 2 measurement modes for sleep - normal and sensitive. Normal pretty much tracks me as sleeping like the dead for 8 hours each night (however long I'm immobile in bed). Sensitive, on the other hand, tracks much more detail and can tell when I'm restless, sleeping deeply, and even when I get up in the night and move around. I highly recommend the sensitive sleep setting for Fitbit users - it gives you more data to play with! Fitbit recently re-worked the way they track sleep in its HR-monitor-equipped devices (Charge 2, Aria HR, etc.). It introduced more details (it was just tracking restless/deep before) and this upgrade greatly improved the sleep tracking for me. Before Sleep Stages, it would track only 3-4 hours of my sleep each night. Now, it catches the full 6-9 hours. Now, all of this is to be taken with a grain of salt in terms of accuracy, but to compare among Fitbit users and to yourself over time, this information is incredibly useful! The strongest example of the usefulness of this for me so far has been the effect of moderate alcohol consumption on the quality of my sleep. I am far more restless and get much poorer sleep after drinking just 1-2 beers on a full stomach. This has led me to cut back almost completely on alcohol and the quality of my rest has dramatically improved!

iClaudiusX 9y ago
How do you know if it's "catching" anything other than noise? Have you done any kind of comparison against a gold standard method like a sleep study? From what I can tell, none of these accelerometer-based methods work at all and the companies pushing these as sleep trackers are making false claims.
- OrwellianChild 9y ago
  Like I said, accuracy isn't so much the goal... The readouts I get show pretty consistent measurements over time, which gives me enough confidence to compare them to other Fitbit users among my friends/family as well as to myself over time (see the alcohol anecdote above). Setting the sensitivity to "normal" just raises the noise floor in their filter so high that it obscures the data.On sensitive, I've never had issues with it identifying "sitting" as "sleeping", yet it still effectively tracks my sleep (including periods of wakefulness immediately pre- and post- slumber in bed as well as mid-night disruptions that wake me up). This suggests that I'm getting pretty good data (though not equivalent to a sleep study with ECG/EKG equipment). In this case, "good" is good enough for my uses.
- SEJeff 9y ago
  I've done a sleep study at a hospital to see if I was an actual insomniac or had sleep apnea. It turns out I slept like absolute crap at the study mainly because the thing they put up my nose tickled my nose hairs and the thing on my head was uncomfortable, so I just was restless the entire night.
0xffff2 9y ago
>The strongest example of the usefulness of this for me so far has been the effect of moderate alcohol consumption on the quality of my sleep. I am far more restless and get much poorer sleep after drinking just 1-2 beers on a full stomach. I'm curious, did you not notice this before? I have an MS Band, but the sleep tracking seems similar to the new Fitbit tracking. It did give me some concrete data on how drinking affects my sleep, but that data wasn't really valuable because I already knew that drinking too much or too late would have a negative impact on my sleep because I felt tired the next morning.

Mahn 9y ago

> It frequently counts me as sleeping when I sit down to watch TV in the evening, or read a book in bed. Sometimes I stay up until 1am reading and it always tracks me as deep sleep from 10pm when I got into bed. I usually take it off for my morning shower (It is waterproof, it just annoys me when washing) and it nearly always counts the walking from bed to shower as a few minutes of restless sleep and the shower itself as deep sleep. Speaking as a Fitbit Charge 2 user, none of that happens with my device. I don't know how accurate the rest of the data is, but it does track sleep only when I'm actually sleeping.

KWD 9y ago
I agree. The Charge 2 is pretty accurate with the sleep start/end times, and when I'm awake during the night.
rconti 9y ago
Ditto. I had zero of those issues when my Charge HR was working. (I just threw it out yesterday as it finally gave up the ghost). It would accurately identify when I was asleep vs just sitting still, and lest you think it was biased towards nights, it accurately counted naps on the rare occasions I took one. The only thing it was BAD at was managing the data when changing timezones. I can hardly blame it. Who really knows what "time" it was when I was half-sleeping on the plane to Australia, and the clock is wrong anyway until I sync it two days later.

rodgerd 9y ago

As someone who migrated from Fitbit to Garmin, the sleep tracking is an awful regression - it's much, much worse with the Garmin.

devbent 9y ago

> It frequently counts me as sleeping when I sit down to watch TV in the evening, or read a book in bed. This is why I pushed for sleep on v1 of Microsoft Band to be manually triggered. Eventually cloud detected sleep was lit up, but we benefited from lots of gathered data at that point, but even so, it is a hard problem to solve. The best solution would be per user training, allowing users to correct auto detection and use that to improve future tracking, but AFAIK no one in industry is doing that. > I'm not sure how it makes an assesment of my sleep level. I don't know if it's just based on movement or if it takes heart rate data into account This is an incredibly hard problem. For sleep studies, multiple doctors look over all the vitals gathered (and for a full sleep clinic study, subjects are very wired up!) And to determine when the subject is in a given phase of sleep, they vote, and it is not always a consensus decision![0] Trying to then make a machine algorithm out of that is difficult, especially working with consumer level sensors. That said, across most users, data tend to be mostly correct, which is what these consumer devices target. Being able to say someone got 7.5hrs of sleep is good enough, the 10 minute bathroom break doesn't impact the overall numbers too much. Being able to give a good summary of weeks and months is huge, and being able to spot large regressions and improvements is key to user satisfaction. The ultimate goal is to gather a lot of data, noise or not, run it through machine learning, and start being able to give people actionable advice that they will see real results from. [0] this was relayed to me by the team who doing sleep classification, other sleep studies may happen differently.

deleted 9y ago

[deleted]

smartbit 9y ago

This review [1] compared the results of 6 sleep trackers with sleeping laboratorium measurements. It concludes that the trackers were able to track the lenght of the sleep quite accurately. OTOH all trackers displayed large errors in measuring the type of sleep e.g. deep sleep, light sleep & REM. They conclude that sleep trackers are good for creating awareness of your sleepquota. [1] https://www.heise.de/ct/ausgabe/2015-25-Schlaftracker-vs-Schlaflabor-2910003.html https://www.heise.de/ct/ausgabe/2015-25-Schlaftracker-vs-Sch...

brlewis 9y ago
Right. That's why this article is describing a breakthrough even though Fitbit isn't the first to claim to track sleep stages.

js2 9y ago

I have a Garmin 935 and had a 235 before it and for me the sleep tracking seems to be quite accurate. I can't vouch for the deep/light sleep, but the time of sleep/wake is almost always correct. The 935 also has "Move IQ" events where it auto-detects certain patterns. It captures when I walk my dog, and even captured some laps I did in the pool the other day. The optical heart rate tracking also looks pretty good.

agumonkey 9y ago

Is it possible to cross correct the data using heart rate ? I believe it's lower when actually sleeping rather than just being in bed.

brlewis 9y ago
Absolute heart rate, no, but heart rate variability, yes. It took a lot of research to make it accurate. https://help.fitbit.com/articles/en_US/Help_article/2163#AutomaticDetection https://help.fitbit.com/articles/en_US/Help_article/2163#Aut...

dmichulke 9y ago

I have a Xiaomi band 2 which I bought for some 30 EUR and it also considers me asleep when playing games before I go to sleep and sometimes when watching TV as well. Other than that it's pretty accurate, it even detects the occasional 15 steps to the toilet and back as awake time and I'm generally happy with it. It has a heart rate sensor but AFAIK you need to put some other firmware on it to use that during the night at predefined intervals. Regarding tracking sleep quality (it only has light, deep and no sleep as categories), I feel it tracks it well enough and I treat the duration of deep sleep as an upper bound (usually around 2h30). So if the band shows me little deep sleep it's almost always correct (i.e. I don't feel I slept well). If it shows lots of deep sleep I can sometime confirm and sometimes not. Of course, all this is subjective, so "sample size = 1" holds ;)

antirez 9y ago

I had a Fitbit and now the Garmin you have. The latter is a better device from almost any POV but sleep tracking. Fitbit sleep tracking is much more reliable and actually almost never reports non true things.