15 ms·
Analysis of Apple Watch running data
- FridayoLeary 4y ago>Because the watch performed better on earlier versions of watchOS, and even my iPhone 4S was better at tracking, I think the issue is software. >The measurement errors are so bad, that they overshadow performance differences between runs, and during runs. At the same time Apple keeps adding more and more incorrect stats and graphs, building on top of a shaky foundation. The kilometer splits measured in seconds imply more accuracy than is delivered. This.
- formerly_proven 4y agoThis sounds like it could all be due to a GPS issue.
- mwidell 4y agoOnly way to get accurate real time data when running is to use a good foot pod like Stryd. GPS just isn't as accurate as people think, no matter if you use Apple Watch or Garmin.
- rajman187 4y agoThat's right, GPS error can be on the order of 10s of meters, worse in urban environments. Google (and presumably Apple) use WPS to enhance location data, meaning a triangulation (simplification here) or location based on known WiFi router and bluetooth beacon positions. They also have a team dedicated to more complex geometric calculations of how the GPS signal may be reflected given the known positions of buildings. But if you have a clear line of sight that's a different issue. I went cycling on a path next to a river and the GPS trace was buttery smooth.
- mtts 4y agoThis used to be true, but in 2000 (!)the US government enabled the full range of GPS features for civilian use, vastly improving the accuracy of GPS receivers. GPS.gov says single band receivers should be able to get less than 2m accuracy. Dual band units, like the latest Garmin Forerunner watches, can do much, much better.
- sroussey 4y agoWi-Fi is stilled used for cold start — gives you a radius before GPS lock. As far as I know at least
- marban 4y agoI've clocked thousands of ~20km runs across all Watch generations and have never seen a significant discrepancy when measured against something like Google Maps. Urban area without skyscrapers or the like.
- cianmm 4y agoI’m part of a running group with a mix of Garmin and Apple Watches and the Apple Watches always have pretty different distant readings according to Strava, maybe 2%-5%. The Garmin devices are generally much closer together. I wonder if the dual-band GPS on the Apple Watch Ultra is an attempt to fix these problems? I would guess that it’s software, with the author, if for no other reason than I’d be surprised if Garmin were all that much better at putting GPS in a tiny housing than Apple.
- kadoban 4y ago> I would guess that it’s software, with the author, if for no other reason than I’d be surprised if Garmin were all that much better at putting GPS in a tiny housing than Apple. After "you're holding it wrong", anything seems possible. But yeah that does seem more likely to be software, it's a surprisingly difficult and fuzzily-defined problem.
- AmericanChopper 4y agoMy Apple Watch seems to get things wrong differently depending on the shape of my route. I often train on a track that I exactly know the lap distance for, and if I do a 10 mile run the watch will usually tell me that I’ve run about 9.5 miles. Which I kinda presume is due to the location sampling rate cutting distance off the oval shaped track. But every marathon I’ve run with the watch, it puts the full distance about .5 to .25 miles before the finish line.
- Clent 4y agoWeird hill to die on with Apple. That's a 10 year old issue with an iPhone 4, said by a CEO that is long dead. They gave everyone free bumpers, because sweaty fingers would close the antenna gaps. The statement was true, the gaps were placed where they were expected to least affect the grip. What exactly would you expect in this case? The messaging worked. If your iPhone 4 was having signal issue, consider readjusting your hand. Apple sent out free cases to compensate for the issue. This is not the nightmare scenario that non-iPhone users made it out to be. Apple haters, like any group of haters are a silly bunch.
- jquery 4y ago“Pausing always fixes erratic mode” makes me suspect they don’t have a GPS lock for some reason.
- fuzzy2 4y agoI wonder if cycling is also affected. Theoretically, the watch should be in an almost ideal position on flat handlebars, if less so on drop bars.
- jerlam 4y agoYou move a lot faster cycling, so GPS errors are less significant because there is a longer distance between readings. But also, cyclists are less concerned about specific speeds.
- fuzzy2 4y agoWell, maybe not specific speeds, but… :-D What you're saying makes sense, of course. I found some hundreds of meters of discrepancies in some rides (Apple Watch 4 vs. Karoo 2). Less than 1% of the total distance, of course.
- jeffbee 4y agoCycling computers blend calibrated wheel sensors with GPS data to give the most accurate distance, too. I believe Apple Watch refuses to use bike wheel and crank rotation sensors, but Garmin does use them.
- PragmaticPulp 4y ago> After about a kilometer I have to cross a busy street, I need to pause there regularly. Pausing always fixes erratic mode. Given that the “erratic mode” only impacts the beginning of the run and disappears suddenly at known locations, this sounds like an issue of delayed GPS lock. And given that the problem didn’t exist in the past, it could be a software issue due to upgrades. Or it could be a hardware issue that developed over time, such as something impacting GPS receive sensitivity. Or it could even be a new source of RF interference in the GPS range near the author’s start point, which impacts GPS lock until they get far enough away from it. Interesting issue, but note that this issue appears to be specific to this one specific person, not a general issue with all Apple Watches as some in this thread are speculating. I certainly have not noticed this behavior on my Watch even with the latest updates.
- icehawk 4y agoSadly, the guy didn't publish the tools they used to produce this. I would have been interested in seeing if I see a similar erratic mode in my own data.
- shp0ngle 4y agoThe issue seems definitely related to GPS.
- matsemann 4y agoIt's a common issue with how the Apple Watch forces you to start the activity before it attempts acquiring gps lock. For many that may work fine, but also for many that gives erratic behavior in the start where they live. All other brands behave the opposite. Based on this article they've fixed it for Ultra, at least https://www.dcrainmaker.com/2022/09/apple-watch-ultra-in-depth-review-its-a-start.html https://www.dcrainmaker.com/2022/09/apple-watch-ultra-in-dep... > Assuming you’re ready to go, then you’ve got two choices with Apple Watch Ultra. You could go the ‘normal’ route for all Apple Watches up till now, and wait for the 3-second countdown. Once that countdown completes, it’s at that point that the watch goes off and gets GPS signal and HR acquisition – not before. However, the Ultra edition includes a new ‘Precision Start’ feature, that lets you first open the workout up, then see the signal status before you begin
- rvba 4y agoApple showing more than people actually run to make them feel better? Seems like a lawsuit incoming due to "ruined health" by overstating measured activities.
- lifeinthevoid 4y agoaside: My girlfriend tracks her runs with Strava through her Iphone 12 mini. Upon zooming in, the "straight" parts never look straight and look a lot more like a triangle wave. She consistently tracks around 10% larger distance than my Garmin watch. I've checked on the map, and the Garmin distance is the accurate one of the 2.
- MuffinFlavored 4y agoWhat % inaccurate is the Apple Watch and what % inaccurate is the Garmin compared to what the map says?
- jerlam 4y agoThis would also depend on how she's carrying the iPhone. While putting the GPS receiver on your wrist isn't optimal, putting the iPhone in a pocket or somewhere closer and lower on the body where it sees even less of the sky is worse.
- brippalcharrid 4y agoMy experience of using Strava on a mid-range Android phone several years ago was that readings were splayed horizontally across (road-side) paths by five or ten meters in many cases. Now that I have a Garmin watch, I am gradually closing in on my former Estimated Best Efforts with readings that appear to be much more accurate in terms of how they correspond to the paths that I've been running on.
- lifeinthevoid 4y agoShe wears it around her waist, might explain a lot. Thanks for the pointer!
- zorlack 4y agoOP should run their route backwards for a week. To see if the effect is symmetrical.
- comment_ran 4y agoAssuming it is an aerboic work out, what's the difference if the watch tells you that it's 9.6k, 9.7k, or 9.8k? The only thing that I need to know the accurate distance is to train a specific event, e.g. mile run, 5k, or 10k. If I train to run a mile within four minutes, all I need to know is to run 60s for each 400m on track.
- macNchz 4y agoAccurate real time pacing information can be useful during races, when it’s easy to get caught up in the pace of the crowd and lose track of the muscle memory for your race pace. It’s also nice when you want to do any sort of interval training without actually going to a track. Having accurate overall distances is less important, but is useful if you’re training for a specific distance and want to run some time trials without using a track or measured course.
- verst 4y agoI pace many half marathon and marathon races. Usually what I need for that is: instantaneous pace, average pace, and lap distance. I tend to plan my pacing for equal effort, so each lap pace is adjusted for elevation such that in the end the overall pace is just slightly faster than what is needed for goal pace. Needless to say, I use my Garmin Forerunner 945 for this. If anyone is running the Seattle Marathon, I'll be there pacing :)
- pjot 4y agoMany runners are racing themselves. To them, seconds matter.
- moonchrome 4y ago>Assuming it is an aerboic work out, what's the difference if the watch tells you that it's 9.6k, 9.7k, or 9.8k? If you're tracking incremental progress than such variability makes the data less useful. But that's why I prefer to use natural landmarks and a stopwatch - the smart watches I've used so far are off to the point where I don't trust them.
- oezi 4y agoDid OP mention if the scripts are published somewhere?
- balderdash 4y agoA friend has both a whoop and an Apple Watch. One or both of them are wrong. He almost never get consistent results between the two, Apple is almost always higher than whoop (avg. heart rate for a given period) it’s a bit depressing.
- s3p 4y agoOn the flip side, Apple has FDA clearance to use their device for single-lead ECGs, so I would trust their heart rate reading over a hacky tech startup's.
- michaelje 4y agoI’ve just completed a 6 month self comparison of the Whoop v4 and Watch S6 for cycling. I found for long endurance efforts they were surprisingly accurate to each other - but during intervals of intense exertion (eg a KOM/hill climb) often the Whoop would read my HR as ~40-60BPM lower while the watch would reflect an expected reading (160-180bpm). Sometimes after 1-2 mins the whoop would “catch up” but it would leave a giant drop in HR graphing for that interval. This also lowered the avg HR for the workout on the whoop, as you noted. Happy to share an example comparison graph if you’re interested, just reach out. For what it’s worth, I ended up cancelling the Whoop this month after trying twice to engage with their data team.
- kmonsen 4y agodcrainmaker had an article where he reviewed the whoop, he liked most things about it, except the accuracy. Without that the rest was meaningless.
- nilsbunger 4y agoThe trouble acquiring a gps lock could be due to 5G towers (remembering all the controversy with the FAA), or a new building, or something else that changed in the built environment.
- oezi 4y agoFrom a discussion on Garmin nav accuracy: > Russia (GLONASS) may intentionally give wrong results in your area. Try switching to just GPS, or GPS & Galileo. https://www.reddit.com/r/Garmin/comments/vo9w0l/comment/ied5chp/ https://www.reddit.com/r/Garmin/comments/vo9w0l/comment/ied5...
- Godel_unicode 4y agoIf you read down in that thread it turns out it was nothing so mysterious; the ephemeris file for Sony gps chips was expired and had to be updated.
- cogogo 4y agoI have never seen and cannot find the pace graph the author shows as a screenshot from the fitness app. Would love that data. Just spent 10 min googling and gave up. Reminds me of this thread from yesterday https://news.ycombinator.com/item?id=32965288 https://news.ycombinator.com/item?id=32965288
- nmlt 4y agoIn iOS 16 it’s tap on the workout, then anywhere under Workout details and scroll a bit. That is really no discoverability issue, because there is even a show more button next to it, that has the same function. But maybe it changed on iOS 16?
- cogogo 4y agoReally weird. The only things I can tap into are the splits and the route map. I can scroll the heart rate over to recovery as well. I’m on ios 15. I wonder if it’s the workout type? Outdoor Run, Open Goal is what I’ve used for years now.
- xt00 4y agoI wonder if at some point Apple reduced the sampling frequency of the gps position to save power? Since this sounds pretty important to the guy, I would buy a Garmin watch or similar to compare to and run with both. I also suspect one of the challenges here is that taking a lot of corners like in an urban environment might result in rounded paths — like if you ran around a rectangular city block it ends up being logged as a square with rounded corners. Whereas if the sampling frequency was higher it would capture those corners much better, or if the authors runs were more of a straight line.
- loufe 4y agoIt would explain his shortened distances, as low sampling would round out corners, "shortening" the total distance.
- s3p 4y agoDoes anyone know how OP exported this data? I want to see if my watch has the same data quality issues
- nmlt 4y agoOpen health app and press on your profile picture in the upper right corner (scroll to the top). Then scroll down and press export all health data. Also OP mentioned the exported XML has some errors in iOS 16, so beware of that.
- asutekku 4y agoHonestly, it doesn't matter for the vast population using these devices. I understand the issue, but for a normal runner, most they care about is how did their pace / heart rate compare to their past runs.
- jalla 4y agoThe watch needs updated GPS ephemeris data to accurately calculate the position, received by an aGPS (Assisted GPS) server (over IP) or by satellite. This takes at least 30 seconds by satellite. Solution: Tether with IPhone before leaving until fix is achieved, otherwise keep watch outside or near a window for 10-15 minutes before running so that it can update its almanac from overhead satellites and get a fix. There is no software solution. Buy a newer watch with cellular connectivity and aGPS-support.
- ciex 4y agoDo you have an explanation for this being a new issue?
- jalla 4y agoOP could have changed exercise habits, leading to more frequently outdated GPS almanac data. Otherwise, newer OS versions could have degraded the fix algorithm for older Watch-models. If newer hardware comes with cellular connectivity, newer OS version could assume that by default and not prioritise (or test) backward compatibility with older Watch-models.
- xani_ 4y agoWhy it would need such common updates tho ? And why it couldn't load that data for say month ahead ?
- minton 4y ago> For me an average pace of 3:52 is great, while 4:12 is very bad. Um, that’s very fast. At my pace (8:45 or 5:26 in km), I barely notice the discrepancies on my Apple Watch 7.
- nmlt 4y agoThe author still seems to be on an Apple Watch 4, if I read correctly. So this seems like a case of Apple shipping a bad update to an older device and not caring. Or OP has a bad device.
- deleted 4y ago[deleted]
- kodisha 4y agoSemi related iWatch Rant - if you think running data is bad, try tracking your sleep. It just doesn't work. - Sleep for 3h - wake up for 1, sleep for 5 more: records 3h only. - (being a parent of a small baby) do not sleep at all during the night, sleep 4+3 hours during the day: 0 hours recorded - sleep trough the night but wake up every 90-100 minutes (baby again): 0 hours recorded Damn, this is a $500+ device, and it cant even get basic sleep data correctly. Also, it takes anywhere from 1min to couple of hours for data to appear in the Health app. OTOH my wife has ~3 year old Huawei Fit watch, which was about $120, and that thing records every 10-15min or longer nap. Without a mistake.
- Tagbert 4y agoYou probably want to checkout Sleep++ or Autosleep for sleep tracking. The third-party apps go beyond the basic, built-in feature.
- icebergonfire 4y agoAs someone who went through the same process, including using a Huawei watch previously, I encourage you to try Autosleep. It has much better sleep detection magic and exposes a few knobs that enable you to fine-tune the autodetection and also manually correct the sleep records in sensible ways. It integrates with Health, so it has the bonus of tidying up the sleep data all across the board. Not related to the developer in any way, just a very happy user.
- kodisha 4y agoOMG thanks! Installing it right away!
- rubicon33 4y agoAutosleep is great. I've had none of the issues the OP mentions and I've used Autosleep since I first got my watch.
- ValentineC 4y ago+1 for AutoSleep. I've been using it since before WatchOS 7 introduced sleep tracking, and it's scarily accurate even at tracking when I doze off for a short while in bed.
- tluyben2 4y agoSince the watchOS 9 update, I am the most unhealthy person ever. I saw on Reddit people have the same results but it really went mental; I walked into the local clinic for a vo2max as the watch keeps warning me; it (the watch) was 15 points off on the negative side, and so was my blood oxygen; 10% off on the wrong side. It’s pretty scary for a software update…
- odysseus 4y agoAs a counterpoint, I saw something similar happen after upgrading to watchOS 8 for 7 months, but then I somehow got my vo2max to go back up by losing 5 pounds in 1 month and eating healthier.
- tluyben2 4y agoWell, so far it made me more healthy as I am trying to change it but it definitely is off from the real numbers as I did an actual vo2max at the clinic here.
- 12ian34 4y agoI'm delighted with my Fitbit Charge 5. Costs a fraction of the Apple Watch, don't need to buy into the Apple Ecosystem to make it useful. Tracks my sleep and fitness shockingly well. The official app is surprisingly good, and there's an API that works pretty well and has endpoints for most if not all the data. That said, I'm a casual exerciser, not a proper hardcore sportsperson. For those I'd say avoid a lifestyle device like the Apple watch or Fitbit watches - it feels like Garmin supports that niche better.
- deleted 4y ago[deleted]
- SSLy 4y ago> Fitbit I'd rather not excercise than have my health data feed the GOOGL advertising engine, thank you. https://arstechnica.com/gadgets/2022/09/google-will-start-assimilating-fitbit-accounts-next-year/ https://arstechnica.com/gadgets/2022/09/google-will-start-as... Because I do not trust them to uphold the EU rules.
- FearlessNebula 4y agoYou’d pass up on reduced risk of all cause mortality, reduced risk of cancer, reduced risk of fall injuries, and improved mental health [1]? Just to stick it to Google? Man, get some exercise. If you don’t like Google either don’t use a tracker, build your own, or use one that’s more trustworthy. 1: https://health.gov/sites/default/files/2019-09/Physical_Activity_Guidelines_2nd_edition.pdf https://health.gov/sites/default/files/2019-09/Physical_Acti...
- SSLy 4y agoThank you for your compassion, but I already have another tracker (Apple Watch) and am running twice a week.
- tr33house 4y agoJust got this too and I love it so far
- notafraudster 4y agoThe use of "small multiples" here in the visualization is excellent, and the layout of the first visual is provocative and eye-catching -- love the vertical line! Great job, author!
- Maursault 4y agoI question the validity of the application. Apple seems to be promoting physical health while simultaneously leveraging mental illness to increase revenue. I grant that this is an actual business space, accessorizing and promoting fitness, but these computerized accessories fundamentally distract from the individual's primary goal of getting in shape, leading to reliance on computerized devices that are unnecessary to meeting fitness goals. Isn't there such a thing as tuning one's ability to know one's own limitations naturally? Run. Run hard. Run fast. Run until you can't run any longer, and discover and accept physical limitations and push against them if your drive is unsatisfied. Is this kind of data gathering really necessary to fitness? I'm nearly certain these applications actually fall under the category of play and entertainment. Maybe watch Rocky (1976) and/or Chariots of Fire (1981) for inspiration. Note the lack of any cybernetics. I'll take a mechanical stopwatch over an Apple Watch running fitness application any day of the week and twice on Sundays. Feel free to use an Apple Watch if it makes you happy, but to accept nothing less than perfection is really quite something else, so maybe there is a different kind of fitness that is immediately more pressing than physical fitness.
- mrosett 4y agoWhat you’re describing is a fantastic way to injure yourself.
- Maursault 4y agoComment is vague, but apparently what you are suggesting is that physical fitness isn't safe without using a computer. Our bodies evolved to run, and it is, actually, possible to recognize and react to what the body reports to the brain through sensation without any intermediary electronic device gathering and reporting false data. Many are capable of sensing injury or illness before overt symptoms appear, and react accordingly, reducing or eliminating negative impact. The more computerized devices are relied on for ordinary activity, the more they become a crutch, and the more difficult it becomes to operate without them. Obsession is unmistakably unhealthy.
- sirsinsalot 4y ago
- samastur 4y agoAccording to my Apple Watch I must be the slowest swimmer in the world. I've just spent 3 weeks on the Adriatic coast, swimming an hour twice a day and I lost count of times my watch reported swimming distance of less than 100m (with an absolute low of 4m). Luckily for me I use it mainly to see how long I've been swimming and don't care about other data because it is obviously useless.
- scorpios77 4y ago
- jupp0r 4y agoI used the Strava App to track my runs on my Apple Watch 2. At some point it started crashing and I tried out the built in app. It's really not for me, the data is somewhat locked in, graphs are non zoomable, etc. Most notably I couldn't send friends a url to the run. I ended up switching to a Garmin Fenix 7 recently and I'm genuinely happy with it so far. Battery lasts more than 2 weeks and it does 90% of what I was using my Apple Watch for.
- zsolt224 4y agoWhen I was running a half marathon with my Apple Watch 6 ( without a phone) I was getting a 1km notifications within 5-15 meters of the km signs. I was super impressed with the accuracy
- robcorn 4y agoGreat writeup. I've noticed the same thing by recording simultaneously with the watch workout app and strava on my iphone. The watch always says pace is slower.
- matsemann 4y agoRelated: DC Rainmaker just released his walk through of the new ultra https://www.dcrainmaker.com/2022/09/apple-watch-ultra-in-depth-review-its-a-start.html https://www.dcrainmaker.com/2022/09/apple-watch-ultra-in-dep... Dc is the gold standard in fitness guides. My interpretation is that if you (like me) want a sports watch, buy a sports watch and not a smart watch with sports features. At least too many deal breakers for me. But they're getting better.
- hnburnsy 4y agoSecond this, his Apple Watch Ultra review is amazingly detailed (like all his reviews), opinionated, and he does his best to participate in the comments.
- xani_ 4y agoReminds me how Sports Tracker (and really most apps) got wonky every time my phone battery got below 15% and got into aggressive power savings (and no amount of convincing would convince Android to fuck off). Kinda smells like something similar, some aggressive power savings cutting on GPS accuracy or how much app is allowed to run in background.
- scott_w 4y agoThis sounds like my experience of the Apple Watch Series 3—absolute garbage for exercise. I think the problem is the GPS hasn’t locked on when you start the run but there’s no way to know this in the UI. The only solution I’ve found is to “start” early, immediately pause, then wait a bit for GPS to locate you before unpausing. Most sports watches solve this problem by explicitly telling you your GPS status when you are getting ready to track your exercise.
- sanderjd 4y agoWhat's the best gps exercise watch you've used?
- notesinthefield 4y agoI have similar thoughts as OP (though I now use a Series 7 and have few to no issues with GPS now) - the Garmin Fenix 5 or Coros Pace 2 are pretty great. Ive done trail ultra's, road halfs, mixed bike races, city to city tours and everything in between with both. Coros doesnt integrate with as many things as I would like.
- killjoywashere 4y agoGarmin's entire line-up is pretty great. They focus on things that matter: honest GPS measurements and battery life being at the top of the list. I really like some of the new Tactix 7 features, like the ability to remember the route but forget the geo coordinates.
- williamscales 4y agoGarmin for sure. I use a Fenix series but they have the Forerunner too. I hear Suunto watches are good too. Basically any company that has been in the GPS space for a long time. Garmin has been doing GPS since long before Apple contemplated making phones.
- cluoma 4y agoIf you already have an Apple Watch, I recommend trying WorkOutDoors before looking for another device. It’s a huge step up from the default workout app that I’m still using my Series 3 for everything up to marathons.
- parker_mountain 4y ago> Disabling wifi on the Apple Watch, Bluetooth on iPhone and combinations of those, to disconnect the watch from iPhone before starting the run Unfortunately, disabling wifi will absolutely wreck havok with getting a quick, precise lock on GPS. Also, the author didn't specify if they have a cellular model or not, which is also a factor. More importantly, this really is an issue for Apple to fix - let people know what the GPS status is! IMHO: the apple watch is the BEST casual fitness device. But, if you compete or are training in any serious way, it falls flat almost immediately.
- Ensorceled 4y agoMy Series 3 was dead on until this spring; there is a 5k run I do 2-3 times a week where the Strava would tell me I had hit 5km within a few meters of the same tree. Then it started misbehaving, usually 100-300m short of the tree, a couple of times 50-100m beyond the tree. The path in Strava was all over the place. I tried rebooting, reinstalling the Strava app. Cleaned up some Watch apps. Didn't get better. Finally I deleted all the watch apps except Strava: apple store, audible, three authenicator apps (why does Authy have a watch app?), my Bank's app (wtf?) etc. etc. etc. Turned off or configured for minimal sync anything I couldn't delete. Things started tracking again.
- fulafel 4y agoIs it rare in these kinds of applications of GPS to buffer up received GPS signal and retroactively fix the data after the lock? If so, why?
- jbverschoor 4y agoSimilar to my experience. It's a fun little stopwatch that you have to charge every day, multiple times per day. Apple support is friendly, but terrible.
- firexcy 4y agoExactly my experience. I run in laps so I'm pretty sure of the distance simply by counting and multiplying, which gives a mostly constant speed of ~5min/km. Apple Watch says my speed is sometimes 3.5min and sometimes 6.5min, and calculates drastically different caloric counts from day to day, which is to the best useless and to the worst misleading and demeaning.
- aanire 4y agoI'm using new SE, prior to that was using Forerunner 245. I went for a run with watch on each wrists, both almost identical GPS wise, 10k run on Apple watch came as 9.99km on Garmin, heart rate almost same, more calories on Apple watch though