8 ms·

I seem to remember someone posting their results of testing Google Maps, Apple Maps, and Waze for ETA accuracy, and what you describe was also what they found:

I seem to remember someone posting their results of testing Google Maps, Apple Maps, and Waze for ETA accuracy, and what you describe was also what they found: Apple Maps was the most accurate, Google Maps was optimistic, but accurate, Waze basically reported the best case scenario. Edit: found the original [0] I've long thought that Waze creates the illusion of saving massive amounts of time, when its benefits are marginal (although real). I know I personally prefer an active commute to one waiting in traffic, even if travel times are identical. Waze seems to maximize for the latter. [0] - https://arturgrabow.ski/2018/02/19/navigation-apps/amp/ https://arturgrabow.ski/2018/02/19/navigation-apps/amp/

killjoywashere 7y ago
I wonder if those who are more likely to measure something like this are more likely to drive closer to the speed limit? I tend to drive fast and find Google gets me there a couple minutes before ETA, Apple gets me there way before ETA, and Waze just overloads me with more information than I want.
SamBam 7y ago
Google Maps also got the person there 5% faster than Apple Maps, in that study. This really does call for a larger study, though. It would be nice to see how the error bars and potential differences come out across different trip times, and different drivers.
- K0SM0S 7y ago
  You'd probably end with significantly different results based on country. Somehow the underlying logic of mapping AI seems to be more-or-less of a fit for national road systems — for instance in Europe, with lots of discrepancies between borders, it's really hit or miss: in France and Germany I end up on stupid country roads to save 2 minutes on a 2h drive, but that seldom happens in the Netherlands (granted they have a great road infrastructure). Switzerland is a mess too. My intuition (listening to AI and DL podcasts) is that they just train too-general US-based (or evaluated or reinforced more specifically) models. Anyhow it's a disappointing situation that e.g. Google maps makes way more mistakes today than it did 5 years ago (but it also does more overall, I guess it's a trade-off, e.g. now it's great for public transportation in large enough cities).
  - Swizec 7y ago
    I remember using some car’s built-in GPS once and it wanted me to take every freeway exit and get right back on after. Because it decided that saves time/distance. Maps really need a “not worth it” features for tiny optimizations. I’d often prefer a slightly slower route that’s less work to drive.
    - K0SM0S 7y ago
      Oh please YES, the "not worth it", followed by an option to "please take me back to previous path before (because) latest fork = mistake". The very idea of judging the overall travel is a good start (at the end, when they prompt you) but individual datapoints should be reportable as well. I'm pretty sure this would massively help build a dataset with "annotated" cases (good / bad) for RNNs. > I’d often prefer a slightly slower route that’s less work to drive. Me too, and I suspect most people — like, when is it ever a good option to avoid a free highway to save 1 minute when the driving is twice or more fatiguing? Are we bodyless machines or animals with a concept for "tired" and "attention"? This is where I feel recent evolutions in DL have been, perhaps, more for the benefit of developers and researchers (efficiency), and maybe not for real human benefits on the user side. I don't claim the problem is easy, but I know second-hand that what users want and what AI projects solve for may be two very different X.
      - johnmaguire2013 7y ago
        > please take me back to previous path before (because) latest fork = mistake Seriously. I regularly travel from Ohio to Michigan and back, and often times if I take an exit for gas, Google Maps will assume I want to take an obscure country backroad for the final 2 hours rather than get back on the highway. I absolutely do not want to worry about deer, a lack of gas stations, food, and light.
        sjwright 7y ago
        This is especially true in the UK where most rural roads are nominally 60 MPH even if they’re barely wide enough for two cars.
pauljonas 7y ago
After using Waze for over a couple of years, I really started getting the impression that I was just a data pawn in a bunch of A/B tests -- like send this one down that route, send that one down the other way & see what happens.
- mattkevan 7y ago
  Absolutely sure Google Maps A/B tests routes too. Even when sitting next to each other in the same car, Google Maps suggests radically different routes to the same destination on my wife’s phone than it does on mine, often with a 15+ min difference in arrival time. It must be testing different routing algorithms as otherwise they should be the same.
  - sgillen 7y ago
    Could also be an optimal routing thing too. If google maps sends everyone down the same path it might become too congested. Especially if the route is a detour around construction or and accident or something.
  - abandonliberty 7y ago
    I think routing is one of those computationally expensive problems (possibly np-complete) to solve. As a result, there's a bunch of clever non-deterministic approaches that get you "pretty good" answers, but depending on the implementation can produce variable results.
    - widforss 7y ago
      A-to-B routing is simple and not very expensive. You can implement it without very much graph theory knowledge. It's when visiting a number of destinations in an arbitrary order that things get difficult.
      - Doxin 7y ago
        A-to-B routing is simple in theory. In practice implementing A* on a graph of all the roads in the world is non-trivial. You'd need to figure out some way to split the data up into chunks or to make neighbourhood lookups fast enough to be workable. I'd imagine you'd end up with something of at least the complexity of hierarchical A, but depending on the size of your chunks you'd need to still special-case the start and end chunks. And that's just the first complexity I can come up with. I imagine the distance is going to be wildly* nontrivial as well. You'd need to factor in traffic density, weather conditions, road works, predicted traffic density, actual distance as well as traffic speed, not to mention fuzzy human things like scenery. Just going for "shortest euclidian distance" will end up with routing through every alley and side-street if it's even the tiniest amount shorter. I wouldn't be surprised if you end up with a distance metric that looks essentially random unless you're deeply inspecting the graph at the time of routing to see why a certain route is shorter than another. Of course that doesn't exclude A/B testing, but with google maps they clearly have a routing graph that's updating all the time. It's no surprise that asking for the same route twice can give you different answers. Not to mention they probably include a random factor on purpose so they don't route all the cars through the same street. Of course you'd want the random factor big enough to spread the load, but not so big that random people get shafted by a significantly longer commute. Seems like an interesting problem to tackle actually, I should look at building an open street map based route planner sometime.
mumblemumble 7y ago
I never really got into Waze; once I get to know my way around a city I'm inclined to just get around the oldschool way, sticking to arterial roads and all that. I also used to work for a company where I occasionally visited an office where Waze was really popular with the folks who worked there. And found that that, when we were meeting up somewhere halfway across town after work, I would typically get there before the Waze users about as often as they got there before me. So, not really much of a time saver. Judging from my experiences on the occasions that someone gave me a ride somewhere, I was probably having a more pleasant, albeit lower-tech, driving experience, too. If I had to hazard a guess, I'd imagine Waze has a law of large numbers problem: The sample size they have for estimating the travel time along any given stretch of side road is smaller, and there's also a lot more ways to route oneself on the side roads, and those two factors compound to mean a relatively high chance that at least one of the routes Waze considers has been assigned a grossly over-optimistic travel time estimate.
thehashrocket 7y ago
I much prefer driving over sitting in traffic. One thing I noticed with Waze is that it would route me way out in the middle of nowhere to my location vs sitting in traffic. It was kind of interesting to see some of these back roads, I'll admit. I did learn some good alternate routes that I would sometimes just use on my own because I knew the freeway was going to be a parking lot. I mostly use apple maps now with car play which is fine. After we moved my commute went from ~20-45 minutes to ~12 minutes so Waze really isn't a necessity anymore. :/
canes123456 7y ago
I highly doubt the benefits of Waze are real, at least in Miami. While the random routes that Waze recommends might be more entertaining, it rarely is faster and sometimes WAY slower.
- fma 7y ago
  Probably a regional thing. I live in Atlanta and overall the routes are good, and Google maps isn't good accounting for traffic. I grew up in Ft. Lauderdale and when I go back I use Google maps. Waze would take you down roads I'd never use. Also I went to the keys and US1 had an accident. Google maps gave me a parallel road to save me 30 minutes. I thought it was wrong. I turned reluctantly and sure enough, it was a good route... Waze was like derp derp...which was probably to my advantage as everyone else sat in traffic.
tinus_hn 7y ago
Waze also likes to overflow neighborhoods with traffic because it thinks there is a shorter path, which just isn’t a very nice thing to do.

killjoywashere 7y ago

I wonder if those who are more likely to measure something like this are more likely to drive closer to the speed limit? I tend to drive fast and find Google gets me there a couple minutes before ETA, Apple gets me there way before ETA, and Waze just overloads me with more information than I want.

SamBam 7y ago

Google Maps also got the person there 5% faster than Apple Maps, in that study. This really does call for a larger study, though. It would be nice to see how the error bars and potential differences come out across different trip times, and different drivers.

K0SM0S 7y ago
You'd probably end with significantly different results based on country. Somehow the underlying logic of mapping AI seems to be more-or-less of a fit for national road systems — for instance in Europe, with lots of discrepancies between borders, it's really hit or miss: in France and Germany I end up on stupid country roads to save 2 minutes on a 2h drive, but that seldom happens in the Netherlands (granted they have a great road infrastructure). Switzerland is a mess too. My intuition (listening to AI and DL podcasts) is that they just train too-general US-based (or evaluated or reinforced more specifically) models. Anyhow it's a disappointing situation that e.g. Google maps makes way more mistakes today than it did 5 years ago (but it also does more overall, I guess it's a trade-off, e.g. now it's great for public transportation in large enough cities).
- Swizec 7y ago
  I remember using some car’s built-in GPS once and it wanted me to take every freeway exit and get right back on after. Because it decided that saves time/distance. Maps really need a “not worth it” features for tiny optimizations. I’d often prefer a slightly slower route that’s less work to drive.
  - K0SM0S 7y ago
    Oh please YES, the "not worth it", followed by an option to "please take me back to previous path before (because) latest fork = mistake". The very idea of judging the overall travel is a good start (at the end, when they prompt you) but individual datapoints should be reportable as well. I'm pretty sure this would massively help build a dataset with "annotated" cases (good / bad) for RNNs. > I’d often prefer a slightly slower route that’s less work to drive. Me too, and I suspect most people — like, when is it ever a good option to avoid a free highway to save 1 minute when the driving is twice or more fatiguing? Are we bodyless machines or animals with a concept for "tired" and "attention"? This is where I feel recent evolutions in DL have been, perhaps, more for the benefit of developers and researchers (efficiency), and maybe not for real human benefits on the user side. I don't claim the problem is easy, but I know second-hand that what users want and what AI projects solve for may be two very different X.
    - johnmaguire2013 7y ago
      > please take me back to previous path before (because) latest fork = mistake Seriously. I regularly travel from Ohio to Michigan and back, and often times if I take an exit for gas, Google Maps will assume I want to take an obscure country backroad for the final 2 hours rather than get back on the highway. I absolutely do not want to worry about deer, a lack of gas stations, food, and light.
      - sjwright 7y ago
        This is especially true in the UK where most rural roads are nominally 60 MPH even if they’re barely wide enough for two cars.

pauljonas 7y ago

After using Waze for over a couple of years, I really started getting the impression that I was just a data pawn in a bunch of A/B tests -- like send this one down that route, send that one down the other way & see what happens.

mattkevan 7y ago
Absolutely sure Google Maps A/B tests routes too. Even when sitting next to each other in the same car, Google Maps suggests radically different routes to the same destination on my wife’s phone than it does on mine, often with a 15+ min difference in arrival time. It must be testing different routing algorithms as otherwise they should be the same.
- sgillen 7y ago
  Could also be an optimal routing thing too. If google maps sends everyone down the same path it might become too congested. Especially if the route is a detour around construction or and accident or something.
- abandonliberty 7y ago
  I think routing is one of those computationally expensive problems (possibly np-complete) to solve. As a result, there's a bunch of clever non-deterministic approaches that get you "pretty good" answers, but depending on the implementation can produce variable results.
  - widforss 7y ago
    A-to-B routing is simple and not very expensive. You can implement it without very much graph theory knowledge. It's when visiting a number of destinations in an arbitrary order that things get difficult.
    - Doxin 7y ago
      A-to-B routing is simple in theory. In practice implementing A* on a graph of all the roads in the world is non-trivial. You'd need to figure out some way to split the data up into chunks or to make neighbourhood lookups fast enough to be workable. I'd imagine you'd end up with something of at least the complexity of hierarchical A, but depending on the size of your chunks you'd need to still special-case the start and end chunks. And that's just the first complexity I can come up with. I imagine the distance is going to be wildly* nontrivial as well. You'd need to factor in traffic density, weather conditions, road works, predicted traffic density, actual distance as well as traffic speed, not to mention fuzzy human things like scenery. Just going for "shortest euclidian distance" will end up with routing through every alley and side-street if it's even the tiniest amount shorter. I wouldn't be surprised if you end up with a distance metric that looks essentially random unless you're deeply inspecting the graph at the time of routing to see why a certain route is shorter than another. Of course that doesn't exclude A/B testing, but with google maps they clearly have a routing graph that's updating all the time. It's no surprise that asking for the same route twice can give you different answers. Not to mention they probably include a random factor on purpose so they don't route all the cars through the same street. Of course you'd want the random factor big enough to spread the load, but not so big that random people get shafted by a significantly longer commute. Seems like an interesting problem to tackle actually, I should look at building an open street map based route planner sometime.

mumblemumble 7y ago

I never really got into Waze; once I get to know my way around a city I'm inclined to just get around the oldschool way, sticking to arterial roads and all that. I also used to work for a company where I occasionally visited an office where Waze was really popular with the folks who worked there. And found that that, when we were meeting up somewhere halfway across town after work, I would typically get there before the Waze users about as often as they got there before me. So, not really much of a time saver. Judging from my experiences on the occasions that someone gave me a ride somewhere, I was probably having a more pleasant, albeit lower-tech, driving experience, too. If I had to hazard a guess, I'd imagine Waze has a law of large numbers problem: The sample size they have for estimating the travel time along any given stretch of side road is smaller, and there's also a lot more ways to route oneself on the side roads, and those two factors compound to mean a relatively high chance that at least one of the routes Waze considers has been assigned a grossly over-optimistic travel time estimate.

thehashrocket 7y ago

I much prefer driving over sitting in traffic. One thing I noticed with Waze is that it would route me way out in the middle of nowhere to my location vs sitting in traffic. It was kind of interesting to see some of these back roads, I'll admit. I did learn some good alternate routes that I would sometimes just use on my own because I knew the freeway was going to be a parking lot. I mostly use apple maps now with car play which is fine. After we moved my commute went from ~20-45 minutes to ~12 minutes so Waze really isn't a necessity anymore. :/

canes123456 7y ago

I highly doubt the benefits of Waze are real, at least in Miami. While the random routes that Waze recommends might be more entertaining, it rarely is faster and sometimes WAY slower.

fma 7y ago
Probably a regional thing. I live in Atlanta and overall the routes are good, and Google maps isn't good accounting for traffic. I grew up in Ft. Lauderdale and when I go back I use Google maps. Waze would take you down roads I'd never use. Also I went to the keys and US1 had an accident. Google maps gave me a parallel road to save me 30 minutes. I thought it was wrong. I turned reluctantly and sure enough, it was a good route... Waze was like derp derp...which was probably to my advantage as everyone else sat in traffic.

tinus_hn 7y ago

Waze also likes to overflow neighborhoods with traffic because it thinks there is a shorter path, which just isn’t a very nice thing to do.