One of the main reasons for this is a lack of data. The Strava Heatmap (crowdsourced data, based on past frequency of use) is pretty much the best. OSM tagging

One of the main reasons for this is a lack of data. The Strava Heatmap (crowdsourced data, based on past frequency of use) is pretty much the best. OSM tagging is insufficient for real good routing, and it doesn't take traffic patterns into account. Thus, a lot of route creation is dependent on hyperlocal knowledge. Strava Route Builder, with the heatmap, goes fairly far on this, but it too isn't the best. This is a really hard problem to solve.