5 ms·

Map Your Google Location History

62 points by wormold 12y ago

tghw 12y ago
For those who don't want to do it themselves: https://maps.google.com/locationhistory/b/0 https://maps.google.com/locationhistory/b/0
- markveronda 12y ago
  Actually, the Google-provided version restricts you to only 30 days. If you're interested in everywhere you have been, ever (according to Google), then this is quite awesome and you get much more options with how you display that data and what else you can incorporate with it.
  - thekos 12y ago
    You can go back further by modifying the startTime parameter in the download link. https://maps.google.com/locationhistory/b/0/kml?startTime=1400003200000&endTime=1410159600000 https://maps.google.com/locationhistory/b/0/kml?startTime=14...
    - jacquesm 12y ago
      Oh my, a hacker. Better be careful. http://theblot.com/man-jailed-because-feds-dont-understand-computer-hacking-7717221 http://theblot.com/man-jailed-because-feds-dont-understand-c...
dalek2point3 12y ago
how does python know what map data to use?
- markveronda 12y ago
  It's fed in by the JSON file that was downloaded in Step 1 from Google when you request an archive of your data. From the site: # Create a dataframe from the json file in the filepath raw = pd.io.json.read_json('/Users/chrisralbon/Downloads/Location History/LocationHistory.json')
acadien 12y ago
Very cool script! I just tried it out myself and it seems that Google's location history is actually really spotty. It missed several of my trips (during which the phone was active and responsive) and it also included some spots that I've never been to. I double checked the JSON and the problem is on Google's end and not the script.
dochtman 12y ago
I built my own version of this about two years ago: https://dirkjan.ochtman.nl/writing/2012/11/28/tracing-a-path.html https://dirkjan.ochtman.nl/writing/2012/11/28/tracing-a-path... Python code available here: https://github.com/djc/galatna https://github.com/djc/galatna
sahaskatta 12y ago
I used a heatmap API with 2+ years worth of Google Location History data to make this last year: https://plus.google.com/+sahaskatta/posts/5bbn9D9ZzuP https://plus.google.com/+sahaskatta/posts/5bbn9D9ZzuP
- IneffablePigeon 12y ago
  This inspired me to do the same. If anyone wants to try, I put my implementation up on github: https://github.com/olane/locationheatmap https://github.com/olane/locationheatmap
edent 12y ago
If you want to get your data programatically, Google do not provide an API. You'll need to "borrow" your own cookies. I documented the process at http://shkspr.mobi/blog/2014/04/extracting-your-own-location-information-from-google-the-hard-way/ http://shkspr.mobi/blog/2014/04/extracting-your-own-location...
- dochtman 12y ago
  You can just use Google Takeout, IIRC.
  - edent 12y ago
    Takeout does not provide an API.
- Tiksi 12y ago
  I once tried to quickly hack together some bash using "Copy as curl" in firebug to do something similar and got this shameful monstrosity after a minute or two of fighting with awk: for i in {10..14};do for j in 1 7; do date -d "20$i-0$j-01T00:11:20.000Z" "+%s000";date -d "20$i-0$j-01T00:11:20.000Z" "+%s000"|tr "\n" " "; done;done|sed '1d;$d'|awk '{system("curl \047https://maps.google.com/locationhistory/b/0/kml?startTime=" $1 "&endTime=" $2 "\047 -H \047Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\047 -H \047Accept-Encoding: gzip, deflate\047 -H \047Accept-Language: en-US,en;q=0.5\047 -H \047Connection: keep-alive\047 -H \047Cookie:cookie-stuff\047 -H \047Host: maps.google.com\047 -H \047User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0\047 -o " $1 ".kml")}' (And the above doesn't look quite as bad as what I was doing without the actual cookie data there, so that's saying something.) At which point I took a few minutes to think deeply about life and what possessed me to write such an obscenity, and decided to use Takeout to get the data instead. Edit: That's not to say it didn't work, it gave me all the data. It was just a bad way to do it.
- adminprof 12y ago
  Thank you. I also spent a lot of time looking for a programmatic way to access the data after Latitude's demise, and decided that using another app was easier.
Tiksi 12y ago
I did something similar with that data a few months ago using qGIS, postGIS, and OSM data: https://packetfire.org/projects/qgis/img/lines.crop.png https://packetfire.org/projects/qgis/img/lines.crop.png https://packetfire.org/projects/qgis/img/points.and.lines.crop.png https://packetfire.org/projects/qgis/img/points.and.lines.cr... https://packetfire.org/projects/qgis/img/too.many.points.png https://packetfire.org/projects/qgis/img/too.many.points.png The data is definitely not completely consistent. Trips I took are missing, even though Google plus made auto-awesome stories with pictures taken there, and got the locations correct in those. There are also some points that shouldn't be there. It shows me in Michigan a few separate times (a few hundred data points at various times), to which I've never been. It also puts me at 0 Lat/0 Long a few times, but the reason for this is pretty obvious, just confusing when I first saw it mapped. Also there's quite a bit of "teleporting" from location to location, even though I had my phone with me and was using it when I was traveling between them. The third image, on the other hand, is seemingly way too much data at one place. It's my workplace at 1 meter resolution. You'd think they could merge it down to a few points a day at the most, but for whatever reason, there are on average 200 points per day there. It just astonishes me how much data there is at some places, yet in others massive chunks seem to be missing. I started working on a write up of how I did all this, including the bash scripts I wrote to parse, clean and transform the data, but one thing came up after another, and I haven't gotten around to finishing it. If anyone is interested, I'll finish that up and post it this week. I still have to finish up my scripts to clean the data of the "teleporting" and the places I haven't actually been. It was definitely a fun project, and now I have an interactive map of (some of) my past travels and habits. qGIS was interesting to play with, learning bits as I went along trying to get it to display properly and actually render what I wanted to see. Definitely gave me a new level of respect for anyone that does cartography.
- Bassetts 12y ago
  I have messed around with mapping my location history and had some similar problems. Apparently I have been to Essex, UK although I have never been there in my life. I tracked down the teleporting problem to the fact my tablet was at home and turned on. It seems you can't set location reporting per device, it is either on or off, so you end up being in two locations at once. I was going to look in to trying to remove problem points be setting a distance or speed threshold. That brings the problem of knowing which data point to keep and which to drop though, which I have not got a solution for yet apart from requiring user interaction.
dzhiurgis 12y ago
Am I the only one whose location data is missing?
- icebraining 12y ago
  Maybe you've disabled the feature? https://support.google.com/gmm/answer/3118687?hl=en https://support.google.com/gmm/answer/3118687?hl=en
ChrisArchitect 12y ago
I've done this before by just downloading location history as KML and then hosting somewhere and directing google maps at it in an embed etc. works fine.
shaurz 12y ago
So apparently I've been to Hong Kong... first I've heard of it. Also there seems to be no record of my visit to Canada last year.

62 points by wormold 12y ago

tghw 12y ago

For those who don't want to do it themselves: https://maps.google.com/locationhistory/b/0 https://maps.google.com/locationhistory/b/0

markveronda 12y ago
Actually, the Google-provided version restricts you to only 30 days. If you're interested in everywhere you have been, ever (according to Google), then this is quite awesome and you get much more options with how you display that data and what else you can incorporate with it.
- thekos 12y ago
  You can go back further by modifying the startTime parameter in the download link. https://maps.google.com/locationhistory/b/0/kml?startTime=1400003200000&endTime=1410159600000 https://maps.google.com/locationhistory/b/0/kml?startTime=14...
  - jacquesm 12y ago
    Oh my, a hacker. Better be careful. http://theblot.com/man-jailed-because-feds-dont-understand-computer-hacking-7717221 http://theblot.com/man-jailed-because-feds-dont-understand-c...

dalek2point3 12y ago

how does python know what map data to use?

markveronda 12y ago
It's fed in by the JSON file that was downloaded in Step 1 from Google when you request an archive of your data. From the site: # Create a dataframe from the json file in the filepath raw = pd.io.json.read_json('/Users/chrisralbon/Downloads/Location History/LocationHistory.json')

acadien 12y ago

Very cool script! I just tried it out myself and it seems that Google's location history is actually really spotty. It missed several of my trips (during which the phone was active and responsive) and it also included some spots that I've never been to. I double checked the JSON and the problem is on Google's end and not the script.

dochtman 12y ago

I built my own version of this about two years ago: https://dirkjan.ochtman.nl/writing/2012/11/28/tracing-a-path.html https://dirkjan.ochtman.nl/writing/2012/11/28/tracing-a-path... Python code available here: https://github.com/djc/galatna https://github.com/djc/galatna

sahaskatta 12y ago

I used a heatmap API with 2+ years worth of Google Location History data to make this last year: https://plus.google.com/+sahaskatta/posts/5bbn9D9ZzuP https://plus.google.com/+sahaskatta/posts/5bbn9D9ZzuP

IneffablePigeon 12y ago
This inspired me to do the same. If anyone wants to try, I put my implementation up on github: https://github.com/olane/locationheatmap https://github.com/olane/locationheatmap

edent 12y ago

If you want to get your data programatically, Google do not provide an API. You'll need to "borrow" your own cookies. I documented the process at http://shkspr.mobi/blog/2014/04/extracting-your-own-location-information-from-google-the-hard-way/ http://shkspr.mobi/blog/2014/04/extracting-your-own-location...

dochtman 12y ago
You can just use Google Takeout, IIRC.
- edent 12y ago
  Takeout does not provide an API.
Tiksi 12y ago
I once tried to quickly hack together some bash using "Copy as curl" in firebug to do something similar and got this shameful monstrosity after a minute or two of fighting with awk: for i in {10..14};do for j in 1 7; do date -d "20$i-0$j-01T00:11:20.000Z" "+%s000";date -d "20$i-0$j-01T00:11:20.000Z" "+%s000"|tr "\n" " "; done;done|sed '1d;$d'|awk '{system("curl \047https://maps.google.com/locationhistory/b/0/kml?startTime=" $1 "&endTime=" $2 "\047 -H \047Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\047 -H \047Accept-Encoding: gzip, deflate\047 -H \047Accept-Language: en-US,en;q=0.5\047 -H \047Connection: keep-alive\047 -H \047Cookie:cookie-stuff\047 -H \047Host: maps.google.com\047 -H \047User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0\047 -o " $1 ".kml")}' (And the above doesn't look quite as bad as what I was doing without the actual cookie data there, so that's saying something.) At which point I took a few minutes to think deeply about life and what possessed me to write such an obscenity, and decided to use Takeout to get the data instead. Edit: That's not to say it didn't work, it gave me all the data. It was just a bad way to do it.
adminprof 12y ago
Thank you. I also spent a lot of time looking for a programmatic way to access the data after Latitude's demise, and decided that using another app was easier.

Tiksi 12y ago

I did something similar with that data a few months ago using qGIS, postGIS, and OSM data: https://packetfire.org/projects/qgis/img/lines.crop.png https://packetfire.org/projects/qgis/img/lines.crop.png https://packetfire.org/projects/qgis/img/points.and.lines.crop.png https://packetfire.org/projects/qgis/img/points.and.lines.cr... https://packetfire.org/projects/qgis/img/too.many.points.png https://packetfire.org/projects/qgis/img/too.many.points.png The data is definitely not completely consistent. Trips I took are missing, even though Google plus made auto-awesome stories with pictures taken there, and got the locations correct in those. There are also some points that shouldn't be there. It shows me in Michigan a few separate times (a few hundred data points at various times), to which I've never been. It also puts me at 0 Lat/0 Long a few times, but the reason for this is pretty obvious, just confusing when I first saw it mapped. Also there's quite a bit of "teleporting" from location to location, even though I had my phone with me and was using it when I was traveling between them. The third image, on the other hand, is seemingly way too much data at one place. It's my workplace at 1 meter resolution. You'd think they could merge it down to a few points a day at the most, but for whatever reason, there are on average 200 points per day there. It just astonishes me how much data there is at some places, yet in others massive chunks seem to be missing. I started working on a write up of how I did all this, including the bash scripts I wrote to parse, clean and transform the data, but one thing came up after another, and I haven't gotten around to finishing it. If anyone is interested, I'll finish that up and post it this week. I still have to finish up my scripts to clean the data of the "teleporting" and the places I haven't actually been. It was definitely a fun project, and now I have an interactive map of (some of) my past travels and habits. qGIS was interesting to play with, learning bits as I went along trying to get it to display properly and actually render what I wanted to see. Definitely gave me a new level of respect for anyone that does cartography.

Bassetts 12y ago
I have messed around with mapping my location history and had some similar problems. Apparently I have been to Essex, UK although I have never been there in my life. I tracked down the teleporting problem to the fact my tablet was at home and turned on. It seems you can't set location reporting per device, it is either on or off, so you end up being in two locations at once. I was going to look in to trying to remove problem points be setting a distance or speed threshold. That brings the problem of knowing which data point to keep and which to drop though, which I have not got a solution for yet apart from requiring user interaction.

dzhiurgis 12y ago

Am I the only one whose location data is missing?

icebraining 12y ago
Maybe you've disabled the feature? https://support.google.com/gmm/answer/3118687?hl=en https://support.google.com/gmm/answer/3118687?hl=en

ChrisArchitect 12y ago

I've done this before by just downloading location history as KML and then hosting somewhere and directing google maps at it in an embed etc. works fine.

shaurz 12y ago

So apparently I've been to Hong Kong... first I've heard of it. Also there seems to be no record of my visit to Canada last year.