5 ms·

i'm very curious about something -- people here rave about the takeout -- does anyone actually use the thing? i've tried the takeout option about a year ago wh

by ir77 6y ago

i'm very curious about something -- people here rave about the takeout -- does anyone actually use the thing? i've tried the takeout option about a year ago when i wanted to switch from google photos to apple photos and it was an absolute mess, it exported random zipped folders, all broken up (i think there were 14 of thema ll together), multiple images were duplicates, somehow it managed to corrupt apple format of pictures and i would have one picture with the same name that was something like 24KB and another one with 1.2MB, etc. if it wasn't for the option (which they took away) of syncing photos to your google drive and copying them out that way, i have no idea what i'd have done, i'd have lost about 120GB of photos and videos going back to 2008. so seriously, do all of you just use the takeout and store it away without actually opening up and trying to use it?

OskarS 6y ago
My brother passed away very suddenly a couple of years ago. Google Takeout and similar services were a godsend for extracting and archiving everything from his accounts. As far as I know, everything exported fine, thiugh he didn’t use Google Photos.
dawnerd 6y ago
I just used it to pull over 350gigs of photos. Not fun. They really make it hard to reimport photos elsewhere. I wish their app just had a download all.
axaxs 6y ago
As I wrote elsewhere, previously. I agree it's a mess, but not the end of the world. Extract the file -not- preserving directory structure(-j flag). Now you just have everything in a single folder. Then delete the metadata files by file extension. This took all of a few seconds to do for each zip.
pydry 6y ago
I think people love the idea of takeout more than the thing itself.
rpdillon 6y ago
I just finished found through the takeout process for photos last night. 39 2GB archives. Request took a couple of days to process, then I got an email with a bunch of links that they said were good for 7 days. I planned to load them into my Synology download manager, but the takeout system seems to rely heavily on browser state, so Synology couldn't download them. Each took 5 minutes to download at home over WiFi, and the takeout interface demands authentication every 10 minutes. It also reloads the page and reset my position in the download list on every click. I've uploaded and expanded about half the archives on my Synology, and it's currently indexing everything, so I can't comment on the photo issues you've mentioned quite yet. Overall, I'm happy there exists a mechanism to get my photos, but the quality of the experience is truly awful.
- mtnGoat 6y ago
  Almost like they made the ux really hard to use for a reason, eh? All that engineering talent and that's the best solution? Lol.
  - 101011 6y ago
    More like, they're just wanting to make sure they're legally covered. There is no added benefit for the company in making that experience better. Source: working on a product engineering team for a larger company. Why would we upheave our roadmap for something that doesn't help provide explicit value to our product offerings? We've got a list a mile long of improvements and new capabilities to deliver on, and no engineer likes working through legal hoops anyways.
    - pdimitar 6y ago
      I don't see how your statement contradicts your parent comment's. Looks like Google just did the bare minimum to cover themselves legally (as you said). But that shouldn't mean that they can't make the experience better. As for no engineer liking working through legal hoops, my observations disagree. Many absolutely don't care. Give them a well-described Jira ticket and they'll happily chip away at it for a year if it's necessary.
karlicoss 6y ago
I think breaking up is a ZIP limitation (max archive size is 4GB) I'm exporting it regularly (although I certainly don't have 120Gb of photos on it). You can choose an option for regular exports (every two months), and delivery method (e.g. Google Drive). Then I have a script that runs daily, mounts google drive, moves the takeout locally if it's present and removes from google drive (so it doesn't take space). Then indeed, inside you have mess with some data in HTML, some in JSON, etc. But well, at least you can parse it... I have a library which I'm using as an API to various data exports, in particular, archived takeouts too (so I don't even have to unpack them to access) - https://github.com/karlicoss/HPI/blob/master/my/google/takeout/html.py https://github.com/karlicoss/HPI/blob/master/my/google/takeo... - https://github.com/karlicoss/HPI/blob/master/my/location/google.py https://github.com/karlicoss/HPI/blob/master/my/location/goo... - https://github.com/karlicoss/HPI/blob/master/my/media/youtube.py https://github.com/karlicoss/HPI/blob/master/my/media/youtub... Described this in more detail here: https://beepb00p.xyz/my-data.html#takeout https://beepb00p.xyz/my-data.html#takeout
- edoceo 6y ago
  Hmm, I've seen ZIP files over 4GiB - not from takeout but from others
  - Closi 6y ago
    The original specification was 4GB, although there is an extension to the specification that allows for over 4GB (64 bit is required).
  - akx 6y ago
    That'd be ZIP64 -- see https://en.wikipedia.org/wiki/ZIP_(file_format)#ZIP64 https://en.wikipedia.org/wiki/ZIP_(file_format)#ZIP64
- ir77 6y ago
  "Then indeed, inside you have mess with some data in HTML, some in JSON, etc. But well, at least you can parse it..." how is some regular schmuck that wants to move his data out of google to another service supposed to determine what they actually have to parse? the user simply uploads pictures into the system but gets garbage out? the scarry part here is that google makes it extermely easy to suck in the data but for an average user it's extremly difficult to get back out and takeout is absolutely not a good solution.
jkingsman 6y ago
I use this one every other month -- download everything and throw it in cold storage. Mainly keeping it for the scary possibility of being kicked out of my account.
- SirBikesALot 6y ago
  What do you use for cold storage? I've had bad luck with external hard drives dying over time.
benlivengood 6y ago
Gzipped tarballs are an option now with what looks like a 40GB max. $ ls -lh takeout* -rw-r--r-- 1 ben ben 39G Sep 26 18:29 takeout-20200925T172738Z-001.tgz -rw-r--r-- 1 ben ben 38G Sep 26 19:20 takeout-20200925T172738Z-002.tgz -rw-r--r-- 1 ben ben 35G Sep 26 20:06 takeout-20200925T172738Z-003.tgz
- parliament32 6y ago
  The limit is 50gb on tgz takeouts.
parliament32 6y ago
Use the tgz option for exporting, the zip format has all sorts of limitations and weirdness.
tomjen3 6y ago
Used it to export all my google plus social stuff (several gb over the years) before they shut it down. It kinda worked okay.
slrey 6y ago
Shameless selfplug: As part of ongoing privacy research me and colleagues developed a website that parses personal data exports/takeouts from google, twitter, instagram and facebook and visualizes the data in a treemap and a timeline. We aim toincrease the awareness of personal data and the effects of online behaviour. The data is not uploaded and instead parsed in the browser. For the Google takeout make sure not to include data from Photos, Gmail, Youtube and Drive as they make the export too big. Also select "JSON" for "My Activity". Feel free to try it out yourself: http://transparency-vis.vx.igd.fraunhofer.de/visualization http://transparency-vis.vx.igd.fraunhofer.de/visualization
tambeb 6y ago
I do a full takeout every month or so, both for myself and also my parents. It's largely so that we don't lose anything if Google does one of its awesome sudden account disablings.
domano 6y ago
Exports as tar instead of zip. Every photo has an additional json with some kind of metadata. For me it was fine.

by ir77 6y ago

OskarS 6y ago

My brother passed away very suddenly a couple of years ago. Google Takeout and similar services were a godsend for extracting and archiving everything from his accounts. As far as I know, everything exported fine, thiugh he didn’t use Google Photos.

dawnerd 6y ago

I just used it to pull over 350gigs of photos. Not fun. They really make it hard to reimport photos elsewhere. I wish their app just had a download all.

axaxs 6y ago

As I wrote elsewhere, previously. I agree it's a mess, but not the end of the world. Extract the file -not- preserving directory structure(-j flag). Now you just have everything in a single folder. Then delete the metadata files by file extension. This took all of a few seconds to do for each zip.

pydry 6y ago

I think people love the idea of takeout more than the thing itself.

rpdillon 6y ago

I just finished found through the takeout process for photos last night. 39 2GB archives. Request took a couple of days to process, then I got an email with a bunch of links that they said were good for 7 days. I planned to load them into my Synology download manager, but the takeout system seems to rely heavily on browser state, so Synology couldn't download them. Each took 5 minutes to download at home over WiFi, and the takeout interface demands authentication every 10 minutes. It also reloads the page and reset my position in the download list on every click. I've uploaded and expanded about half the archives on my Synology, and it's currently indexing everything, so I can't comment on the photo issues you've mentioned quite yet. Overall, I'm happy there exists a mechanism to get my photos, but the quality of the experience is truly awful.

mtnGoat 6y ago
Almost like they made the ux really hard to use for a reason, eh? All that engineering talent and that's the best solution? Lol.
- 101011 6y ago
  More like, they're just wanting to make sure they're legally covered. There is no added benefit for the company in making that experience better. Source: working on a product engineering team for a larger company. Why would we upheave our roadmap for something that doesn't help provide explicit value to our product offerings? We've got a list a mile long of improvements and new capabilities to deliver on, and no engineer likes working through legal hoops anyways.
  - pdimitar 6y ago
    I don't see how your statement contradicts your parent comment's. Looks like Google just did the bare minimum to cover themselves legally (as you said). But that shouldn't mean that they can't make the experience better. As for no engineer liking working through legal hoops, my observations disagree. Many absolutely don't care. Give them a well-described Jira ticket and they'll happily chip away at it for a year if it's necessary.

karlicoss 6y ago

I think breaking up is a ZIP limitation (max archive size is 4GB) I'm exporting it regularly (although I certainly don't have 120Gb of photos on it). You can choose an option for regular exports (every two months), and delivery method (e.g. Google Drive). Then I have a script that runs daily, mounts google drive, moves the takeout locally if it's present and removes from google drive (so it doesn't take space). Then indeed, inside you have mess with some data in HTML, some in JSON, etc. But well, at least you can parse it... I have a library which I'm using as an API to various data exports, in particular, archived takeouts too (so I don't even have to unpack them to access) - https://github.com/karlicoss/HPI/blob/master/my/google/takeout/html.py https://github.com/karlicoss/HPI/blob/master/my/google/takeo... - https://github.com/karlicoss/HPI/blob/master/my/location/google.py https://github.com/karlicoss/HPI/blob/master/my/location/goo... - https://github.com/karlicoss/HPI/blob/master/my/media/youtube.py https://github.com/karlicoss/HPI/blob/master/my/media/youtub... Described this in more detail here: https://beepb00p.xyz/my-data.html#takeout https://beepb00p.xyz/my-data.html#takeout

edoceo 6y ago
Hmm, I've seen ZIP files over 4GiB - not from takeout but from others
- Closi 6y ago
  The original specification was 4GB, although there is an extension to the specification that allows for over 4GB (64 bit is required).
- akx 6y ago
  That'd be ZIP64 -- see https://en.wikipedia.org/wiki/ZIP_(file_format)#ZIP64 https://en.wikipedia.org/wiki/ZIP_(file_format)#ZIP64
ir77 6y ago
"Then indeed, inside you have mess with some data in HTML, some in JSON, etc. But well, at least you can parse it..." how is some regular schmuck that wants to move his data out of google to another service supposed to determine what they actually have to parse? the user simply uploads pictures into the system but gets garbage out? the scarry part here is that google makes it extermely easy to suck in the data but for an average user it's extremly difficult to get back out and takeout is absolutely not a good solution.

jkingsman 6y ago

I use this one every other month -- download everything and throw it in cold storage. Mainly keeping it for the scary possibility of being kicked out of my account.

SirBikesALot 6y ago
What do you use for cold storage? I've had bad luck with external hard drives dying over time.

benlivengood 6y ago

Gzipped tarballs are an option now with what looks like a 40GB max. $ ls -lh takeout* -rw-r--r-- 1 ben ben 39G Sep 26 18:29 takeout-20200925T172738Z-001.tgz -rw-r--r-- 1 ben ben 38G Sep 26 19:20 takeout-20200925T172738Z-002.tgz -rw-r--r-- 1 ben ben 35G Sep 26 20:06 takeout-20200925T172738Z-003.tgz

parliament32 6y ago
The limit is 50gb on tgz takeouts.

parliament32 6y ago

Use the tgz option for exporting, the zip format has all sorts of limitations and weirdness.

tomjen3 6y ago

Used it to export all my google plus social stuff (several gb over the years) before they shut it down. It kinda worked okay.

slrey 6y ago

Shameless selfplug: As part of ongoing privacy research me and colleagues developed a website that parses personal data exports/takeouts from google, twitter, instagram and facebook and visualizes the data in a treemap and a timeline. We aim toincrease the awareness of personal data and the effects of online behaviour. The data is not uploaded and instead parsed in the browser. For the Google takeout make sure not to include data from Photos, Gmail, Youtube and Drive as they make the export too big. Also select "JSON" for "My Activity". Feel free to try it out yourself: http://transparency-vis.vx.igd.fraunhofer.de/visualization http://transparency-vis.vx.igd.fraunhofer.de/visualization

tambeb 6y ago

I do a full takeout every month or so, both for myself and also my parents. It's largely so that we don't lose anything if Google does one of its awesome sudden account disablings.

domano 6y ago

Exports as tar instead of zip. Every photo has an additional json with some kind of metadata. For me it was fine.