3 ms·

Slightly related to this, the other day I tried to download every youtube video in my watch history. Turns out it's pretty much impossible. The Youtube API onl

Slightly related to this, the other day I tried to download every youtube video in my watch history. Turns out it's pretty much impossible. The Youtube API only delivers about 20 results, which is a bug that has existed for about 2 years. I tried manually loading the watch history page, and was only able to get about 1000 out of ~8000 results. When I selected those 1000 results and tried to add them to a playlist, the interface crashed. Does anyone have a solution for this, or is my watch history just at the mercy of Google?

lutusp 12y ago
> The Youtube API only delivers about 20 results, which is a bug that has existed for about 2 years. That may not be a bug. It might be a way to limit the traffic created by scrapers who submit requests and then download all the videos listed on the result page.
- sroerick 12y ago
  Hrm, then why offer the feature? I'm kind of still a beginner when it comes to APIs, but it seems disingenuous to offer an API feature and then intentionally break it so that it can't be used.
  - lutusp 12y ago
    > Hrm, then why offer the feature? Because people can still get 20 results. I mean, 20 results is way better than no results. > I'm kind of still a beginner when it comes to APIs, but it seems disingenuous to offer an API feature and then intentionally break it so that it can't be used. It's not broken. It returns 20 results. Maybe Google decided that was enough of a hit on their database. One could also argue that most people wouldn't want more than 20 results on their small-screen Android device served by a slow connection.
    - sroerick 12y ago
      https://code.google.com/p/gdata-issues/issues/detail?can=2&start=0&num=100&q=&colspec=API%20ID%20Type%20Status%20Priority%20Stars%20Summary&groupby=&sort=&id=4642 https://code.google.com/p/gdata-issues/issues/detail?can=2&s... The number of results returned is random. Some people report seeing videos watched on computers but not locally. One person says it works. That's not a limit. That's broken.
yuvadam 12y ago
You can use youtube-dl to do this: $ youtube-dl ':ythistory' -u USER -p PWD --write-pages Of course, you won't get your entire history since g00gle would rather mine all your personal data itself, and never share it back with you again.
- sroerick 12y ago
  I tried this earlier. The result: [youtube:history] playlist Youtube Watch History: Collected 0 video ids (downloading 0 of them)
  - eric_h 12y ago
    Just a thought, do you have google two factor auth turned on? Maybe you need to generate an application password and use that to get access.
hnha 12y ago
It baffles me how we got from "tracking is creepy" to "please track all I do on your site" in ~10 years. Your watch history should be stored by your browser on your machine. It is only your business and only locally it is fully in your control.
- sroerick 12y ago
  I agree with you. My watch history is currently in the hands of Google. I'm trying to copy it so that I have it saved locally.
- thegeomaster 12y ago
  A week ago, I had my phone forcefully smashed into a floor. The screen ended up completely dead, but fortunately, the phone worked, and the remote debugging option was on, so I was able to pull all the data on it via ADB. But if the motherboard or some other vital had component died, I would've beem locked out of my data. It's at times like this I'm glad that Google Keep syncs my notes to their servers, and that Viber stores my contacts' phone numbers remotely. If it hadn't been for them and my phone, say, fell in water and completely broke, I'd have lost all of that. Same argument for a hard drive death. In 2004 or so, we were expected to live with that risk. But I don't think we should have to now. Witnessing the explosion of the "cloud" buzzword everywhere, you'd think everyone in the world has convenient access to their own private space "in the cloud" (or, as those soooo-2000s people would say, "on the Internet"). And we do have access to such space, but for the most part, not quite so for the "private" thing. Of course, ideally, I'd like to generate random keys, keep safe local backups of them and then sync encrypted data to remote servers. But I don't think the companies are eager to accept that as they probably owe a lot of their statistics and targeted advertising opportunities to big-scale mining of that plaintext data we provide them in exchange for convenience.

lutusp 12y ago

> The Youtube API only delivers about 20 results, which is a bug that has existed for about 2 years. That may not be a bug. It might be a way to limit the traffic created by scrapers who submit requests and then download all the videos listed on the result page.

sroerick 12y ago
Hrm, then why offer the feature? I'm kind of still a beginner when it comes to APIs, but it seems disingenuous to offer an API feature and then intentionally break it so that it can't be used.
- lutusp 12y ago
  > Hrm, then why offer the feature? Because people can still get 20 results. I mean, 20 results is way better than no results. > I'm kind of still a beginner when it comes to APIs, but it seems disingenuous to offer an API feature and then intentionally break it so that it can't be used. It's not broken. It returns 20 results. Maybe Google decided that was enough of a hit on their database. One could also argue that most people wouldn't want more than 20 results on their small-screen Android device served by a slow connection.
  - sroerick 12y ago
    https://code.google.com/p/gdata-issues/issues/detail?can=2&start=0&num=100&q=&colspec=API%20ID%20Type%20Status%20Priority%20Stars%20Summary&groupby=&sort=&id=4642 https://code.google.com/p/gdata-issues/issues/detail?can=2&s... The number of results returned is random. Some people report seeing videos watched on computers but not locally. One person says it works. That's not a limit. That's broken.

yuvadam 12y ago

You can use youtube-dl to do this: $ youtube-dl ':ythistory' -u USER -p PWD --write-pages Of course, you won't get your entire history since g00gle would rather mine all your personal data itself, and never share it back with you again.

sroerick 12y ago
I tried this earlier. The result: [youtube:history] playlist Youtube Watch History: Collected 0 video ids (downloading 0 of them)
- eric_h 12y ago
  Just a thought, do you have google two factor auth turned on? Maybe you need to generate an application password and use that to get access.

hnha 12y ago

It baffles me how we got from "tracking is creepy" to "please track all I do on your site" in ~10 years. Your watch history should be stored by your browser on your machine. It is only your business and only locally it is fully in your control.

sroerick 12y ago
I agree with you. My watch history is currently in the hands of Google. I'm trying to copy it so that I have it saved locally.
thegeomaster 12y ago
A week ago, I had my phone forcefully smashed into a floor. The screen ended up completely dead, but fortunately, the phone worked, and the remote debugging option was on, so I was able to pull all the data on it via ADB. But if the motherboard or some other vital had component died, I would've beem locked out of my data. It's at times like this I'm glad that Google Keep syncs my notes to their servers, and that Viber stores my contacts' phone numbers remotely. If it hadn't been for them and my phone, say, fell in water and completely broke, I'd have lost all of that. Same argument for a hard drive death. In 2004 or so, we were expected to live with that risk. But I don't think we should have to now. Witnessing the explosion of the "cloud" buzzword everywhere, you'd think everyone in the world has convenient access to their own private space "in the cloud" (or, as those soooo-2000s people would say, "on the Internet"). And we do have access to such space, but for the most part, not quite so for the "private" thing. Of course, ideally, I'd like to generate random keys, keep safe local backups of them and then sync encrypted data to remote servers. But I don't think the companies are eager to accept that as they probably owe a lot of their statistics and targeted advertising opportunities to big-scale mining of that plaintext data we provide them in exchange for convenience.