4 ms·

I would argue that the fact that video content is often so frustrating to scrape is symptomatic of the massive cultural shift the web has seen over the past dec

by simias 6y ago

I would argue that the fact that video content is often so frustrating to scrape is symptomatic of the massive cultural shift the web has seen over the past decade or two. In the "old" web you could basically right-click -> save anything you want (ignoring plugin blobs for the sake of making my point more compelling). Meanwhile I tried manually scrapping a video from reddit the other day, I succeeded but it was quite the puzzle. First you have to find the right element in the DOM, then find the source which is a playlist for the various qualities available, then there was a 2nd indirection I think, then finally I got the URL of my video file. My browser obviously knows about all of this but it won't expose it easily. No "Save video as" menu. For some reason I don't expect that Chrome will implement it any time soon...

chrismorgan 6y ago
Firefox exposes this stuff just fine: its context menu on videos has “View Video”, “Copy Video Location” and “Save Video As…”. (I thought that Chrome did too, but maybe my memory is faulty or maybe they removed it at some point.) But sites can easily prevent all of this stuff from happening by putting something transparent on top of the video, so that you’re not actually right clicking on the video, but on another element. And here’s the thing—there’s a sound technical reason why they all do that, and it’s not just legal nonsense. Those operations only make sense if you’re playing one video file, but as you remark, there are all the various qualities available, and all of these platforms are designed to be able to switch seamlessly between sources, so that the browser never does get told “play this single video file, it’s the whole thing”, but rather “here, play this (1080p) chunk”, then “here, now this (144p) chunk”, then “and now this 360p”—with the JavaScript monitoring everything and trying to make it flow as smoothly as possible at each step. So… no, your browser actually doesn’t know about all of this, and that’s the reason that YouTube and the likes have stopped it from exposing it even in browsers that are otherwise willing to. (You could still easily say that there’s a culture shift in this: in the distant past, the web valued simplicity and openness at the cost of performance and effectiveness; but as the internet went more and more mainstream, more and more developers broke ranks and insisted on unbreaking things for their users and improving performance and effectiveness, even at the cost of openness and simplicity.)
- simias 6y ago
  > Firefox exposes this stuff just fine: its context menu on videos has “View Video”, “Copy Video Location” and “Save Video As…”. (I thought that Chrome did too, but maybe my memory is faulty or maybe they removed it at some point.) You're right, I never noticed that. Although I just tried on youtube and the option are there... but greyed out. Not sure why. As for the various formats, if the web wasn't a javascript shitfest that would be exposed to the browser and it would decide what format it wants to use. That would help with all these websites with a crappy streaming implementation. That would also make downloading very easy, you could just select which format and bitrate you want.
  - moonchild 6y ago
    > greyed out. Not sure why. DRM, maybe?
  - chrismorgan 6y ago
    Greyed out because there is no video location for you to view or save. It’s playing a stream, put together from many pieces, so that it performs and functions better. Switching between sources mid-play is a lot harder than most realise, because random seek is expensive in videos. So instead you tend to treat it as a large number of files of a few seconds each that get glued together as you play it, and you can switch between them as you go. The web platform could be extended to cover multiple-bitrate streaming and let the browser take control, but that’s a pretty complex affair, and I suspect that it still wouldn’t be enough to make these operations work—the browser would need a way of putting it back together again into one file.
userbinator 6y ago
What's most ironic is that HTML5 added a <video> element specifically to make videos as easy and convenient to manipulate as images.

by simias 6y ago

chrismorgan 6y ago

Firefox exposes this stuff just fine: its context menu on videos has “View Video”, “Copy Video Location” and “Save Video As…”. (I thought that Chrome did too, but maybe my memory is faulty or maybe they removed it at some point.) But sites can easily prevent all of this stuff from happening by putting something transparent on top of the video, so that you’re not actually right clicking on the video, but on another element. And here’s the thing—there’s a sound technical reason why they all do that, and it’s not just legal nonsense. Those operations only make sense if you’re playing one video file, but as you remark, there are all the various qualities available, and all of these platforms are designed to be able to switch seamlessly between sources, so that the browser never does get told “play this single video file, it’s the whole thing”, but rather “here, play this (1080p) chunk”, then “here, now this (144p) chunk”, then “and now this 360p”—with the JavaScript monitoring everything and trying to make it flow as smoothly as possible at each step. So… no, your browser actually doesn’t know about all of this, and that’s the reason that YouTube and the likes have stopped it from exposing it even in browsers that are otherwise willing to. (You could still easily say that there’s a culture shift in this: in the distant past, the web valued simplicity and openness at the cost of performance and effectiveness; but as the internet went more and more mainstream, more and more developers broke ranks and insisted on unbreaking things for their users and improving performance and effectiveness, even at the cost of openness and simplicity.)

simias 6y ago
> Firefox exposes this stuff just fine: its context menu on videos has “View Video”, “Copy Video Location” and “Save Video As…”. (I thought that Chrome did too, but maybe my memory is faulty or maybe they removed it at some point.) You're right, I never noticed that. Although I just tried on youtube and the option are there... but greyed out. Not sure why. As for the various formats, if the web wasn't a javascript shitfest that would be exposed to the browser and it would decide what format it wants to use. That would help with all these websites with a crappy streaming implementation. That would also make downloading very easy, you could just select which format and bitrate you want.
- moonchild 6y ago
  > greyed out. Not sure why. DRM, maybe?
- chrismorgan 6y ago
  Greyed out because there is no video location for you to view or save. It’s playing a stream, put together from many pieces, so that it performs and functions better. Switching between sources mid-play is a lot harder than most realise, because random seek is expensive in videos. So instead you tend to treat it as a large number of files of a few seconds each that get glued together as you play it, and you can switch between them as you go. The web platform could be extended to cover multiple-bitrate streaming and let the browser take control, but that’s a pretty complex affair, and I suspect that it still wouldn’t be enough to make these operations work—the browser would need a way of putting it back together again into one file.

userbinator 6y ago

What's most ironic is that HTML5 added a <video> element specifically to make videos as easy and convenient to manipulate as images.