8 ms·

I have friends in law enforcement who use youtube-dl to save copies of videos involving crimes or criminal allegations, from quite a few different social media

I have friends in law enforcement who use youtube-dl to save copies of videos involving crimes or criminal allegations, from quite a few different social media and video sites. Speaking with them, it seems the associated metadata (json/xml?) and output log contain valuable information to go along with the video from an evidentiary perspective. Also the fact that youtube-dl has many test cases in the code goes towards demonstrating its reliability as an instrument for collecting evidence. Certainly seems a better approach than installing a 'DownloadVideosEzy' extension for chrome or similar.

bigiain 6y ago
Why was my first though here "I bet Palantir have their own fork of youtube-dl which they rent out to cops in a SaaS thing for the low low price of $ASTRONOMICAL-SUM-COMPARED-TO-ALL-THE-DONATIONS-THE-PROJECT-HAS-EVER-RECIEVED ???:
- dang 6y ago
  "Please don't use uppercase for emphasis. If you want to emphasize a word or phrase, put asterisks around it and it will get italicized." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. (I understand the environment variable convention but if you think about the reason for the above rule, it's clear what needs to take precedence.)
paulryanrogers 6y ago
Doesn't law enforcement have a means to subpoena Google for such evidence?
- kgog 6y ago
  Which way do you think takes less time and effort?
- neuronflux 6y ago
  Assuming the content is public, why would they need a subpoena?
  - echelon 6y ago
    Is availability on a locked, walled garden "public"? I think we can tell what the RIAA thinks about that.
    - deleted 6y ago
      [deleted]
    - JAlexoid 6y ago
      RIAA doesn't dispute that it's public. Being public doesn't mean publicly owned. It means publicly accessible... Like your local shopping mall.
  - cheeze 6y ago
    Because the RIAA has shown that this specific tool can't be 'public'. If we have to play by those rules, LEO should too. Boo hoo if it's harder.
    - bonoboTP 6y ago
      > RIAA has shown They haven't shown anything. They made a claim. We'll know what rules we have to play by if it comes to a court ruling.
      - cheeze 6y ago
        I mean, GitHub took it down, no?
        ekianjo 6y ago
        They follow the DMCA process.
        6y ago
- neodymiumphish 6y ago
  The process for a subpoena takes forever and isn't guaranteed. If I have a subject who has relevant video on a service like YouTube, and I decide to go the subpoena route, here's the process: - Find the exact video, document the URL, Content creator's account name, and video upload date. - Submit a preservation letter to Google to ensure the video will still be accessible when the subpoena is auctioned. - go to my prosecutor/legal team to determine what jurisdiction the subpoena has to go through. It could be the county I'm in, the county the subject lives in, the county the video was uploaded in (unlikely), or the county where Google hosts the video. - draft the subpoena and get it reviewed by whichever judge/magistrate applies. - wait who knows how long for the subpoena response, hopefully it comes back with the real video and important content. - face defense arguments in court that my subpoena was improperly obtained/submitted
  - bigiain 6y ago
    "It's _hard word_ doing policing right! Why don't we just chokehold this black guy till he confesses to a crime?" :sigh:
    - danielheath 6y ago
      I mean, 'taking a copy of publicly-available evidence' instead of asking someone else to provide it isn't exactly 'not doing policing properly'.
    - nullc 6y ago
      Maybe save that argument for where it applies? In this thread we're talking about publicly available data. The fact that the path that works for non-public data is burdensome is good because it reduces abuse. Demanding that it be used for public data too just creates more pressure to undermine the (already insufficient, IMO) protections that exist for private data.
    - vkou 6y ago
      I think that using youtube-dl to download a video that a suspect has uploaded to YouTube, and presenting it in court is an example of doing policing right. If you don't want your videos to be used against you in court, I would recommend not publicly broadcasting them to the entire world.
- hiisukun 6y ago
  A few points you might want to consider: * youtube-dl is not just for Youtube (ie. google). It supports heaps of sites, and youtube is probably less likely to have the sort of content I'm referring to than twitter, pornhub, liveleak, who knows. * I am in Australia, so for a subpoena (or similar) my friends would need to issue an MLAT request, which would indeed take a very long time. In fact, this often does happen, but in the meantime a formal preservation request (to the service provider) and a local copy (via something like youtube-dl) are important steps. * Many investigations don't go to court, for a litany of reasons (that's a pun I guess). In such cases, a subpoena isn't in context: law enforcement have to investigate what has happened, to figure out if it needs to go to court. * Some investigations are important but do not meet the 'serious crime' threshold for forcing companies or parties to provide information. If you can imagine being the victim of a minor crime, that someone filmed and put on twitter, I'm sure you can imagine the local police may want a copy of the video before the tweet gets deleted - but also know that there will never be an international cooperative justice process. It's just a local issue. * Publically posted video is very common and often important, and it is not surprising that the police need a tool to download it. There are so many sites and web browser technologies, and the cops are not efficient spending their time coding youtube-dl equivalents. I feel quite strongly about youtube-dl being taken down for copyright infringement by the RIAA, but have tried to answer in an informative way. Sorry if it comes across harsh - no ill will intended :)
dredmorbius 6y ago
Law enforcement already has an anti-circ exemption under 17 USC 1201(e), though a finding against youtube-dl would limit it's (or similar tools') availability and/or functionality. https://www.law.cornell.edu/uscode/text/17/1201 https://www.law.cornell.edu/uscode/text/17/1201
- nneonneo 6y ago
  Nit: I presume you mean 17 USC 1201 (e) " Law Enforcement, Intelligence, and Other Government Activities."
  - dredmorbius 6y ago
    Good catch, fixed, thanks.
- AnthonyMouse 6y ago
  Honest question: How is Section 1201 anything other than incoherent? Suppose there is some legitimate non-infringing use of some material. Interoperability, law enforcement use, fair use, accessing public domain material which is distributed using the same technological measures, whatever. You create and distribute a piece of software to interface with the material for those purposes. It's the same piece of software as one designed for any other purposes, because the software has no knowledge of your intent and your intent has no technical effect on its operation. If that is allowed, doesn't that make 1201 a dead letter? If you make the tool available for non-infringing uses, the tool is available. But if that isn't allowed, doesn't that make 1201 a contemptible offense against all of those important interests? And possibly unconstitutional as a result?
  - CobrastanJorji 6y ago
    The tool itself is not inherently illegal. Intent and marketing is the only important factor. A tool designed to be used by police and marketed towards police is legal, as its intended purpose is not infringement. A tool advertised as "hey, kids, use this to infringe those copyrights" is totally different, even if it's literally the same tool.
tracker1 6y ago
It seems the referenced URLs in question in the source are part of unit testing. Aside: Wonder how gitlab handles DMCA requests.
- filmgirlcw 6y ago
  It looks pretty consistent with GitHub https://about.gitlab.com/handbook/dmca/ https://about.gitlab.com/handbook/dmca/ For a self-hosted instance, obviously it would be different. In that case, I assume the notice would go to the webhost.
MKinley 6y ago
I am not in law enforcement myself, but do some online investigations, and like many others in my field we use YouTube-dl to save a copy of video evidence relevant to a case we are working on. It can be instrumental for archiving the evidence if it is ever removed or taken down, can also be used to grab extra info (CC text log is one example) and then manually searched for specific strings. This tool has made ma y investigations pay off in ways they never could have otherwise.
- beagle3 6y ago
  How is this significantly different than placing a video camera in front of a screen? Yt-dL is surely nicer and more convenient, but from an evidence standpoint, a one time $500 (to go wild) setup should have you covered, no? (Mourning the loss of ytdl myself but trying to be realistic)
  - judge2020 6y ago
    Using a screen recorder like shadowplay or obs also doesn't seem like it would be out of the realm of possibility.
  - talaketu 6y ago
    Obviously CC stream is not readily searchable if you record it with a video camera.
    - beagle3 6y ago
      Well, sure, but google/YouTube search does find in it. The GP was talking about their work collecting evidence - they can find it just as well, and record a copy for posterity just as well. I am not saying it’s as convenient (more options >> less options except for analysis-paralysis). But I don’t understand how it tips the scale to making any archiving or evidence gathering unusable or uneconomical. (I am not saying GP is wrong - just want more explanation so I can understand)
      - deleted 6y ago
b0tzzzzzzman 6y ago
Well the RIAA will most likely create the opposite outcome by removing YTDL. Now boat loads of people who did not know about this tool will find it. Am I mistaken in thinking this source code only existed on one website .. Now owned by m$ ? Imagine if source code was posted on some type of blockchain like ETH2.. Or a decentralized file sharing service, like IPFS. I wonder if websites or self hosted interpreters using some kind of services like those would make serving a take down request nearly impossible. Well for now we can pay m$ to host code in ways people rely on for the long term. Until, they don't and the codes all taken down in something analogous to modern book burning.

bigiain 6y ago

Why was my first though here "I bet Palantir have their own fork of youtube-dl which they rent out to cops in a SaaS thing for the low low price of $ASTRONOMICAL-SUM-COMPARED-TO-ALL-THE-DONATIONS-THE-PROJECT-HAS-EVER-RECIEVED ???:

dang 6y ago
"Please don't use uppercase for emphasis. If you want to emphasize a word or phrase, put asterisks around it and it will get italicized." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. (I understand the environment variable convention but if you think about the reason for the above rule, it's clear what needs to take precedence.)

paulryanrogers 6y ago

Doesn't law enforcement have a means to subpoena Google for such evidence?

kgog 6y ago
Which way do you think takes less time and effort?
neuronflux 6y ago
Assuming the content is public, why would they need a subpoena?
- echelon 6y ago
  Is availability on a locked, walled garden "public"? I think we can tell what the RIAA thinks about that.
  - deleted 6y ago
    [deleted]
  - JAlexoid 6y ago
    RIAA doesn't dispute that it's public. Being public doesn't mean publicly owned. It means publicly accessible... Like your local shopping mall.
- cheeze 6y ago
  Because the RIAA has shown that this specific tool can't be 'public'. If we have to play by those rules, LEO should too. Boo hoo if it's harder.
  - bonoboTP 6y ago
    > RIAA has shown They haven't shown anything. They made a claim. We'll know what rules we have to play by if it comes to a court ruling.
    - cheeze 6y ago
      I mean, GitHub took it down, no?
      - ekianjo 6y ago
        They follow the DMCA process.
      - 6y ago
neodymiumphish 6y ago
The process for a subpoena takes forever and isn't guaranteed. If I have a subject who has relevant video on a service like YouTube, and I decide to go the subpoena route, here's the process: - Find the exact video, document the URL, Content creator's account name, and video upload date. - Submit a preservation letter to Google to ensure the video will still be accessible when the subpoena is auctioned. - go to my prosecutor/legal team to determine what jurisdiction the subpoena has to go through. It could be the county I'm in, the county the subject lives in, the county the video was uploaded in (unlikely), or the county where Google hosts the video. - draft the subpoena and get it reviewed by whichever judge/magistrate applies. - wait who knows how long for the subpoena response, hopefully it comes back with the real video and important content. - face defense arguments in court that my subpoena was improperly obtained/submitted
- bigiain 6y ago
  "It's _hard word_ doing policing right! Why don't we just chokehold this black guy till he confesses to a crime?" :sigh:
  - danielheath 6y ago
    I mean, 'taking a copy of publicly-available evidence' instead of asking someone else to provide it isn't exactly 'not doing policing properly'.
  - nullc 6y ago
    Maybe save that argument for where it applies? In this thread we're talking about publicly available data. The fact that the path that works for non-public data is burdensome is good because it reduces abuse. Demanding that it be used for public data too just creates more pressure to undermine the (already insufficient, IMO) protections that exist for private data.
  - vkou 6y ago
    I think that using youtube-dl to download a video that a suspect has uploaded to YouTube, and presenting it in court is an example of doing policing right. If you don't want your videos to be used against you in court, I would recommend not publicly broadcasting them to the entire world.
hiisukun 6y ago
A few points you might want to consider: * youtube-dl is not just for Youtube (ie. google). It supports heaps of sites, and youtube is probably less likely to have the sort of content I'm referring to than twitter, pornhub, liveleak, who knows. * I am in Australia, so for a subpoena (or similar) my friends would need to issue an MLAT request, which would indeed take a very long time. In fact, this often does happen, but in the meantime a formal preservation request (to the service provider) and a local copy (via something like youtube-dl) are important steps. * Many investigations don't go to court, for a litany of reasons (that's a pun I guess). In such cases, a subpoena isn't in context: law enforcement have to investigate what has happened, to figure out if it needs to go to court. * Some investigations are important but do not meet the 'serious crime' threshold for forcing companies or parties to provide information. If you can imagine being the victim of a minor crime, that someone filmed and put on twitter, I'm sure you can imagine the local police may want a copy of the video before the tweet gets deleted - but also know that there will never be an international cooperative justice process. It's just a local issue. * Publically posted video is very common and often important, and it is not surprising that the police need a tool to download it. There are so many sites and web browser technologies, and the cops are not efficient spending their time coding youtube-dl equivalents. I feel quite strongly about youtube-dl being taken down for copyright infringement by the RIAA, but have tried to answer in an informative way. Sorry if it comes across harsh - no ill will intended :)

dredmorbius 6y ago

Law enforcement already has an anti-circ exemption under 17 USC 1201(e), though a finding against youtube-dl would limit it's (or similar tools') availability and/or functionality. https://www.law.cornell.edu/uscode/text/17/1201 https://www.law.cornell.edu/uscode/text/17/1201

nneonneo 6y ago
Nit: I presume you mean 17 USC 1201 (e) " Law Enforcement, Intelligence, and Other Government Activities."
- dredmorbius 6y ago
  Good catch, fixed, thanks.
AnthonyMouse 6y ago
Honest question: How is Section 1201 anything other than incoherent? Suppose there is some legitimate non-infringing use of some material. Interoperability, law enforcement use, fair use, accessing public domain material which is distributed using the same technological measures, whatever. You create and distribute a piece of software to interface with the material for those purposes. It's the same piece of software as one designed for any other purposes, because the software has no knowledge of your intent and your intent has no technical effect on its operation. If that is allowed, doesn't that make 1201 a dead letter? If you make the tool available for non-infringing uses, the tool is available. But if that isn't allowed, doesn't that make 1201 a contemptible offense against all of those important interests? And possibly unconstitutional as a result?
- CobrastanJorji 6y ago
  The tool itself is not inherently illegal. Intent and marketing is the only important factor. A tool designed to be used by police and marketed towards police is legal, as its intended purpose is not infringement. A tool advertised as "hey, kids, use this to infringe those copyrights" is totally different, even if it's literally the same tool.

tracker1 6y ago

It seems the referenced URLs in question in the source are part of unit testing. Aside: Wonder how gitlab handles DMCA requests.

filmgirlcw 6y ago
It looks pretty consistent with GitHub https://about.gitlab.com/handbook/dmca/ https://about.gitlab.com/handbook/dmca/ For a self-hosted instance, obviously it would be different. In that case, I assume the notice would go to the webhost.

MKinley 6y ago

I am not in law enforcement myself, but do some online investigations, and like many others in my field we use YouTube-dl to save a copy of video evidence relevant to a case we are working on. It can be instrumental for archiving the evidence if it is ever removed or taken down, can also be used to grab extra info (CC text log is one example) and then manually searched for specific strings. This tool has made ma y investigations pay off in ways they never could have otherwise.

beagle3 6y ago
How is this significantly different than placing a video camera in front of a screen? Yt-dL is surely nicer and more convenient, but from an evidence standpoint, a one time $500 (to go wild) setup should have you covered, no? (Mourning the loss of ytdl myself but trying to be realistic)
- judge2020 6y ago
  Using a screen recorder like shadowplay or obs also doesn't seem like it would be out of the realm of possibility.
- talaketu 6y ago
  Obviously CC stream is not readily searchable if you record it with a video camera.
  - beagle3 6y ago
    Well, sure, but google/YouTube search does find in it. The GP was talking about their work collecting evidence - they can find it just as well, and record a copy for posterity just as well. I am not saying it’s as convenient (more options >> less options except for analysis-paralysis). But I don’t understand how it tips the scale to making any archiving or evidence gathering unusable or uneconomical. (I am not saying GP is wrong - just want more explanation so I can understand)
    - deleted 6y ago

b0tzzzzzzman 6y ago

Well the RIAA will most likely create the opposite outcome by removing YTDL. Now boat loads of people who did not know about this tool will find it. Am I mistaken in thinking this source code only existed on one website .. Now owned by m$ ? Imagine if source code was posted on some type of blockchain like ETH2.. Or a decentralized file sharing service, like IPFS. I wonder if websites or self hosted interpreters using some kind of services like those would make serving a take down request nearly impossible. Well for now we can pay m$ to host code in ways people rely on for the long term. Until, they don't and the codes all taken down in something analogous to modern book burning.