14 ms·

I need to copy 2000+ DVDs in 3 days. What are my options?

393 points by huhtenberg 8y ago

huhtenberg 8y ago
The context - > Maybe you should disclose your city or at least state/general location Washington DC, specifically College Park, even more specifically The National Archives at College Park. > What is the content of the discs? Years ago in partnership with the National Archives Amazon.com digitized a tremendous amount of film for the National Archives, with the catch that the material cannot be freely disseminated online by the National Archives until Amazon breaks even on their digitization investment. It’s been years now and for a variety of reasons - many of which are Amazon’s fault - there still exist a solid number of discs that Amazon hasn’t sold even one of. (In part because amazon hasn’t even had them all available to sell. Pretty ridiculous). The flipside of the catch is that the DVDs can be viewed and even copied for $0 on site by researchers. I am doing research there and I have have a research pass. I was looking at many of these titles yesterday. It’s time to set these national treasures free.
- sergiotapia 8y ago
  Didn't Aaron Swartz get prosecuted and ultimately died because of doing what OP is attempting to do?
  - oh_sigh 8y ago
    No. Aaron died because he put a rope around his own neck and tightened it to the point that he could not breathe. He put the rope around his neck because he was depressed and suicidal, both characteristics which were present in him before he came to the attention of any prosecutor.
    - deleted 8y ago
      [deleted]
    - packetized 8y ago
      Factual information isn’t always useful in another context.
  - gammateam 8y ago
    No. And even if it was, millions of people can do the time, including programmers.
    - somabc 8y ago
      It's actually very similar to Aaron Schwartz's plan of using academic access to copy and widely distribute papers. This researcher does not have a legal basis to copy 2000+ disks in order to free them from the national archives. At most he has a restricted right of access to use some of them in his academic research. To test he could ask the national archive for permission to carry out his plan and observe how quickly his access is revoked. ----- Research Room Rules. The rules are intended to provide clear instructions to both staff and researchers on the research room: * registration process; * entry and exit procedures; * items authorized and not authorized for use in the * research rooms; * pull requests; * records handling guidelines; and * copying and equipment approvals. The rules will be available for researcher use in each facility or can be requested in writing from the facility managers listed below. Archives II, College Park, MD – Michael Knight, michael.knight@nara.gov
- veridies 8y ago
  I wonder what this material is. Are these commercial films, historical footage, or what?
  - eli 8y ago
    The OP says they're public domain, so I'd guess it's films produced by the government.
    - vsl 8y ago
      ...or past the copyright window.
  - Semaphor 8y ago
    > Years ago in partnership with the National Archives Amazon.com digitized a tremendous amount of film for the National Archives, with the catch that the material cannot be freely disseminated online by the National Archives until Amazon breaks even on their digitization investment. It’s been years now and for a variety of reasons - many of which are Amazon’s fault - there still exist a solid number of discs that Amazon hasn’t sold even one of. (In part because amazon hasn’t even had them all available to sell. Pretty ridiculous). > The flipside of the catch is that the DVDs can be viewed and even copied for $0 on site by researchers. I am doing research there and I have have a research pass. I was looking at many of these titles yesterday. It’s time to set these national treasures free. -- https://old.reddit.com/r/DataHoarder/comments/a6fkpm/i_have_temporary_3_days_access_to_more_than_2000/ebuf75j/?context=1 https://old.reddit.com/r/DataHoarder/comments/a6fkpm/i_have_...
    - traek 8y ago
      That doesn’t answer what type of media is on the disks.
      - lurker_primo 8y ago
        It says film. So, it could be movies or images. What else would be possible? Are you thinking of quickest ways to rip the DVDs if they are movies?
- shawn-butler 8y ago
  Seems counter-intuitive to Amazon's own public dataset program / policy: https://aws.amazon.com/opendata/public-datasets/ https://aws.amazon.com/opendata/public-datasets/
  - kkarakk 8y ago
    >Democratize access to data by making it available for analysis on AWS. >AWS evaluates applications to the AWS Public Dataset Program every three months. >If we bring your dataset into the AWS Public Dataset Program, we will cover the costs of storage and data transfer for a period of two years yeah i don't think that's public,more like a rental service. i wouldn't bother working with that unless i had free AWS access through corporate
- rasz 8y ago
  >In part because amazon hasn’t even had them all available to sell. Pretty ridiculous not ridiculous, good lawyer preparing the deal
  - morrbo 8y ago
    Not ridiculous, but very sad
  - SmellyGeekBoy 8y ago
    What's "good" about deliberately keeping public domain content out of the public domain?
    - allwein 8y ago
      Not "good" as in "good vs. evil", but rather "good" as in "proficient at the skill of lawyering".
- brett40324 8y ago
  Im wondering how the already preserved and publicly available film content is being considered being made freed ? Theres very clear information on rights, reproduction restrictions, and the preservation efforts that have already been made here: https://www.archives.gov/research/order/film-sound-video-dc.html https://www.archives.gov/research/order/film-sound-video-dc....
- amluto 8y ago
  Easy: get a bunch of DVD drives and a ~1 Gbps uplink. 2000 4.7GB single layer DVDs in three working days is about 870Mbps. Upload them as you rip to S3 and then send a pull request! https://github.com/awslabs/open-data-registry/ https://github.com/awslabs/open-data-registry/
  - rconti 8y ago
    So the 3 days is for figuring out how to use git, then?
    - halis 8y ago
      lmao
    - briffle 8y ago
      obligatory xkcd: https://xkcd.com/1597/ https://xkcd.com/1597/ If anything is out of your few commands, just save work elsewhere, delete the project, and download a fresh copy.
      - tracker1 8y ago
        I've, frankly, done that many times at this point... when your merge request gets too wonky, it's just easier. I do try to remember to squash, then rebase generally, though there are times I just forget or mess up.
        rconti 8y ago
        When in doubt, "how does the team here handle these conflicts? At my old job we just did <made up thing> to solve it, I've forgotten the process here."
  - reaperducer 8y ago
    Easy: get a bunch of DVD drives and a ~1 Gbps uplink. Can you detail how one easily gets a 1 gigabit uplink installed temporarily in the National Archives? I've had telcos drop fiber into the middle of fields in the middle of nowhere, so it's probably possible. But it feels like there's a lot of complicated bureaucracy to navigate, and the researcher OP is time-restricted.
patrickg_zill 8y ago
700 per day. About 30 per hour. Each DVD is what, 4gbytes? Maybe 10 laptops with SSD drives or external hard drives would be enough. And lots of caffeine!
- marak830 8y ago
  Surely that could be pumped up, write a script to auto-grab the video files, rename them to $dvd_name(n) and place in folder. Deal with combining those files later(when you have more time). Note:it's been a looking time since I have tried to pull DVD files, but if my (rusty) memory serves certain types of DVD readers will just let you copy the vob.
  - toomuchtodo 8y ago
    You’d want a block level image of the DVD for archival purposes, such as an ISO or BIN format. Using dd (Linux) or Disk Utility (Mac) makes this straightforward.
    - cookiecaper 8y ago
      Though it's probably not the case here, there are some discs with copy protection that specifically trips up block-level copies. The drive will continually error and misread sectors. I'm not sure what open-source tooling can be used to circumvent; I always end up falling back to AnyDVD.
      - jacobush 8y ago
        https://www.makemkv.com/forum/viewtopic.php?t=17138 https://www.makemkv.com/forum/viewtopic.php?t=17138 scripted
  - seltzered_ 8y ago
    https://github.com/automatic-ripping-machine/automatic-ripping-machine https://github.com/automatic-ripping-machine/automatic-rippi... can be modified fairly easily to just copy the disc image and not attempt any encoding.
NoblePublius 8y ago
I literally did this at a startup. We had eight eight-at-time Netflix accounts (DVD accounts). This was in 2009. We made a search engine for movies. Yeah, like Mr Skin. We ripped them six at a time. Took us about six months to do 4000 films. The DMCA violations would have been in the trillions.
- quickthrower2 8y ago
  “Hypothetically” you mean
  - mgamache 8y ago
    Normally the Statute of limitations would apply, but it seems like US Law is becoming even more inscrutable in the last few years. https://en.wikipedia.org/wiki/Statute_of_limitations https://en.wikipedia.org/wiki/Statute_of_limitations
    - torstenvl 8y ago
      There's no such thing as "the" statute of limitations. There may or may not be a statute of limitations governing each possible legal action. I count at least three: criminal copyright violation (U.S. v. NoblePublius), civil claim for copyright infringement (MPAA v. NoblePublius), and breach of contract (Netflix v. NoblePublius). Possibly others, e.g., if the DCMA has a separate criminal penalty for circumventing the copy protection.
  - paulie_a 8y ago
    No need to use that word. The DMCA is basically toothless at this point and can outright be ignored. You get a notice that can be safely thrown away. It's basically "don't do that again". Then you do it again. Honestly it's always been pretty easy to ignore it since it's inception unless you are running a counterfeiting operation.
    - tracker1 8y ago
      Some ISPs will cut you off after X repeat notices in under Y timeframe.
toomuchtodo 8y ago
You need DVD readers, storage, and the ability to rip to disk images (try to not reencode or modify the bits on disk for archival purposes). Maybe put out a call to Jason Scott/ArchiveTeam/ArchiveCorps to saddle up. ArchiveCorps (run by Jason) has a Slack team for such Call To Arms. OP: Have you attempted to renew your research card with the National Archive to get more time?
- rhema 8y ago
  Jason Scott has a nice podcast where he talks about a range of topics of interest to the HN community.
  - deleted 8y ago
    [deleted]
  - pronoiac 8y ago
    Jason Scott's podcast: http://ascii.textfiles.com/podcast http://ascii.textfiles.com/podcast
mgamache 8y ago
Make sure and test the copies. I used to do bulk copies years ago and the media + writing speed would make a difference. Writing at full speed would technically work, but the disks were sometimes unreadable in some readers. 'Back in my day' I would chain FireWire drives together (better than USB/Hubs at the time)
- userbinator 8y ago
  With that time constraint I wouldn't bother re-burning to discs, just rip them to a HDD (or several) to save the data for later.
newname2018 8y ago
Dd if=/(name of drive) Bs=1 Of=/( name of drive to copy to include the path Conv=notrunc Create a directory and a file on the target machine first a name like /myfile/this file Then to create a bit by bit copy of that file use this command Something like this Dd if=/dev/cdrom bs=1 of=/dev/sdc/myfile/thisfile.iso conv=notrunc
- pronoiac 8y ago
  I’d use gddrescue - it logs what it’s successfully read, so you can try multiple drives. And there should be some way of reading the dvd / volume title from a script.
- Dylan16807 8y ago
  Why set the block size to 1? Isn't that just going to cause a ridiculous amount of syscalls?
  - ars 8y ago
    Yah, and be incredibly slow as well. It should be at a minimum 32K which is the DVD block size, but even larger would be better (but may interleave badly with IO to other devices).
    - anticensor 8y ago
      It should be 2048 which is DVD sector size.
      - ars 8y ago
        Because of ECC it doesn't read a single sector at a time. And actually even 32K is not large enough for good performance, you should go higher, maybe 1MB.
- TomMarius 8y ago
  That does not deal with DRM
nuguy 8y ago
I wanted to do this but with Blu-ray’s. My goal was to build a library of every movie ever made that was worth seeing. It seemed like 2018 was a good time to do this since good movies have apparently stopped being made. Anyway, If you wanted to watch tons of good movies, you would normally end up paying tons of money to rent it from iTunes. And even then you only get to see it once and you have to have an internet connection. And streaming services don’t have even a fraction of the selection needed. But Netflix’s mail dvd service seems to have every movie I can think of. So why not open a few Netflix accounts, order disks in the mail and just save all the disk images? It seemed like a good idea until I looked into Blu-ray copy protection. Of course, I wanted to have my library consist of only the highest quality and highest fidelity so Blu-ray’s were called for. But Blu-ray copy protection is devious, ingenious and very effective. Each disk consists of two regions: A region that holds encrypted movie data and a region that holds a key. It is illegal for players to be sold that read the key and then forward it to user-facing interface like a computer. Players must always only read the key only in order to use it internally to decrypt the movie data. This stops all legitimate entities from selling players that reveal the key to the user. But what about the illegitimate entities that might want to sell modified players that provide the key? Or just publish keys online? Well, the key on the disk is itself actually encrypted. And it is encrypted in such a way that multiple keys can decrypt it. Blu-ray players come with special hardware that is flashed with a key at the factory. This hardware uses that key to decrypt the Blu-ray’s key. In the event that a key is compromised and published online, or used widely in any way, that key is depreciated and all Blu-ray’s from that point onward contain keys that cannot be decrypted with the compromised hardware key. Instead, a newer key is used. This new key is still able to decrypt all the old Blu-ray keys as well as all the new ones. This effectively defeats people publishing keys online. It’s ingenious in that the people who conceived it realized that the only time key compromise is a problem is when those keys are disseminated widely, and that when keys are disseminated widely they are easy for authorities to detect. If you want to get perfect rips of any blu-Ray you might come across, you are forced to go through the pain of probing the hardware yourself to get that key, which is quite difficult. There’s no way around it.
- fao_ 8y ago
  Why not just read the unencrypted data that the Blu-ray players give you, though.
nickysielicki 8y ago
I've done this with an old PC, 6 DVD drive bays and a bootable USB stick, ripping to network storage, did like 200 movies in a weekend. That PC sits in a basement now, I think. If anyone lives in Milwaukee and wants to rip some DVDs over the holidays, get in touch, more than happy to loan the machine out.
simplecomplex 8y ago
Contact archive.org, this is what they do.
turtlebits 8y ago
You need one find one of those Firewire Sony 200 DVD changers. Used to have on hooked up to my HTPC. Just a standard DVD writer in there. Maybe you could reach out and borrow one? From a quick google, it's the VGP-XL1B/VGP-XL1B2/VGP-XL1B3
baroffoos 8y ago
Do DVDs offer better error correction than CDs? I know from ripping CDs it can take over an hour per CD to get an accurate rip because the program has to read the disk multiple times to get the correct data out.
- jonhohle 8y ago
  Do you have context on this? I’ve been ripping/verifying old CD based games using raw cdrdao and can rip a CD with error correction in a few minutes. I often sell games and use the same method to verify the integrity of the game before sending it off.
  - oceanghost 8y ago
    Your rips are full of errors unless the drive you're using has C2 (many do), even then-- you've got a few errors. The only way to get an accurate rip of audio data is to compare it to a known good source. This is why Accurate Rip exists... Source: I've ripped 4000cds :) EDIT: This applies only to audio tracks. Data tracks don't suffer from this issue.
    - Dylan16807 8y ago
      It's worth noting that those problems are entirely caused by drives acting like black boxes, not the CD format itself.
      - jacobush 8y ago
        Sort of though. CD Audio is meant to be played back with a minimal buffer before the DAC and to be kept spinning at a very precise speed. It's more like a digital vinyl record than it is a file system, for instance.
        Dylan16807 8y ago
        There's enough error detection that if you can actually get access to what bits come off the laser you won't have any major problems making a perfect rip. Your whole-disk alignment might be off by a couple sectors, but that's about it.
gpav 8y ago
Judging from the date on the reddit post, it's a bit late for my comment (two days after the post), but the real issue is not how to duplicate so many DVDs, the issue is how to RIP those DVDs onto some cheap storage for later burning onto DVDs. So high data bandwidth between stacks of DVD players with lots of memory for caching (or however you cache DVD data streams). Fry's had a 4TB external hard drive for $89 this weekend. Maybe have a bunch of SSDs as the intake point for the DVDs, and then offload the ripped copies from the SSDs onto physical drives while you're changing out the DVDs. Would want to use the fastest interfaces available. I've no idea these days what the cool kids are using. (My first hard drive was a 5-1/4" full-height 10 MB MFM. Thought I would never need more storage than that. I think my second HD was 20 MB and used RLL.) Organizing the DVDs by length would allow optimizing the loading/ripping process to assure minimum time lost waiting for the operator's hands to be free. This kind of planning makes for an interesting project. It might make an interesting crowd-funded project, if it's reasonably easy to get a research permit. Plan it out, go in with the hardware, come out with the images. Use all the error correction opportunities you've got available. Do a web search for "bulk dvd ripping" (without quotes) and you'll find lots and lots of discussion and advice, including some about building a dedicated DVD ripping rig. MakeMKV gets good press, in my very quick read of a few posts. And there's always the option of crowd-funding to raise the exact amount needed to pay off the break-even for Amazon's investment. I can't imagine they'd fight back too hard when looking at a large check vs. a non-performing asset, unless Bezos personally never intended to let the footage go free.
- sokoloff 8y ago
  A 16x DVD drive is around 21MB/s output. Any modern hard drive is likely to have 5-12x the sequential write performance as a 16x DVD drive. Shouldn't be any need for SSDs.
  - dspillett 8y ago
    > Any modern hard drive is likely to have 5-12x the sequential write performance The data rate falls to the floor as soon as the access pattern isn't sequential though, which if you are using multiple readers it won't be. While an OS might be bright enough to organise data flowing out of write buffers so it isn't as random as it could be there is a limit to how far they will go with this because they are general purpose OSs and optimising for multiple bulk streams will punish more interactive activity. If you have a tool they bypasses the OS cache and works in large enough blocks you might see better results except if the write activity from each lines up at which point this will make things worse. Pulling the data off multiple DVD drives onto an SSD, swapping to output to another once near full to continue while its contents are dumped sequentially to cheaper-per-Gb traditional drives, would probably be the way I'd suggest. In fact, you would get away without swapping between two SSDs: the read activity pulling data from the SSD to a traditional drive is unlikely to have much effect on the write performance for the data coming off the DVDs unless you have a great many readers in one machine. If doing this all relatively manually, to reduce manual steps once a DVD copy is complete add it to a queue to be moved using something like https://en.wikipedia.org/wiki/TeraCopy https://en.wikipedia.org/wiki/TeraCopy so you don't have to worry about manually coordinating the SSD-to-cheaper copy operation to keep it sequential. Assuming 15 minutes to read each disk (it is a long time since I pulled data off a DVD in bulk so this is guess work based on old memory of it taking a little more than 10 minutes to read a full DVD9 disk, and rounding up to 15 to allow for manual process inefficiencies and some disks being slower to extract due to condition causing rereads, etc) you are looking at wanting 21 or more drives constantly on the go to get the job done in 3 solid 8-hour days (2,000x15/3/8/60 = 20.8). Five laptops each with an internal SSD (128G+) to extract to, five DVD readers on USB3 to extract from, and a 4+Tb spinning disk (also external) to finally write to, might do the job and have the space (2,000x8.5/5=~3.5Tb output per laptop). You'll need a powered USB dock/ for each laptop instead of a passive hub, and you are going to want to add more of everything to allow for the possibility of device failures. Of course significantly less resource is needed (or you get more contingency time (and/or spare kit to deal with failures) from the same resource) if most of the media is DVD5 and/or not full disks. I've assumed the initial three days is just for obtaining the content - I've not accounted for any other processing (such as indexing and transcoding) or further distribution.
bane 8y ago
Give a call to Jason Scott at the internet archive? Get lots of friends to help?
onion2k 8y ago
Complete speculation here and I have no idea if it's actually possible, but couldn't you take a high resolution photograph of the DVD and then use software to turn the image data back in to DVD data? Maybe with a couple of different colored light sources at obtuse angles to the disc? It's obviously not a practical solution but it'd be really fast if you could make it work. It's definitely possible with phonograph records.
- Dylan16807 8y ago
  Consider that you'd be trying to photograph 100 billion bits of pit and land. To capture it properly a terapixel image might work. Nothing can do that in one photo, or short series of photos.
- beojan 8y ago
  No. DVDs are much higher density than phonograph records.
  - U2EF1 8y ago
    [citation needed]
    - dikei 8y ago
      Just a rough calculation: Each pit on the DVD represents 1 bit, but need at least 1 pixel to stored. You need 8 bit to store a pixel. So you have write amplification of at least 8 when taking picture of DVD.
      - onion2k 8y ago
        That's why I suggested using a couple of light sources - you could take an image that's lower resolution than 1 pixel per bit and still resolve the image data to the DVD data based on the color. Eg 10 would be a different color to 01. It's not even remotely practical but I thought it was an interesting idea. Judging by the way the votes are yoyo'ing up and down, about half of HN readers agree. :)
lph 8y ago
My hands are bleeding just thinking about the logistics of opening and closing 2000+ DVD cases, and loading/unloading that many DVDs, over the span of three days. Ouch.
rhplus 8y ago
The real question here is how the heck does Amazon have a deal with the National Archives that prevents that public body from distributing public domain content that it now holds in its archive? Is this typical for digitally archived content?
- rrix2 8y ago
  Ancestry.com has a similar agreement with the National Archive: https://www.archives.gov/files/digitization/pdf/ancestry-2015-agreement.pdf https://www.archives.gov/files/digitization/pdf/ancestry-201... Five year embargo on the National Archive releasing whatever Ancestry chooses to digitise and host on their proprietary database. This model of digitzation appears to be the principle way that the records are digitized now: https://www.archives.gov/digitization/principles.html https://www.archives.gov/digitization/principles.html
- simonh 8y ago
  The National Archive doesn't have the resources to digitize the material, but Amazon does. If the National Archive isn't interested in having the material digitized for them, nobody is forcing them to do a deal.
- sudhirj 8y ago
  Think the Archives have the film reels, strips, which you're free to go look at and copy whenever you want. Amazon paid to digitize them and put them on the DVDs, so Amazon gets to make the rules for the DVDs. I assume the film is still around in boxes, so event he DVD window drops someone is still free to go digitize them again, but given how fragile film is that's touch and go.
- Spooky23 8y ago
  It is done when the digitization is done in exchange. Archives preserve and disseminate. Budgets usually make this a challenge. They probably (rightly) view the preservation of content on decaying media as a big win.
- Sir_Cmpwn 8y ago
  I think the main takeaway here is that we (citizens) should be volunteering to digitize material for the National Archives.
taurath 8y ago
My tip would be - find a nearby game company, especially a publisher. They tend to have large amounts of extra burning towers from the pre-USB-drive days - 2000 1-1 is a bit more difficult than 2000 copies of one though. I'd bet there's a copy function on some of them.
PinkMilkshake 8y ago
Look for a local gaming cafe and rent all of their machines.
z3t4 8y ago
40 TB of HDD and 10 (high speed) DVD-readers with two guys should get the job done in one day. I wonder though if Amazon did use DRM ? That would be an issue.
jcelerier 8y ago
Child labor
crispyambulance 8y ago
3 days isn't enough time to do a quality job on all of them given "one shot". Better to just prioritize some sample of discs and do the best you can-- consider it a pilot run. If you monitor everything you'll be able to use the pilot run to estimate and propose what it would take in terms of equipment, time and process to do the whole thing in a reasonable time.
unqueued 8y ago
I had a similar challenge many years ago. I worked at a computer lab and had access to about 25 Dell Optiplex 380s, and I had about 200 DVDs I wanted to rip. They were for a friend, who was compiling city council meeting videos for a county. So, I slightly modified Knoppix and started a netboot server, and made a dvd ripping cluster. I didn't end up doing exactly this, but it's worth trying DVD::Rip. It specifically has support for ripping clusters[1]. But it doesn't even have to be something as complicated as a cluster, just as long as you parallelize things. How many laptops are you allowed to bring with you? Could you bring five laptops and 10 dvd drives? [1]: https://ubuntuforums.org/showthread.php?t=1217643 https://ubuntuforums.org/showthread.php?t=1217643
- reaperducer 8y ago
  I had a similar challenge when the first iPod came out. I got one on launch day in late October 2001 for my wife as a Christmas present Two HP towers, each with two optical drives, I spent nights and weekends ripping her CD collection. I just barely got it all done before Christmas. She still has no idea how much work that was.
slfnflctd 8y ago
At 12 hours of labor per day, that's roughly one per minute. So, at minimum you need one machine more than the number of minutes it takes each machine to rip the DVD. If you also have to burn a second set of DVDs, you will need at least one machine more than the number of minutes it takes to burn a DVD plus the aforementioned ripping machines. If you're just storing the DVD images after ripping, that part can be at least partly automated, set up a machine as a central server (about 20 terabytes should do it if nothing goes wrong) and have each 'ripper' run a script to copy the image up after the rip is done (which hopefully it can do while also starting the next rip, if not you'll even need more machines).
JustSomeNobody 8y ago
Get several cheap laptops with dvd players. install ubuntu. Buy a bunch of USB HDs. Use DVD::Rip to extract the VOBs onto the USB drives. At some point later, convert the VOB data to ISOs. Burn to DVD as time permits.
social_quotient 8y ago
I’d call a staffing company or a very active meetup group and ask for help. Everyone could meet in a conference room and write the contents locally to their machines. I’d create Pod groups of 5-6 people to assist each other with the physical parts of the task, organizing and labeling what’s completed etc. Thinking 15 minutes per you have 30k minutes. 1 person can get 4 done per hour - let’s think 2 12 hour shift you just need about 25 people + their laptops. If you need the equipment it could be rented by someone like this that does it for trade shows https://meetingtomorrow.com/austin-computer-rentals https://meetingtomorrow.com/austin-computer-rentals
- mikeryan 8y ago
  There are more and more “active” senior living communities that may likely have a trove of willing and interested volunteers.
umvi 8y ago
There is a company in Utah called "VidAngel" which I believe had built up a fairly sophisticated DVD ripping/streaming operation until they got sued by Disney. They are still around but have since switched to a streaming only model. You could reach out and see if they still have their DVD ripping infrastructure, I'm sure they would love to help.
baldfat 8y ago
One PC with 8 DVD R/W Drives. I use to own one in the 90s. I would do work for churches and conferences. I could get a thousand done in a few days. They use to have automatic feeder drives for producing more. No one sells them anymore it appears.
deleted 8y ago
[deleted]
PakG1 8y ago
I manage the IT for a school. As we've gone more and more away from DVDs (thanks, Apple), teachers have been asking me to rip their old DVDs. Tried doing that, but a lot of the DVDs did not rip perfectly, so that the ripped video ended up having a lot of corruption. I'm not entirely clear why software could not play the ripped DVDs and just deal with the jitter and corrupted data, but playing from a DVD drive could deal with it. It was weird. Googled a bit, couldn't find much information other than the fact that CDs and DVDs can deteriorate over time. My question is not necessarily related to OP. But can anyone tell me why my ripped DVDs could not play in DVD software properly when there was jitter or corrupted data, but it could play properly when in the DVD drive? When I got lucky, the ripped DVDs would play fine. Otherwise, it was play fine for a few minutes until it ran into the first bit of corruption. Confused the heck out of me.
- verisimilitude 8y ago
  As I understand it, software playing a ripped DVD doesn't "know" when a data source is a ripped DVD and does not expect any corruption -- it will not seek over or ignore issues like a DVD player connected to a TV will. I've run into this issue a lot, but fortunately MakeMKV, my software of choice, will fail to rip DVDs when it is unable to make a perfect rip. Then, I know there is a problem before I begin ffmpeg compression. The solution: a wet polisher for DVDs. Luckily, my local used-game shop polishes disks for $3 each. They use an [Elm Eco AutoSmart][1] and it makes the discs like-new. I have never seen a disc it could not fix. You'd have to ask/search around locally to find one near you... I think some libraries have these machines, too, to maintain their DVD collections. [1]: https://www.elm-usa.com/products/eco-autosmart https://www.elm-usa.com/products/eco-autosmart
- deleted 8y ago
  [deleted]
- StillBored 8y ago
  That's generally caused by a combination of a cheap/bad high speed DVD-ROM drive and/or bad ripping software/options settings. What seems to happen is the ripping software is running a marginal drive/dvd at full speed and disabling the drive retry on error. Combined with all the soft ways these drives fail it results in a bad rip. This used to happen to me a few years ago, and it was overwhelmingly just a sign of a marginal drive. For about 10 years, I was replacing drives about once a year. It doesn't happen as frequently since I started using bluray drives (usually LG) to rip DVDs and a piece of software designed for "piracy". Anyway, two software things to try, are lower the ripping speed to 1x-2x, and make sure to leave the drive retry on bad sectors enabled. Most decent ripping software will have options to control the error correction/retry logic. The problem is some copy protection schemes leave bad sectors on the disks, and this will hang a lot of the "honest" ripping software that doesn't know how to deal with it.
netrap 8y ago
There are many old SCSI and Firewire DVD changers available. Just gotta ask the right people.
porlune 8y ago
Does anyone know why they can't just network the file system? If he has network access, then it might be faster to just upload the entire set directly to another drive over the internet, since most universities have fairly good connections, that shouldn't be any slower then trying to write them all to DVDs.

393 points by huhtenberg 8y ago

The context - > Maybe you should disclose your city or at least state/general location Washington DC, specifically College Park, even more specifically The National Archives at College Park. > What is the content of the discs? Years ago in partnership with the National Archives Amazon.com digitized a tremendous amount of film for the National Archives, with the catch that the material cannot be freely disseminated online by the National Archives until Amazon breaks even on their digitization investment. It’s been years now and for a variety of reasons - many of which are Amazon’s fault - there still exist a solid number of discs that Amazon hasn’t sold even one of. (In part because amazon hasn’t even had them all available to sell. Pretty ridiculous). The flipside of the catch is that the DVDs can be viewed and even copied for $0 on site by researchers. I am doing research there and I have have a research pass. I was looking at many of these titles yesterday. It’s time to set these national treasures free.

sergiotapia 8y ago
Didn't Aaron Swartz get prosecuted and ultimately died because of doing what OP is attempting to do?
- oh_sigh 8y ago
  No. Aaron died because he put a rope around his own neck and tightened it to the point that he could not breathe. He put the rope around his neck because he was depressed and suicidal, both characteristics which were present in him before he came to the attention of any prosecutor.
  - deleted 8y ago
    [deleted]
  - packetized 8y ago
    Factual information isn’t always useful in another context.
- gammateam 8y ago
  No. And even if it was, millions of people can do the time, including programmers.
  - somabc 8y ago
    It's actually very similar to Aaron Schwartz's plan of using academic access to copy and widely distribute papers. This researcher does not have a legal basis to copy 2000+ disks in order to free them from the national archives. At most he has a restricted right of access to use some of them in his academic research. To test he could ask the national archive for permission to carry out his plan and observe how quickly his access is revoked. ----- Research Room Rules. The rules are intended to provide clear instructions to both staff and researchers on the research room: * registration process; * entry and exit procedures; * items authorized and not authorized for use in the * research rooms; * pull requests; * records handling guidelines; and * copying and equipment approvals. The rules will be available for researcher use in each facility or can be requested in writing from the facility managers listed below. Archives II, College Park, MD – Michael Knight, michael.knight@nara.gov
veridies 8y ago
I wonder what this material is. Are these commercial films, historical footage, or what?
- eli 8y ago
  The OP says they're public domain, so I'd guess it's films produced by the government.
  - vsl 8y ago
    ...or past the copyright window.
- Semaphor 8y ago
  > Years ago in partnership with the National Archives Amazon.com digitized a tremendous amount of film for the National Archives, with the catch that the material cannot be freely disseminated online by the National Archives until Amazon breaks even on their digitization investment. It’s been years now and for a variety of reasons - many of which are Amazon’s fault - there still exist a solid number of discs that Amazon hasn’t sold even one of. (In part because amazon hasn’t even had them all available to sell. Pretty ridiculous). > The flipside of the catch is that the DVDs can be viewed and even copied for $0 on site by researchers. I am doing research there and I have have a research pass. I was looking at many of these titles yesterday. It’s time to set these national treasures free. -- https://old.reddit.com/r/DataHoarder/comments/a6fkpm/i_have_temporary_3_days_access_to_more_than_2000/ebuf75j/?context=1 https://old.reddit.com/r/DataHoarder/comments/a6fkpm/i_have_...
  - traek 8y ago
    That doesn’t answer what type of media is on the disks.
    - lurker_primo 8y ago
      It says film. So, it could be movies or images. What else would be possible? Are you thinking of quickest ways to rip the DVDs if they are movies?
shawn-butler 8y ago
Seems counter-intuitive to Amazon's own public dataset program / policy: https://aws.amazon.com/opendata/public-datasets/ https://aws.amazon.com/opendata/public-datasets/
- kkarakk 8y ago
  >Democratize access to data by making it available for analysis on AWS. >AWS evaluates applications to the AWS Public Dataset Program every three months. >If we bring your dataset into the AWS Public Dataset Program, we will cover the costs of storage and data transfer for a period of two years yeah i don't think that's public,more like a rental service. i wouldn't bother working with that unless i had free AWS access through corporate
rasz 8y ago
>In part because amazon hasn’t even had them all available to sell. Pretty ridiculous not ridiculous, good lawyer preparing the deal
- morrbo 8y ago
  Not ridiculous, but very sad
- SmellyGeekBoy 8y ago
  What's "good" about deliberately keeping public domain content out of the public domain?
  - allwein 8y ago
    Not "good" as in "good vs. evil", but rather "good" as in "proficient at the skill of lawyering".
brett40324 8y ago
Im wondering how the already preserved and publicly available film content is being considered being made freed ? Theres very clear information on rights, reproduction restrictions, and the preservation efforts that have already been made here: https://www.archives.gov/research/order/film-sound-video-dc.html https://www.archives.gov/research/order/film-sound-video-dc....
amluto 8y ago
Easy: get a bunch of DVD drives and a ~1 Gbps uplink. 2000 4.7GB single layer DVDs in three working days is about 870Mbps. Upload them as you rip to S3 and then send a pull request! https://github.com/awslabs/open-data-registry/ https://github.com/awslabs/open-data-registry/
- rconti 8y ago
  So the 3 days is for figuring out how to use git, then?
  - halis 8y ago
    lmao
  - briffle 8y ago
    obligatory xkcd: https://xkcd.com/1597/ https://xkcd.com/1597/ If anything is out of your few commands, just save work elsewhere, delete the project, and download a fresh copy.
    - tracker1 8y ago
      I've, frankly, done that many times at this point... when your merge request gets too wonky, it's just easier. I do try to remember to squash, then rebase generally, though there are times I just forget or mess up.
      - rconti 8y ago
        When in doubt, "how does the team here handle these conflicts? At my old job we just did <made up thing> to solve it, I've forgotten the process here."
- reaperducer 8y ago
  Easy: get a bunch of DVD drives and a ~1 Gbps uplink. Can you detail how one easily gets a 1 gigabit uplink installed temporarily in the National Archives? I've had telcos drop fiber into the middle of fields in the middle of nowhere, so it's probably possible. But it feels like there's a lot of complicated bureaucracy to navigate, and the researcher OP is time-restricted.

patrickg_zill 8y ago

700 per day. About 30 per hour. Each DVD is what, 4gbytes? Maybe 10 laptops with SSD drives or external hard drives would be enough. And lots of caffeine!

marak830 8y ago
Surely that could be pumped up, write a script to auto-grab the video files, rename them to $dvd_name(n) and place in folder. Deal with combining those files later(when you have more time). Note:it's been a looking time since I have tried to pull DVD files, but if my (rusty) memory serves certain types of DVD readers will just let you copy the vob.
- toomuchtodo 8y ago
  You’d want a block level image of the DVD for archival purposes, such as an ISO or BIN format. Using dd (Linux) or Disk Utility (Mac) makes this straightforward.
  - cookiecaper 8y ago
    Though it's probably not the case here, there are some discs with copy protection that specifically trips up block-level copies. The drive will continually error and misread sectors. I'm not sure what open-source tooling can be used to circumvent; I always end up falling back to AnyDVD.
    - jacobush 8y ago
      https://www.makemkv.com/forum/viewtopic.php?t=17138 https://www.makemkv.com/forum/viewtopic.php?t=17138 scripted
- seltzered_ 8y ago
  https://github.com/automatic-ripping-machine/automatic-ripping-machine https://github.com/automatic-ripping-machine/automatic-rippi... can be modified fairly easily to just copy the disc image and not attempt any encoding.

NoblePublius 8y ago

I literally did this at a startup. We had eight eight-at-time Netflix accounts (DVD accounts). This was in 2009. We made a search engine for movies. Yeah, like Mr Skin. We ripped them six at a time. Took us about six months to do 4000 films. The DMCA violations would have been in the trillions.

quickthrower2 8y ago
“Hypothetically” you mean
- mgamache 8y ago
  Normally the Statute of limitations would apply, but it seems like US Law is becoming even more inscrutable in the last few years. https://en.wikipedia.org/wiki/Statute_of_limitations https://en.wikipedia.org/wiki/Statute_of_limitations
  - torstenvl 8y ago
    There's no such thing as "the" statute of limitations. There may or may not be a statute of limitations governing each possible legal action. I count at least three: criminal copyright violation (U.S. v. NoblePublius), civil claim for copyright infringement (MPAA v. NoblePublius), and breach of contract (Netflix v. NoblePublius). Possibly others, e.g., if the DCMA has a separate criminal penalty for circumventing the copy protection.
- paulie_a 8y ago
  No need to use that word. The DMCA is basically toothless at this point and can outright be ignored. You get a notice that can be safely thrown away. It's basically "don't do that again". Then you do it again. Honestly it's always been pretty easy to ignore it since it's inception unless you are running a counterfeiting operation.
  - tracker1 8y ago
    Some ISPs will cut you off after X repeat notices in under Y timeframe.

toomuchtodo 8y ago

You need DVD readers, storage, and the ability to rip to disk images (try to not reencode or modify the bits on disk for archival purposes). Maybe put out a call to Jason Scott/ArchiveTeam/ArchiveCorps to saddle up. ArchiveCorps (run by Jason) has a Slack team for such Call To Arms. OP: Have you attempted to renew your research card with the National Archive to get more time?

rhema 8y ago
Jason Scott has a nice podcast where he talks about a range of topics of interest to the HN community.
- deleted 8y ago
  [deleted]
- pronoiac 8y ago
  Jason Scott's podcast: http://ascii.textfiles.com/podcast http://ascii.textfiles.com/podcast

mgamache 8y ago

Make sure and test the copies. I used to do bulk copies years ago and the media + writing speed would make a difference. Writing at full speed would technically work, but the disks were sometimes unreadable in some readers. 'Back in my day' I would chain FireWire drives together (better than USB/Hubs at the time)

userbinator 8y ago
With that time constraint I wouldn't bother re-burning to discs, just rip them to a HDD (or several) to save the data for later.

newname2018 8y ago

Dd if=/(name of drive) Bs=1 Of=/( name of drive to copy to include the path Conv=notrunc Create a directory and a file on the target machine first a name like /myfile/this file Then to create a bit by bit copy of that file use this command Something like this Dd if=/dev/cdrom bs=1 of=/dev/sdc/myfile/thisfile.iso conv=notrunc

pronoiac 8y ago
I’d use gddrescue - it logs what it’s successfully read, so you can try multiple drives. And there should be some way of reading the dvd / volume title from a script.
Dylan16807 8y ago
Why set the block size to 1? Isn't that just going to cause a ridiculous amount of syscalls?
- ars 8y ago
  Yah, and be incredibly slow as well. It should be at a minimum 32K which is the DVD block size, but even larger would be better (but may interleave badly with IO to other devices).
  - anticensor 8y ago
    It should be 2048 which is DVD sector size.
    - ars 8y ago
      Because of ECC it doesn't read a single sector at a time. And actually even 32K is not large enough for good performance, you should go higher, maybe 1MB.
TomMarius 8y ago
That does not deal with DRM

nuguy 8y ago

I wanted to do this but with Blu-ray’s. My goal was to build a library of every movie ever made that was worth seeing. It seemed like 2018 was a good time to do this since good movies have apparently stopped being made. Anyway, If you wanted to watch tons of good movies, you would normally end up paying tons of money to rent it from iTunes. And even then you only get to see it once and you have to have an internet connection. And streaming services don’t have even a fraction of the selection needed. But Netflix’s mail dvd service seems to have every movie I can think of. So why not open a few Netflix accounts, order disks in the mail and just save all the disk images? It seemed like a good idea until I looked into Blu-ray copy protection. Of course, I wanted to have my library consist of only the highest quality and highest fidelity so Blu-ray’s were called for. But Blu-ray copy protection is devious, ingenious and very effective. Each disk consists of two regions: A region that holds encrypted movie data and a region that holds a key. It is illegal for players to be sold that read the key and then forward it to user-facing interface like a computer. Players must always only read the key only in order to use it internally to decrypt the movie data. This stops all legitimate entities from selling players that reveal the key to the user. But what about the illegitimate entities that might want to sell modified players that provide the key? Or just publish keys online? Well, the key on the disk is itself actually encrypted. And it is encrypted in such a way that multiple keys can decrypt it. Blu-ray players come with special hardware that is flashed with a key at the factory. This hardware uses that key to decrypt the Blu-ray’s key. In the event that a key is compromised and published online, or used widely in any way, that key is depreciated and all Blu-ray’s from that point onward contain keys that cannot be decrypted with the compromised hardware key. Instead, a newer key is used. This new key is still able to decrypt all the old Blu-ray keys as well as all the new ones. This effectively defeats people publishing keys online. It’s ingenious in that the people who conceived it realized that the only time key compromise is a problem is when those keys are disseminated widely, and that when keys are disseminated widely they are easy for authorities to detect. If you want to get perfect rips of any blu-Ray you might come across, you are forced to go through the pain of probing the hardware yourself to get that key, which is quite difficult. There’s no way around it.

fao_ 8y ago
Why not just read the unencrypted data that the Blu-ray players give you, though.

nickysielicki 8y ago

I've done this with an old PC, 6 DVD drive bays and a bootable USB stick, ripping to network storage, did like 200 movies in a weekend. That PC sits in a basement now, I think. If anyone lives in Milwaukee and wants to rip some DVDs over the holidays, get in touch, more than happy to loan the machine out.

simplecomplex 8y ago

Contact archive.org, this is what they do.

turtlebits 8y ago

You need one find one of those Firewire Sony 200 DVD changers. Used to have on hooked up to my HTPC. Just a standard DVD writer in there. Maybe you could reach out and borrow one? From a quick google, it's the VGP-XL1B/VGP-XL1B2/VGP-XL1B3

baroffoos 8y ago

Do DVDs offer better error correction than CDs? I know from ripping CDs it can take over an hour per CD to get an accurate rip because the program has to read the disk multiple times to get the correct data out.

jonhohle 8y ago
Do you have context on this? I’ve been ripping/verifying old CD based games using raw cdrdao and can rip a CD with error correction in a few minutes. I often sell games and use the same method to verify the integrity of the game before sending it off.
- oceanghost 8y ago
  Your rips are full of errors unless the drive you're using has C2 (many do), even then-- you've got a few errors. The only way to get an accurate rip of audio data is to compare it to a known good source. This is why Accurate Rip exists... Source: I've ripped 4000cds :) EDIT: This applies only to audio tracks. Data tracks don't suffer from this issue.
  - Dylan16807 8y ago
    It's worth noting that those problems are entirely caused by drives acting like black boxes, not the CD format itself.
    - jacobush 8y ago
      Sort of though. CD Audio is meant to be played back with a minimal buffer before the DAC and to be kept spinning at a very precise speed. It's more like a digital vinyl record than it is a file system, for instance.
      - Dylan16807 8y ago
        There's enough error detection that if you can actually get access to what bits come off the laser you won't have any major problems making a perfect rip. Your whole-disk alignment might be off by a couple sectors, but that's about it.

gpav 8y ago

Judging from the date on the reddit post, it's a bit late for my comment (two days after the post), but the real issue is not how to duplicate so many DVDs, the issue is how to RIP those DVDs onto some cheap storage for later burning onto DVDs. So high data bandwidth between stacks of DVD players with lots of memory for caching (or however you cache DVD data streams). Fry's had a 4TB external hard drive for $89 this weekend. Maybe have a bunch of SSDs as the intake point for the DVDs, and then offload the ripped copies from the SSDs onto physical drives while you're changing out the DVDs. Would want to use the fastest interfaces available. I've no idea these days what the cool kids are using. (My first hard drive was a 5-1/4" full-height 10 MB MFM. Thought I would never need more storage than that. I think my second HD was 20 MB and used RLL.) Organizing the DVDs by length would allow optimizing the loading/ripping process to assure minimum time lost waiting for the operator's hands to be free. This kind of planning makes for an interesting project. It might make an interesting crowd-funded project, if it's reasonably easy to get a research permit. Plan it out, go in with the hardware, come out with the images. Use all the error correction opportunities you've got available. Do a web search for "bulk dvd ripping" (without quotes) and you'll find lots and lots of discussion and advice, including some about building a dedicated DVD ripping rig. MakeMKV gets good press, in my very quick read of a few posts. And there's always the option of crowd-funding to raise the exact amount needed to pay off the break-even for Amazon's investment. I can't imagine they'd fight back too hard when looking at a large check vs. a non-performing asset, unless Bezos personally never intended to let the footage go free.

sokoloff 8y ago
A 16x DVD drive is around 21MB/s output. Any modern hard drive is likely to have 5-12x the sequential write performance as a 16x DVD drive. Shouldn't be any need for SSDs.
- dspillett 8y ago
  > Any modern hard drive is likely to have 5-12x the sequential write performance The data rate falls to the floor as soon as the access pattern isn't sequential though, which if you are using multiple readers it won't be. While an OS might be bright enough to organise data flowing out of write buffers so it isn't as random as it could be there is a limit to how far they will go with this because they are general purpose OSs and optimising for multiple bulk streams will punish more interactive activity. If you have a tool they bypasses the OS cache and works in large enough blocks you might see better results except if the write activity from each lines up at which point this will make things worse. Pulling the data off multiple DVD drives onto an SSD, swapping to output to another once near full to continue while its contents are dumped sequentially to cheaper-per-Gb traditional drives, would probably be the way I'd suggest. In fact, you would get away without swapping between two SSDs: the read activity pulling data from the SSD to a traditional drive is unlikely to have much effect on the write performance for the data coming off the DVDs unless you have a great many readers in one machine. If doing this all relatively manually, to reduce manual steps once a DVD copy is complete add it to a queue to be moved using something like https://en.wikipedia.org/wiki/TeraCopy https://en.wikipedia.org/wiki/TeraCopy so you don't have to worry about manually coordinating the SSD-to-cheaper copy operation to keep it sequential. Assuming 15 minutes to read each disk (it is a long time since I pulled data off a DVD in bulk so this is guess work based on old memory of it taking a little more than 10 minutes to read a full DVD9 disk, and rounding up to 15 to allow for manual process inefficiencies and some disks being slower to extract due to condition causing rereads, etc) you are looking at wanting 21 or more drives constantly on the go to get the job done in 3 solid 8-hour days (2,000x15/3/8/60 = 20.8). Five laptops each with an internal SSD (128G+) to extract to, five DVD readers on USB3 to extract from, and a 4+Tb spinning disk (also external) to finally write to, might do the job and have the space (2,000x8.5/5=~3.5Tb output per laptop). You'll need a powered USB dock/ for each laptop instead of a passive hub, and you are going to want to add more of everything to allow for the possibility of device failures. Of course significantly less resource is needed (or you get more contingency time (and/or spare kit to deal with failures) from the same resource) if most of the media is DVD5 and/or not full disks. I've assumed the initial three days is just for obtaining the content - I've not accounted for any other processing (such as indexing and transcoding) or further distribution.

bane 8y ago

Give a call to Jason Scott at the internet archive? Get lots of friends to help?

onion2k 8y ago

Complete speculation here and I have no idea if it's actually possible, but couldn't you take a high resolution photograph of the DVD and then use software to turn the image data back in to DVD data? Maybe with a couple of different colored light sources at obtuse angles to the disc? It's obviously not a practical solution but it'd be really fast if you could make it work. It's definitely possible with phonograph records.

Dylan16807 8y ago
Consider that you'd be trying to photograph 100 billion bits of pit and land. To capture it properly a terapixel image might work. Nothing can do that in one photo, or short series of photos.
beojan 8y ago
No. DVDs are much higher density than phonograph records.
- U2EF1 8y ago
  [citation needed]
  - dikei 8y ago
    Just a rough calculation: Each pit on the DVD represents 1 bit, but need at least 1 pixel to stored. You need 8 bit to store a pixel. So you have write amplification of at least 8 when taking picture of DVD.
    - onion2k 8y ago
      That's why I suggested using a couple of light sources - you could take an image that's lower resolution than 1 pixel per bit and still resolve the image data to the DVD data based on the color. Eg 10 would be a different color to 01. It's not even remotely practical but I thought it was an interesting idea. Judging by the way the votes are yoyo'ing up and down, about half of HN readers agree. :)

lph 8y ago

My hands are bleeding just thinking about the logistics of opening and closing 2000+ DVD cases, and loading/unloading that many DVDs, over the span of three days. Ouch.

rhplus 8y ago

The real question here is how the heck does Amazon have a deal with the National Archives that prevents that public body from distributing public domain content that it now holds in its archive? Is this typical for digitally archived content?

rrix2 8y ago
Ancestry.com has a similar agreement with the National Archive: https://www.archives.gov/files/digitization/pdf/ancestry-2015-agreement.pdf https://www.archives.gov/files/digitization/pdf/ancestry-201... Five year embargo on the National Archive releasing whatever Ancestry chooses to digitise and host on their proprietary database. This model of digitzation appears to be the principle way that the records are digitized now: https://www.archives.gov/digitization/principles.html https://www.archives.gov/digitization/principles.html
simonh 8y ago
The National Archive doesn't have the resources to digitize the material, but Amazon does. If the National Archive isn't interested in having the material digitized for them, nobody is forcing them to do a deal.
sudhirj 8y ago
Think the Archives have the film reels, strips, which you're free to go look at and copy whenever you want. Amazon paid to digitize them and put them on the DVDs, so Amazon gets to make the rules for the DVDs. I assume the film is still around in boxes, so event he DVD window drops someone is still free to go digitize them again, but given how fragile film is that's touch and go.
Spooky23 8y ago
It is done when the digitization is done in exchange. Archives preserve and disseminate. Budgets usually make this a challenge. They probably (rightly) view the preservation of content on decaying media as a big win.
Sir_Cmpwn 8y ago
I think the main takeaway here is that we (citizens) should be volunteering to digitize material for the National Archives.

taurath 8y ago

My tip would be - find a nearby game company, especially a publisher. They tend to have large amounts of extra burning towers from the pre-USB-drive days - 2000 1-1 is a bit more difficult than 2000 copies of one though. I'd bet there's a copy function on some of them.

PinkMilkshake 8y ago

Look for a local gaming cafe and rent all of their machines.

z3t4 8y ago

40 TB of HDD and 10 (high speed) DVD-readers with two guys should get the job done in one day. I wonder though if Amazon did use DRM ? That would be an issue.

jcelerier 8y ago

Child labor

crispyambulance 8y ago

3 days isn't enough time to do a quality job on all of them given "one shot". Better to just prioritize some sample of discs and do the best you can-- consider it a pilot run. If you monitor everything you'll be able to use the pilot run to estimate and propose what it would take in terms of equipment, time and process to do the whole thing in a reasonable time.

unqueued 8y ago

I had a similar challenge many years ago. I worked at a computer lab and had access to about 25 Dell Optiplex 380s, and I had about 200 DVDs I wanted to rip. They were for a friend, who was compiling city council meeting videos for a county. So, I slightly modified Knoppix and started a netboot server, and made a dvd ripping cluster. I didn't end up doing exactly this, but it's worth trying DVD::Rip. It specifically has support for ripping clusters[1]. But it doesn't even have to be something as complicated as a cluster, just as long as you parallelize things. How many laptops are you allowed to bring with you? Could you bring five laptops and 10 dvd drives? [1]: https://ubuntuforums.org/showthread.php?t=1217643 https://ubuntuforums.org/showthread.php?t=1217643

reaperducer 8y ago
I had a similar challenge when the first iPod came out. I got one on launch day in late October 2001 for my wife as a Christmas present Two HP towers, each with two optical drives, I spent nights and weekends ripping her CD collection. I just barely got it all done before Christmas. She still has no idea how much work that was.

slfnflctd 8y ago

At 12 hours of labor per day, that's roughly one per minute. So, at minimum you need one machine more than the number of minutes it takes each machine to rip the DVD. If you also have to burn a second set of DVDs, you will need at least one machine more than the number of minutes it takes to burn a DVD plus the aforementioned ripping machines. If you're just storing the DVD images after ripping, that part can be at least partly automated, set up a machine as a central server (about 20 terabytes should do it if nothing goes wrong) and have each 'ripper' run a script to copy the image up after the rip is done (which hopefully it can do while also starting the next rip, if not you'll even need more machines).

JustSomeNobody 8y ago

Get several cheap laptops with dvd players. install ubuntu. Buy a bunch of USB HDs. Use DVD::Rip to extract the VOBs onto the USB drives. At some point later, convert the VOB data to ISOs. Burn to DVD as time permits.

social_quotient 8y ago

I’d call a staffing company or a very active meetup group and ask for help. Everyone could meet in a conference room and write the contents locally to their machines. I’d create Pod groups of 5-6 people to assist each other with the physical parts of the task, organizing and labeling what’s completed etc. Thinking 15 minutes per you have 30k minutes. 1 person can get 4 done per hour - let’s think 2 12 hour shift you just need about 25 people + their laptops. If you need the equipment it could be rented by someone like this that does it for trade shows https://meetingtomorrow.com/austin-computer-rentals https://meetingtomorrow.com/austin-computer-rentals

mikeryan 8y ago
There are more and more “active” senior living communities that may likely have a trove of willing and interested volunteers.

umvi 8y ago

There is a company in Utah called "VidAngel" which I believe had built up a fairly sophisticated DVD ripping/streaming operation until they got sued by Disney. They are still around but have since switched to a streaming only model. You could reach out and see if they still have their DVD ripping infrastructure, I'm sure they would love to help.

baldfat 8y ago

One PC with 8 DVD R/W Drives. I use to own one in the 90s. I would do work for churches and conferences. I could get a thousand done in a few days. They use to have automatic feeder drives for producing more. No one sells them anymore it appears.

deleted 8y ago

[deleted]

PakG1 8y ago

I manage the IT for a school. As we've gone more and more away from DVDs (thanks, Apple), teachers have been asking me to rip their old DVDs. Tried doing that, but a lot of the DVDs did not rip perfectly, so that the ripped video ended up having a lot of corruption. I'm not entirely clear why software could not play the ripped DVDs and just deal with the jitter and corrupted data, but playing from a DVD drive could deal with it. It was weird. Googled a bit, couldn't find much information other than the fact that CDs and DVDs can deteriorate over time. My question is not necessarily related to OP. But can anyone tell me why my ripped DVDs could not play in DVD software properly when there was jitter or corrupted data, but it could play properly when in the DVD drive? When I got lucky, the ripped DVDs would play fine. Otherwise, it was play fine for a few minutes until it ran into the first bit of corruption. Confused the heck out of me.

verisimilitude 8y ago
As I understand it, software playing a ripped DVD doesn't "know" when a data source is a ripped DVD and does not expect any corruption -- it will not seek over or ignore issues like a DVD player connected to a TV will. I've run into this issue a lot, but fortunately MakeMKV, my software of choice, will fail to rip DVDs when it is unable to make a perfect rip. Then, I know there is a problem before I begin ffmpeg compression. The solution: a wet polisher for DVDs. Luckily, my local used-game shop polishes disks for $3 each. They use an [Elm Eco AutoSmart][1] and it makes the discs like-new. I have never seen a disc it could not fix. You'd have to ask/search around locally to find one near you... I think some libraries have these machines, too, to maintain their DVD collections. [1]: https://www.elm-usa.com/products/eco-autosmart https://www.elm-usa.com/products/eco-autosmart
deleted 8y ago
[deleted]
StillBored 8y ago
That's generally caused by a combination of a cheap/bad high speed DVD-ROM drive and/or bad ripping software/options settings. What seems to happen is the ripping software is running a marginal drive/dvd at full speed and disabling the drive retry on error. Combined with all the soft ways these drives fail it results in a bad rip. This used to happen to me a few years ago, and it was overwhelmingly just a sign of a marginal drive. For about 10 years, I was replacing drives about once a year. It doesn't happen as frequently since I started using bluray drives (usually LG) to rip DVDs and a piece of software designed for "piracy". Anyway, two software things to try, are lower the ripping speed to 1x-2x, and make sure to leave the drive retry on bad sectors enabled. Most decent ripping software will have options to control the error correction/retry logic. The problem is some copy protection schemes leave bad sectors on the disks, and this will hang a lot of the "honest" ripping software that doesn't know how to deal with it.

netrap 8y ago

There are many old SCSI and Firewire DVD changers available. Just gotta ask the right people.

porlune 8y ago

Does anyone know why they can't just network the file system? If he has network access, then it might be faster to just upload the entire set directly to another drive over the internet, since most universities have fairly good connections, that shouldn't be any slower then trying to write them all to DVDs.