7 ms·

Counterpoint: archiving costs money, even for public domain content. Who pays the archivist, if not the users of the archive? And the more obscure a work is, th

by Joeri 4y ago

Counterpoint: archiving costs money, even for public domain content. Who pays the archivist, if not the users of the archive? And the more obscure a work is, the fewer users of that work the cost can be spread over, so the oldest and most obscure materials would have the highest access cost. It is tricky, because one could argue government could take on the role of archiver, but do we really trust government to maintain an unfiltered archive indefinitely? Or we may say archives should be privately funded and there should be redundancy, but then how do we guarantee continuity of funding?

vlovich123 4y ago
IPFS kind of solves this I think. If content is interesting enough to archivers, they can pay to pin that data. If you make it open, then conceivably your costs could be reduced. I don’t know if IPFS supports it but you could image a fractional ownership model.
- lozenge 4y ago
  What's interesting today to people with money is not necessarily the same as what future historians and researchers will be interested in.
  - vlovich123 4y ago
    The amount of money to require you to record your entire waking moment in full high depth video is about 6 tib / year. The cost of storing that on the commercial cloud is relatively inexpensive. Less than a few hundred a month at most. My point is that the cost is not about what people with money choose to archive, it’s that there’s a curation challenge that is the bigger financial obstacle. Storage costs might also start to skyrocket if everyone starts to record 24/7 video about their lives (already a problem for video cameras).
    - tmtvl 4y ago
      Are you certain of that? If we say every waking moment in full high-def means 16 hours per day in glorious 1080p, I'd back-of-napkin that to 17 petabytes.
      - vlovich123 4y ago
        Hmm. Was I off by an order of magnitude? I took 1gib per hour which feels like a reasonable level of quality. 8760 hours per year is 8TiB per year, no? For 17 PiB you’d be using 1TiB per hour which is very very off (uncompressed video?). Is my math off somehow?
        tmtvl 4y ago
        1GB per hour of high-def video? Which encoder are you using, that sounds really good. ...Actually, now that I am looking up some actual data it does look like many formats come in lower than I expected for an hour, though I don't know whether those formats are: A) Free to use, and B) Full HD (1920x1080, what we in the EU call HD) video losslessly compressed. My back-of-napkin mistake was looking at a random 1080 video, seeing it was 2.2GB for 24 minutes and extrapolating from there. Let's see, Apple Animation (which I don't know whether it's free, but it can compress losslessly) comes in at 597.2 GB/h of 1080p video. I can't find numbers for VP9, but it is patent-encumbered. I can't find numbers for Lagarith, but it is GPL, although as a fork of HuffYUV it's optimized for encoding speed rather than storage. Well, according to Wikipedia, lossless compression can achieve up to 12x compression, so let's call it 80GB/h (I'm back-of-napkining again, bad habit I know), so 640TB/y?
tokai 4y ago
>do we really trust government to maintain an unfiltered archive indefinitely Several national libraries have done this for ~500 years by now with no end in sight.
- WeylandYutani 4y ago
  And companies have been known to lose source code and prints.
- bawolff 4y ago
  The Library of alexanderia only had to burn once.
  - bell-cot 4y ago
    That's an off-site backup issue, not a "who might start censoring?" issue.
  - deleted 4y ago
    [deleted]
em-bee 4y ago
isn't that the purpose of the library of congress in the US? and wasn't the original intention of the US system of registering copyright, that a copy of the registered work was shared with the library? the problem today is that there is to much content to collect it all, but i believe something like the MLK speech should have a copy in the library. apparently any US publisher is still required to send two copies of each publication to the library.
kmeisthax 4y ago
Distributing public domain content is a solved problem. You can put it on a government website with a BitTorrent seed, whereupon it'll be mirrored to Wikimedia Commons, Internet Archive, and everywhere else. Archival & digitization is the actually expensive part. I will note that "who pays" is a problem even before the content hits the public domain. What you're arguing was specifically one of the arguments used to justify the Sonny Bono Copyright Term Extension Act[0]. Unfortunately, copyright is the wrong tool for the job. It merely says that if you produce a work someone wants to buy, then they have to buy a copy of it from the person who made the work. The problem is, the vast majority of old works are not in any kind of demand. Congress and the EU made sure that copyright would vastly outlive the actual commercial lifespan of the work. So marginal[1] increases in term length do not result in marginal increases in preservation. Because nobody is going to pay for it regardless. In fact, nobody is even going to be aware that the work in question even exists. The way that archival works nowadays is that you preserve everything you can get your hands on. Because it's the common garbage that nobody cares about that is the most likely to be lost. Any market-based solution like copyright is likely to miss such works. [0] Derisively called the Mickey Mouse Protection Act, due to Disney lobbying for it, even though it was primarily started by the EU wanting everyone to adopt Germany's copyright term lengths [1] In the economics sense. e.g. "on the margin, does adding one more year of copyright result in one more work preserved"
pessimizer 4y ago
You're conflating archiving and archive maintenance here. There are great arguments that can still be made for funding by charging for access the people who search out rare and wonderful things and create fine copies of those things for people to enjoy. Taking an old film from being found in a barn or an attic to a beautifully restored video is sacred, and if people stop doing it films burn and books rot. Maintenance of a digital archive? Just put it out there with no legal threats or restrictions and let the right people download it once or twice. It'll last forever. Instead, everything is tied up in restricted access, legal knots or just the impression of legal knots. Eventually local hard drives will be as illegal as the RIAA wanted to make blank cassette tapes. All of your data will have to be in clouds managed by state-intimate monopolies to insure future legal compliance. Datahoarders, academics, filmmakers and others would build communities around peer-to-peer film archives with no legal encumbrances.

by Joeri 4y ago

vlovich123 4y ago

IPFS kind of solves this I think. If content is interesting enough to archivers, they can pay to pin that data. If you make it open, then conceivably your costs could be reduced. I don’t know if IPFS supports it but you could image a fractional ownership model.

lozenge 4y ago
What's interesting today to people with money is not necessarily the same as what future historians and researchers will be interested in.
- vlovich123 4y ago
  The amount of money to require you to record your entire waking moment in full high depth video is about 6 tib / year. The cost of storing that on the commercial cloud is relatively inexpensive. Less than a few hundred a month at most. My point is that the cost is not about what people with money choose to archive, it’s that there’s a curation challenge that is the bigger financial obstacle. Storage costs might also start to skyrocket if everyone starts to record 24/7 video about their lives (already a problem for video cameras).
  - tmtvl 4y ago
    Are you certain of that? If we say every waking moment in full high-def means 16 hours per day in glorious 1080p, I'd back-of-napkin that to 17 petabytes.
    - vlovich123 4y ago
      Hmm. Was I off by an order of magnitude? I took 1gib per hour which feels like a reasonable level of quality. 8760 hours per year is 8TiB per year, no? For 17 PiB you’d be using 1TiB per hour which is very very off (uncompressed video?). Is my math off somehow?
      - tmtvl 4y ago
        1GB per hour of high-def video? Which encoder are you using, that sounds really good. ...Actually, now that I am looking up some actual data it does look like many formats come in lower than I expected for an hour, though I don't know whether those formats are: A) Free to use, and B) Full HD (1920x1080, what we in the EU call HD) video losslessly compressed. My back-of-napkin mistake was looking at a random 1080 video, seeing it was 2.2GB for 24 minutes and extrapolating from there. Let's see, Apple Animation (which I don't know whether it's free, but it can compress losslessly) comes in at 597.2 GB/h of 1080p video. I can't find numbers for VP9, but it is patent-encumbered. I can't find numbers for Lagarith, but it is GPL, although as a fork of HuffYUV it's optimized for encoding speed rather than storage. Well, according to Wikipedia, lossless compression can achieve up to 12x compression, so let's call it 80GB/h (I'm back-of-napkining again, bad habit I know), so 640TB/y?

tokai 4y ago

>do we really trust government to maintain an unfiltered archive indefinitely Several national libraries have done this for ~500 years by now with no end in sight.

WeylandYutani 4y ago
And companies have been known to lose source code and prints.
bawolff 4y ago
The Library of alexanderia only had to burn once.
- bell-cot 4y ago
  That's an off-site backup issue, not a "who might start censoring?" issue.
- deleted 4y ago
  [deleted]

em-bee 4y ago

isn't that the purpose of the library of congress in the US? and wasn't the original intention of the US system of registering copyright, that a copy of the registered work was shared with the library? the problem today is that there is to much content to collect it all, but i believe something like the MLK speech should have a copy in the library. apparently any US publisher is still required to send two copies of each publication to the library.

kmeisthax 4y ago

Distributing public domain content is a solved problem. You can put it on a government website with a BitTorrent seed, whereupon it'll be mirrored to Wikimedia Commons, Internet Archive, and everywhere else. Archival & digitization is the actually expensive part. I will note that "who pays" is a problem even before the content hits the public domain. What you're arguing was specifically one of the arguments used to justify the Sonny Bono Copyright Term Extension Act[0]. Unfortunately, copyright is the wrong tool for the job. It merely says that if you produce a work someone wants to buy, then they have to buy a copy of it from the person who made the work. The problem is, the vast majority of old works are not in any kind of demand. Congress and the EU made sure that copyright would vastly outlive the actual commercial lifespan of the work. So marginal[1] increases in term length do not result in marginal increases in preservation. Because nobody is going to pay for it regardless. In fact, nobody is even going to be aware that the work in question even exists. The way that archival works nowadays is that you preserve everything you can get your hands on. Because it's the common garbage that nobody cares about that is the most likely to be lost. Any market-based solution like copyright is likely to miss such works. [0] Derisively called the Mickey Mouse Protection Act, due to Disney lobbying for it, even though it was primarily started by the EU wanting everyone to adopt Germany's copyright term lengths [1] In the economics sense. e.g. "on the margin, does adding one more year of copyright result in one more work preserved"

pessimizer 4y ago

You're conflating archiving and archive maintenance here. There are great arguments that can still be made for funding by charging for access the people who search out rare and wonderful things and create fine copies of those things for people to enjoy. Taking an old film from being found in a barn or an attic to a beautifully restored video is sacred, and if people stop doing it films burn and books rot. Maintenance of a digital archive? Just put it out there with no legal threats or restrictions and let the right people download it once or twice. It'll last forever. Instead, everything is tied up in restricted access, legal knots or just the impression of legal knots. Eventually local hard drives will be as illegal as the RIAA wanted to make blank cassette tapes. All of your data will have to be in clouds managed by state-intimate monopolies to insure future legal compliance. Datahoarders, academics, filmmakers and others would build communities around peer-to-peer film archives with no legal encumbrances.