6 ms·

> You can still do that now, but it's not as important I think you hit the nail here. Most contemporary journalism is just not worth archiving. There is a sm

> You can still do that now, but it's not as important I think you hit the nail here. Most contemporary journalism is just not worth archiving. There is a small percentage that definitely is worth archiving, but the bulk of it is produced for the moment.

fellerts 2y ago
Who are we to decide? Many television tapes were wiped or lost last century because people didn't think them important, including a big chunk of early Doctor Who and a lot of the Apollo moon landing coverage.
- bayindirh 2y ago
  > a lot of the Apollo moon landing coverage. Even NASA (accidentally?) recycled the master tapes containing the moon landing AFAIK, and some of them are ultimately recovered. Shortsightedness, I may say.
  - largelady97 2y ago
    [flagged]
    - lores 2y ago
      What are you doing posting this on Hacker News? Don't you know it's majority-owned by Jon Danilovsky, the son of the moon landing movie producer? Oh god they have your IP now, you only have hours.
bayindirh 2y ago
> Most contemporary journalism is just not worth archiving. I don't agree. There are many interesting tidbits which go unnoticed as bog-standard / disposable news until the dots connect in the future, and one wants to construct the history backwards from that tipping point. When these "contemporary articles" go missing, many important details of the history is lost.
- brisketbbq 2y ago
  Are you really interested in reading a 50 year old newspaper article about a collision that happened midday on Saturday afternoon in your community newspaper?
  - xp84 2y ago
    Of course not. BUT on the other hand that particular collision may become highly relevant in those 50 years. Imagine your own scenario where the facts reported on the day might conflict with something claimed to be true much later, and the person wasn’t notable at the time but they’re highly notable now. The issue is we don’t know what may be important after many years, decades or centuries have passed. Given how easily text compresses, it would be a shame to not have at least text of real news sources archived perpetually. I know there is also an endless stream of listicles which are just generated to get clicks and ad views, but newspapers are worth archiving.
    - brisketbbq 2y ago
      > Imagine your own scenario where the facts reported on the day might conflict with something claimed to be true much later, and the person wasn’t notable at the time but they’re highly notable now. I do see your point. I'm reminded of the photograph who snapped basically a digital throw away picture of Monica Lewinsky weeks/months/years before anyone really knew who she was. later on, he was happy to have that picture, since it was one (or the only one) of her at some event hugging Clinton. Text is easy enough to store, but making it useful to search and access seems like another problem to solve.
- ghaff 2y ago
  The challenge is that it's hard to know. One of the people who recovered a lot of old Usenet archives once remarked that the reality was that no one cares about the details of some long ago SunOS bug whereas a lot of the cultural discussions about government policies etc. provide a window into the time.
- sirspacey 2y ago
pavlov 2y ago
Storage is also massively cheaper than in the past. It's worth archiving everything that was published on the Internet because you never know what it might reveal in a future context. Maybe the comment thread on a trivial clickbait article contains a post by a future dictator, and it will be a crucial piece of her biography fifty years from now.
dspillett 2y ago
> but the bulk of it is produced for the moment. But to a historian it is all a series of moments. To understand how/why one happened, it is useful to know the ones that came before, and things created for the moment might better describe it than those with a wider purview. Of course you don't need it all, but deciding what to keep is a complex (and bias ridden, accidental or otherwise) problem in its own right.
s_dev 2y ago
>Most contemporary journalism is just not worth archiving Considering how much historians love ancient rubbish bins, one mans junk is another mans treasure. Even the junk produced during and before the Trump/Brexit 2016 election/referendum would be massively valuable to a future historian in explaining how those things happened.
Dylan16807 2y ago
The text of every article is easily worth archiving. If we take average article size to be 500-800 words, then 2KB each is plenty. You want to hang on to 10 billion articles? That will cost you 1 (ONE) hard drive. Remember to make backups. If we allocate 200KB per article for pretty strongly compressed images, then it's still 100 million on a single hard drive. Why not archive the whole lot and let God or future historians sort it out? You only need to start getting picky when video is involved, but on the other hand when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience. And even 8MB Shrek gets the point across.
- robertlagrant 2y ago
  > when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience That's still pushing a terabyte a day[0] on YouTube alone. What about TikTok as well? [0] https://www.statista.com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute https://www.statista.com/statistics/259477/hours-of-video-up...
  - estebank 2y ago
    So, a single laptop hard drive a day? 50 tape drives a year? For the biggest repository of video there is? That's literally peanuts. There's some logistics in handling backup tapes, but that's a well understood process.
  - Dylan16807 2y ago
    I think you undershot that by a factor of 60. But when I said "get picky" I meant things like not archiving all of youtube. And for articles specifically, archiving every video article is a lot smaller. https://routenote.com/blog/wp-content/uploads/2020/08/pex-youtube-analysis-2019-total-videos-tiers-700x475.png https://routenote.com/blog/wp-content/uploads/2020/08/pex-yo... And going by this, restricting to a reasonable view count will cut the space you need by a factor of 10 to 25. If we round that to a petabyte per year, it's not in the range of a typical personal archive, but it's a reasonable thing to picture several big libraries doing. (Though a single person could handle that much if they really wanted to, spending $10 a day on data tapes.)
dalore 2y ago
In a way it was archived, into the minds and training data of AI models.
- xp84 2y ago
  This may in fact be one of the most effective forms of archival available to us. (Handwaving away the preservation of compatibility with future hardware or the maintenance of an “emulation chain” to get us back from whatever GPUs are there in 2124 to something our today models would run on) We could train a model every year and preserve it, then future historians can quiz this model that thinks it’s 2024 and ask it whatever they need to know. It’s fascinating because it will probably “know” the kinds of everyday normal people things that are very hard to glean from only reading old news stories. Things like how the average person feels about their world, or how they feel about current events and why.
SoftTalker 2y ago
Most contemporary journalism isn't even worth reading, let alone archiving. It's published at virtually no cost compared to what newspapers and magazines used to cost. And with LLMs it's going to get even cheaper as you no longer need people to write the stories. We are spiraling into an era where we will be drowned in content that is all worth next to nothing.
bityard 2y ago
> Most contemporary journalism is just not worth archiving. It is impossible to know presently, how valuable something will be to someone in the future. I don't know if any historians have this as their motto, but I like to think they do.

fellerts 2y ago

Who are we to decide? Many television tapes were wiped or lost last century because people didn't think them important, including a big chunk of early Doctor Who and a lot of the Apollo moon landing coverage.

bayindirh 2y ago
> a lot of the Apollo moon landing coverage. Even NASA (accidentally?) recycled the master tapes containing the moon landing AFAIK, and some of them are ultimately recovered. Shortsightedness, I may say.
- largelady97 2y ago
  [flagged]
  - lores 2y ago
    What are you doing posting this on Hacker News? Don't you know it's majority-owned by Jon Danilovsky, the son of the moon landing movie producer? Oh god they have your IP now, you only have hours.

bayindirh 2y ago

> Most contemporary journalism is just not worth archiving. I don't agree. There are many interesting tidbits which go unnoticed as bog-standard / disposable news until the dots connect in the future, and one wants to construct the history backwards from that tipping point. When these "contemporary articles" go missing, many important details of the history is lost.

brisketbbq 2y ago
Are you really interested in reading a 50 year old newspaper article about a collision that happened midday on Saturday afternoon in your community newspaper?
- xp84 2y ago
  Of course not. BUT on the other hand that particular collision may become highly relevant in those 50 years. Imagine your own scenario where the facts reported on the day might conflict with something claimed to be true much later, and the person wasn’t notable at the time but they’re highly notable now. The issue is we don’t know what may be important after many years, decades or centuries have passed. Given how easily text compresses, it would be a shame to not have at least text of real news sources archived perpetually. I know there is also an endless stream of listicles which are just generated to get clicks and ad views, but newspapers are worth archiving.
  - brisketbbq 2y ago
    > Imagine your own scenario where the facts reported on the day might conflict with something claimed to be true much later, and the person wasn’t notable at the time but they’re highly notable now. I do see your point. I'm reminded of the photograph who snapped basically a digital throw away picture of Monica Lewinsky weeks/months/years before anyone really knew who she was. later on, he was happy to have that picture, since it was one (or the only one) of her at some event hugging Clinton. Text is easy enough to store, but making it useful to search and access seems like another problem to solve.
ghaff 2y ago
The challenge is that it's hard to know. One of the people who recovered a lot of old Usenet archives once remarked that the reality was that no one cares about the details of some long ago SunOS bug whereas a lot of the cultural discussions about government policies etc. provide a window into the time.
sirspacey 2y ago

pavlov 2y ago

Storage is also massively cheaper than in the past. It's worth archiving everything that was published on the Internet because you never know what it might reveal in a future context. Maybe the comment thread on a trivial clickbait article contains a post by a future dictator, and it will be a crucial piece of her biography fifty years from now.

dspillett 2y ago

> but the bulk of it is produced for the moment. But to a historian it is all a series of moments. To understand how/why one happened, it is useful to know the ones that came before, and things created for the moment might better describe it than those with a wider purview. Of course you don't need it all, but deciding what to keep is a complex (and bias ridden, accidental or otherwise) problem in its own right.

s_dev 2y ago

>Most contemporary journalism is just not worth archiving Considering how much historians love ancient rubbish bins, one mans junk is another mans treasure. Even the junk produced during and before the Trump/Brexit 2016 election/referendum would be massively valuable to a future historian in explaining how those things happened.

Dylan16807 2y ago

The text of every article is easily worth archiving. If we take average article size to be 500-800 words, then 2KB each is plenty. You want to hang on to 10 billion articles? That will cost you 1 (ONE) hard drive. Remember to make backups. If we allocate 200KB per article for pretty strongly compressed images, then it's still 100 million on a single hard drive. Why not archive the whole lot and let God or future historians sort it out? You only need to start getting picky when video is involved, but on the other hand when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience. And even 8MB Shrek gets the point across.

robertlagrant 2y ago
> when the alternative is total obliteration you can crush video down to 1MB per minute and have a tolerable VHS-like experience That's still pushing a terabyte a day[0] on YouTube alone. What about TikTok as well? [0] https://www.statista.com/statistics/259477/hours-of-video-uploaded-to-youtube-every-minute https://www.statista.com/statistics/259477/hours-of-video-up...
- estebank 2y ago
  So, a single laptop hard drive a day? 50 tape drives a year? For the biggest repository of video there is? That's literally peanuts. There's some logistics in handling backup tapes, but that's a well understood process.
- Dylan16807 2y ago
  I think you undershot that by a factor of 60. But when I said "get picky" I meant things like not archiving all of youtube. And for articles specifically, archiving every video article is a lot smaller. https://routenote.com/blog/wp-content/uploads/2020/08/pex-youtube-analysis-2019-total-videos-tiers-700x475.png https://routenote.com/blog/wp-content/uploads/2020/08/pex-yo... And going by this, restricting to a reasonable view count will cut the space you need by a factor of 10 to 25. If we round that to a petabyte per year, it's not in the range of a typical personal archive, but it's a reasonable thing to picture several big libraries doing. (Though a single person could handle that much if they really wanted to, spending $10 a day on data tapes.)

dalore 2y ago

In a way it was archived, into the minds and training data of AI models.

xp84 2y ago
This may in fact be one of the most effective forms of archival available to us. (Handwaving away the preservation of compatibility with future hardware or the maintenance of an “emulation chain” to get us back from whatever GPUs are there in 2124 to something our today models would run on) We could train a model every year and preserve it, then future historians can quiz this model that thinks it’s 2024 and ask it whatever they need to know. It’s fascinating because it will probably “know” the kinds of everyday normal people things that are very hard to glean from only reading old news stories. Things like how the average person feels about their world, or how they feel about current events and why.

SoftTalker 2y ago

Most contemporary journalism isn't even worth reading, let alone archiving. It's published at virtually no cost compared to what newspapers and magazines used to cost. And with LLMs it's going to get even cheaper as you no longer need people to write the stories. We are spiraling into an era where we will be drowned in content that is all worth next to nothing.

bityard 2y ago

> Most contemporary journalism is just not worth archiving. It is impossible to know presently, how valuable something will be to someone in the future. I don't know if any historians have this as their motto, but I like to think they do.