4 ms·

I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there for

I'm not surprised this post opens with a link to /r/DataHoarder. Hot take ... I understand the sentiment that you can't trust content on the web to be there forever, but there is also the other side of the argument which is: compulsively saving data is a waste of time and it introduces a cognitive overhead that you'd be better off without.

atrus 2y ago
I'd disagree a bit there. I do something similar, saving interesting webpages, and it's really really nice being able to quickly search for something that I halfway remember a few months down the line. I'm not saving everything, and it just gets stuffed unedited into a folder that I can search. Not too much in the cognitive overhead department.
- turtlebits 2y ago
  Until run you out of space and/or storage bill is too expensive.
  - packetlost 2y ago
    if you're storing text that is realistically not going to be a problem.
packetlost 2y ago
> If it's worth saving Idk, I think if it's worth saving it's worth saving and the only person who can determine if it's "worth it" is me. I agree that some people have an obsession where they save data that isn't worth it, but r/DataHoarder is a great place with a lot of information on building and maintaining large data systems for hobbyists, regardless of what you actually store.
- xpe 2y ago
  Actually, “future you” will know even better in many cases. Also, some people other than yourself might turn out to be a better predictor of “future you” than “current you”. So many people are hyper individualistic and forget just how predictable individuals can be to others!
theshackleford 2y ago
> I understand the sentiment that you can't trust content on the web to be there forever The thing is, people say this, and I am sure for some amount of content it's true. However I eventually realised I have never had a single issue if required, in retrieving literally any piece of software or digital content after the fact from somewhere on the internet. It's pretty much why I care so little about what happens to my steam library when Gaben kicks it. If I get the urge to replay something in twenty years that I paid $3 dollars for and its suddenly gone, i'll just go find it elsewhere. Once I had gotten pretty much everything I wanted from running a large scale storage system (largely to learn the in's and outs of linux/general storage concepts) I pretty much just gave it up. Its a lot of money to hold onto things that at this point, I pretty much know i'll always be able to recover elsewhere. I'd rather someone else pay the electricity/drive cost for me.
- weikju 2y ago
  That’s only possible because some people are hoarders who save this data or software and put it out there..
  - theshackleford 2y ago
    > That’s only possible because some people are hoarders who save this data or software and put it out there.. Sure. And I appreciate it. Like I appreciate the internet archive etc.
    - johnfernow 2y ago
      There is no comparable alternative to the Internet Archive though. They've gotten involved in several lawsuits and their future is far from guaranteed. They're an incredibly important organization, but I think it's too important of a project to be limited to one organization, or even one country or region of the earth. A solar flare could destroy a lot of history. I don't know that the economics of having multiple Internet Archive-like organizations is currently feasible (I imagine getting funding for one of them is hard enough), but even a partial offline mirror hosted someplace else would be nice. Maybe to save space they could take the oldest version of a page, the newest, and the midmost version timewise, discarding all other versions. They could also heavily compress images, video and audio to save storage space (would increase processing costs, but if willing to throw out quality, could compress quickly and still save a bunch of space. E.g. downscale all videos to 480p and use veryfast preset and CRF 28 with ffmpeg. Even 240p is a lot better than nothing. A pixelated form of history is better than no history.)
NeutralForest 2y ago
When I find a blog post/article I'm interested in, I save it to my laptop with the SingleFile extension and I take quick notes as well I write my thoughts about it in org-mode. It has a very low cognitive threshold and I can always read it back in the browser. I'm find if not all the outbound links are still working, I'd just like to read back sometimes.
- DaSHacka 2y ago
  This is almost exactly what I do too, though I also throw the link in the wayback machine so it's easier to share with others should the source go down (and to be courteous to any like-minded fellows who also wanted to see the content, but unfortunately came too late)
filcuk 2y ago
I agree, but I also think that data hoarding is similar to regular hoarding. All of this content and information seems like it could be useful, invaluable even. It's a problem in a world where we have excess to sort through all that information and only focus on what's important right now.
maxbond 2y ago
An archivist saves everything, because it's impossible to anticipate what future historians will need or want; they are working from a context you cannot access. You can winnow down a large dataset to it's relevant subset, but you can't study what wasn't preserved.
polytely 2y ago
If I read something and then remember it 5 years later because it becomes relevant, I want to be able to find it. It's not even that I will look at it, I just want to have the option if I want to.
euroderf 2y ago
Nobody is taking their data hoard and feeding it into a personal AI ?

atrus 2y ago

I'd disagree a bit there. I do something similar, saving interesting webpages, and it's really really nice being able to quickly search for something that I halfway remember a few months down the line. I'm not saving everything, and it just gets stuffed unedited into a folder that I can search. Not too much in the cognitive overhead department.

turtlebits 2y ago
Until run you out of space and/or storage bill is too expensive.
- packetlost 2y ago
  if you're storing text that is realistically not going to be a problem.

packetlost 2y ago

> If it's worth saving Idk, I think if it's worth saving it's worth saving and the only person who can determine if it's "worth it" is me. I agree that some people have an obsession where they save data that isn't worth it, but r/DataHoarder is a great place with a lot of information on building and maintaining large data systems for hobbyists, regardless of what you actually store.

xpe 2y ago
Actually, “future you” will know even better in many cases. Also, some people other than yourself might turn out to be a better predictor of “future you” than “current you”. So many people are hyper individualistic and forget just how predictable individuals can be to others!

theshackleford 2y ago

> I understand the sentiment that you can't trust content on the web to be there forever The thing is, people say this, and I am sure for some amount of content it's true. However I eventually realised I have never had a single issue if required, in retrieving literally any piece of software or digital content after the fact from somewhere on the internet. It's pretty much why I care so little about what happens to my steam library when Gaben kicks it. If I get the urge to replay something in twenty years that I paid $3 dollars for and its suddenly gone, i'll just go find it elsewhere. Once I had gotten pretty much everything I wanted from running a large scale storage system (largely to learn the in's and outs of linux/general storage concepts) I pretty much just gave it up. Its a lot of money to hold onto things that at this point, I pretty much know i'll always be able to recover elsewhere. I'd rather someone else pay the electricity/drive cost for me.

weikju 2y ago
That’s only possible because some people are hoarders who save this data or software and put it out there..
- theshackleford 2y ago
  > That’s only possible because some people are hoarders who save this data or software and put it out there.. Sure. And I appreciate it. Like I appreciate the internet archive etc.
  - johnfernow 2y ago
    There is no comparable alternative to the Internet Archive though. They've gotten involved in several lawsuits and their future is far from guaranteed. They're an incredibly important organization, but I think it's too important of a project to be limited to one organization, or even one country or region of the earth. A solar flare could destroy a lot of history. I don't know that the economics of having multiple Internet Archive-like organizations is currently feasible (I imagine getting funding for one of them is hard enough), but even a partial offline mirror hosted someplace else would be nice. Maybe to save space they could take the oldest version of a page, the newest, and the midmost version timewise, discarding all other versions. They could also heavily compress images, video and audio to save storage space (would increase processing costs, but if willing to throw out quality, could compress quickly and still save a bunch of space. E.g. downscale all videos to 480p and use veryfast preset and CRF 28 with ffmpeg. Even 240p is a lot better than nothing. A pixelated form of history is better than no history.)

NeutralForest 2y ago

When I find a blog post/article I'm interested in, I save it to my laptop with the SingleFile extension and I take quick notes as well I write my thoughts about it in org-mode. It has a very low cognitive threshold and I can always read it back in the browser. I'm find if not all the outbound links are still working, I'd just like to read back sometimes.

DaSHacka 2y ago
This is almost exactly what I do too, though I also throw the link in the wayback machine so it's easier to share with others should the source go down (and to be courteous to any like-minded fellows who also wanted to see the content, but unfortunately came too late)

filcuk 2y ago

I agree, but I also think that data hoarding is similar to regular hoarding. All of this content and information seems like it could be useful, invaluable even. It's a problem in a world where we have excess to sort through all that information and only focus on what's important right now.

maxbond 2y ago

An archivist saves everything, because it's impossible to anticipate what future historians will need or want; they are working from a context you cannot access. You can winnow down a large dataset to it's relevant subset, but you can't study what wasn't preserved.

polytely 2y ago

If I read something and then remember it 5 years later because it becomes relevant, I want to be able to find it. It's not even that I will look at it, I just want to have the option if I want to.

euroderf 2y ago

Nobody is taking their data hoard and feeding it into a personal AI ?