8 ms·
The online data that's being deleted
- jedberg 5y agoMy wife and I used to get prints of all of our photos when we first started dating. Then we switched to digital, a lot of which has been lost on old hard drives. When we got married we had a really hard time finding photos of us as a couple because most had been lost. Now that we have kids we back up our photos in multiple places, but we also make sure to print out the ones we really like, just in case. The really good ones we print on canvas and hang on the walls. But it's true, most of records would be lost if there were a global cataclysm that took out all digital records. That being said, I'm pretty sure our mortgage is only digital so at least we would own our house free and clear!
- mceachen 5y agoOh man, I hear you: I had a mess just like that, tried to find something to automatically sweep everything into a neat pile, couldn't find it, and ended up writing PhotoStructure: https://photostructure.com/about/introducing-photostructure/ https://photostructure.com/about/introducing-photostructure/
- myself248 5y agoThis looks fantastic. I have a lot of comments about my old photos stored in 4DOS Descript.ion files, since I was using ACDsee to organize them. Can you speculate how to save those?
- mceachen 5y agoHuh, I've never heard of that format. If you send an example to hello@photostructure.com I can take a look: best case, ExifTool will be able to read them already, and you can just add .ion as a sidecar extension.
- trompetenaccoun 5y agoThis is just what I wanted for a long time, amazing to randomly come across it because I'd already given up thinking it doesn't exist. The only feature it's missing - but I know this would be complicated and advanced - would be to scan for similar images and duplicates that don't exactly match. Ideally it would present them all and then I could manually mark for deletion what I want to get rid of. So many duplicates. We were all so naive when digital photography first became a thing. I remember back in the day everyone recommended "keep all files, delete nothing, storage is cheap".
- judge2020 5y ago> That being said, I'm pretty sure our mortgage is only digital so at least we would own our house free and clear! Very similar to what happens in Mr. Robot where the plan is to encrypt all of the biggest worldwide bank's records so that they're inaccessible, thus wiping out the majority of debt.
- jedberg 5y agoSpoiler alert!
- gruez 5y agobut what about backups and/or other supporting documents? You'd have to hit multiple places at the same time, some of which aren't fully digital.
- daanvd 5y agoIf the world suddenly turns into a Mad Max movie set, your debtors will still hunt you down. Don't just assume your debt will magically disappear.
- drclau 5y agoHow will they know who to hunt if all the records are gone?
- new_guy 5y agoWouldn't matter. It'd be on you to prove you don't owe them money.
- readflaggedcomm 5y agoCreditors would be doing the hunting.
- floydnoel 5y agoThanks for the spoiler…
- thordenmark 5y agoOne of the worst data losses of photos I had was when apple switched from iPhoto (which I'd organized meticulously) to Photos. It wasn't until a few hard drive crashes later did I realize how poorly Photos imported the pictures. I found old iPhoto data files on an old Time Machine backup and was able to restore some, but anything that was linked was long gone. So most of the digital photos from my 20's are gone.
- Rd6n6 5y agoI met somebody once who told me that digital drives expire and will lose data unless they are “fired up” from time to time. Because of that, his company had a system of taking out and doing reads on their old storage in a clunky and manual way throughout the year. I wasn’t able to find out why this was necessary or the time frames involved. Can anybody confirm or deny or explain?
- Someone 5y agoThere were (¿are?) hard drives where the drive heads would stick to the surface if the drive wasn’t used for long periods (https://burgessforensics.com/data-recovery-myths-and-misconceptions/ https://burgessforensics.com/data-recovery-myths-and-misconc...)
- dehrmann 5y agoOver time, flash memory leaks the charge that holds your data. Here's an explanation I found: https://old.reddit.com/r/askscience/comments/5cexmj/how_stable_are_usb_thumbdrives_for_longterm/ https://old.reddit.com/r/askscience/comments/5cexmj/how_stab...
- Rd6n6 5y ago> FLASH memory works by essentially placing some electrons on an insulated island of material through quantum tunnelling and then "trapping" them there. However, there is some probability that they can tunnel off, and thus the stored charge does leak out over time. How long it takes for enough charge to leak off that a "read" can't tell the difference (signal to noise wise) depend on the manufacturer and specifications but 10 years is definitely in the danger zone. There's also a stress degradation issue but that probably doesn't apply here. Thanks for the link!
- WalterBright 5y agoWith the advent of cheap 16T drives, just buy a new drive and copy the stuff forward once a year or two.
- ksec 5y agoYeap. And we don't have a reliable Data Storage solution. Given the extreme low chances of disaster like fire, having a physical item is still so much better. We need a cheap storage solution that is reliable. Doesn't need to be fast. Even DVD dont last longer than 10 years. Someday this whole thing is going to back fire and we move back to hybrid / photobooks.
- Andrex 5y agoGlass seems fairly promising. > Microsoft and Warner Bros. have collaborated to successfully store and retrieve the entire 1978 iconic “Superman” movie on a piece of glass roughly the size of a drink coaster, 75 by 75 by 2 millimeters thick. > The hard silica glass can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetized and other environmental threats that can destroy priceless historic archives or cultural treasures if things go wrong. > “One big thing we wanted to eliminate is this expensive cycle of moving and rewriting data to the next generation. We really want something you can put on the shelf for 50 or 100 or 1,000 years and forget about until you need it,” Rowstron said. https://news.microsoft.com/innovation-stories/ignite-project-silica-superman/ https://news.microsoft.com/innovation-stories/ignite-project...
- myself248 5y agoBut I can't buy it for love or money, so AFAIC, it might as well not exist. Ditto with the mosquito-zapping laser that was supposed to change the world.
- XorNot 5y agoThis is really the storage technology I want. It's what CD-R's were promised to be (but came nowhere near delivering - organic dye is, well, organic). The big question I suppose is whether you can consumerize it. Though I suppose you could imagine the return of something like mall photobooths: you buy an organic dye substrate which something like a CD drive can write to, then drop it off to be "developed" into an impervious substrate by a harsher process (guessing there's an NH4F etch in that process).
- bonoboTP 5y agoIt's okay!! "What are my children or any potential grandchildren […] going to do with the 400 pictures of my pet that are on my phone?" Nothing, they don't need it. You sit your grandkid on your lap and look in their eyes and tell them stories about the family dog. They'll be happy to listen, they don't gain much by seeing your 400 photos. I guess it's a natural human impulse to try to leave a mark for the future. But the future will be okay, they don't need our crap data dumps. There's way more useless fluff being generated than what needs to be preserved. People survived when there was no photography. Yes, preserve your wedding photos, a few important life events here and there, that can be fun to look at for your kids. The full camera roll on your phone? Who cares? Let's not burden the coming generations with all this stuff. The future is theirs, not ours. For the few things that you want to preserve, make high quality physical prints. Their huge benefit is that they don't break silently. It's utterly obvious when looking at the physical object that it still "holds your data", as opposed to an HDD, DVD or tape which you need to put in a machine to check if it's still readable, which you won't reliably do. Analog printouts are a very robust preservation format. They can burn, but DVDs and HDDs aren't fireproof either (backup printouts are a solution too). Yes, prints may degrade in color, but it's a transparent process. But overall, forests need to clean themselves by fires, there needs to be a clean slate. I don't want to know who my great-great-grandparents were. I don't want their family photos. Maybe they were bad people, criminals etc. Maybe there was domestic abuse, trauma, tragedy etc. I don't need to know. I want to live my life with the people that are alive in the now. I don't want my ancestors to haunt me from the grave. And I don't want to haunt my descendants.
- throwawayboise 5y agoSo true. When I had to clean out my parents' house after my mom died, there were boxes and boxes of photos. Just in boxes, not really organized. Had been saved for years. I gave some thought to scanning them, but ultimately was honest and told myself that nobody had ever looked at these and nobody ever would. So they just got thrown away. Nobody really cares, years later, about day to day minutia preserved like this.
- WalterBright 5y ago
- caseysoftware 5y agoI started at the Library of Congress 20 years ago this month working on their digitization effort. It was HUGE even then - an estimated 50TB/day needed to be digitized - and that did not address the 200+ years of backlog or the advent of blogging (rare at the time), streaming media, and everything else online. Our mission was to ensure that the assets we were working with were usable by someone ~100 years from now. We would take something like an LP (Ray Charles was one of my favorites) and play it and record it in a lossless wav at a high bitrate. Then we'd take the physical record, the entire sleeve, and anything else included and scan that at 300dpi in tiff. Of course, we'd document every piece of equipment we used (down to the serial number) with every setting and who did the work. If we later realized a given piece of equipment was misconfigured, we could detect what had been created with it and adjust or re-create as necessary. I wrote some of the standards (XML schemas) for metadata collection which were still in use as late as 2015. (They may still be, I haven't checked.) Whew. Then we'd take a version of the audio and images to create mp3s and jpgs for listening+viewing in the reading rooms. The original physical media would go into nitrogen filled vaults in Culpepper, Virginia to prevent oxidization and further deterioration.* We had similar processes for other media like books, wire spool recordings (popular in WW2), wax cylinder recordings, and pictures around the Civil War. Many of them were exceptionally fragile and required special equipment. My least favorite but most fascinating were Thomas Edison's first motion pictures. Pop quiz: What does cotton and silver nitrate degrade into? ;) But in all of those cases, we could go back to the original media which - hopefully - hadn't deteriorated beyond recovery. In a digital-first or digital-only environment, we're counting on fragile components storing rotting file formats on deprecated operating systems in physical locations that may suffer disasters. * At the time, the RIAA was complaining endlessly and we may have made jokes about RIAA execs and those vaults but I can neither confirm nor deny any such proposals.
- detaro 5y ago> Pop quiz: What does cotton and silver nitrate degrade into? ;) Bright light if you look at it wrong?
- WalterBright 5y agoThis excessive detail seems nutty to me, and ultimately counterproductive. It's counterproductive in that it reduces the amount of things that can be preserved. What's more important - audio detail that cannot be heard, or more records? The best way to archive data long term is to make diverse copies. The easiest way to make diverse copies is to simply make them available to download off of the internet. Don't restrict it to "reading rooms".
- bromuro 5y agoOn the other side, our facebook data will never be deleted.
- abeppu 5y ago> Scott says he thinks there should be legal or regulatory requirements on companies that give people the option to retrieve their data, for a certain period – say, five years – after an online service is due to shut down. I get the motivation here, but in addition to keeping the stored data, this means one must keep account and authentication systems, maintain a service and UI for people to request this data, etc for years. That seems like an excessive obligation, depending on what the product was which took in the data to begin with. What data should be eligible? If I ordered delivery from a service which folded, should I really be able to request my order records years later?
- BeFlatXIII 5y agoAlso, who’d pay for that obligation if the service shuts down because the provider went bankrupt? Would such a regulation only apply to paid services?
- abeppu 5y agoThe idea of protecting data against catastrophic events reminds me of the KEO satellite project, which I think is not yet officially dead but has been delayed to the point that it seems doomed. But they aspired to put a lot of data in a very durable format on a satellite, to act as a kind of time capsule, intended to be recovered tens of thousands of years into the future. I think the intent was to use optical media, but made of glass. Clearly this isn't yet an example of successfully preserving data. But the mindset of setting a goal to make something readable in the very distant future seems admirable.
- WalterBright 5y ago> She studied maths at university and has copies of her handwritten notes. “There’s a point when I started taking digital notes and I can’t find them,” she says with a laugh. I've lost much of my handwritten notes from university. I used to handwrite code in notebooks before typing it in. That's all gone.
- xyzzy21 5y agoI'm always amazed that people actually think "online/digital is forever". I NEVER has been and never will be. And as we each "create" more data, the life span will decrease as fast. Even websites can't be assumed to remain at the other end of the bookmark you keep. You MUST download it if you really want an archive you can count on. If you aren't buying a new mass storage device EVERY year and moving anything older than 3 years to the new storage, you are taking a risk. This is why I still buy physical books rather than Kindle et al. This is why I convert personal documents and momentos like photos to paper, et al. if it actually matters. I still have photos of my ancestors from the 1900s-1920s only because they are on paper. And I've worked on most of the advanced technologies in use today over the last few decades - high priest level contact. But the truth you learn if you learn anything is: you should not put technology first in your life! It's a tool but not even the best tool.
- WalterBright 5y ago> only because they are on paper One side of my family lost the photos from a leak in the roof. The other side lost the photos because the house was destroyed. (The handful left were ones that were copies given to friends and relatives.) Paper rots and burns.
- ReactiveJelly 5y agoOnline is forever when you want something gone, and ephemeral when you want it preserved. Kinda like how conceiving a child is hard if you want one and easy if you don't.
- a1369209993 5y agoOnline/digital is forever if, and only if, it is disadvantageous to you for that particular thing to be forever.
- exikyut 5y ago> I've worked on most of the advanced technologies in use today over the last few decades The "war stories for hours" is very strong with this statement, I feel. I figure that a lot of them can sadly never be told :( but I'd like to firmly +1 the idea of considering what can be shared. Sign me up :P
- Stampo00 5y agoI know this is an unusual view, and I'm not endorsing it for others, but please delete my data. I want to have as little as possible. Obviously, if it's something precious and needs saving, do the due diligence of backups and hard copies when applicable. But I fear becoming a digital hoarder. I fear it becoming a cross to bear. Not to mention, the less you keep, the less you stand to lose.
- deleted 5y ago[deleted]
- alexanderdmitri 5y agoBBC often embed links in their articles as references. I am a little surprised they don't have an archiving mechanism to manage this content, especially things like tweets (although maybe a bad example b/c twitter should be banned from journalism altogether, kind of like separation of church and state).
- spoonjim 5y agoOne upside of this is that the nonconsensual "revenge porn" that used to plague many victims is sometimes actually disappearing from the Internet for good. I had a friend whose ex-boyfriend uploaded a consensually filmed video in the early 2000s of her giving him oral sex, with her real name, onto porn sites. She went the police and they said they couldn't do anything. She would file takedowns with the sites and the video would always pop back up later. It wasn't the ex-boyfriend doing it because he was in jail for some of that time. Online trolls sent the video to her parents, to her employers, and to her friends. It seemed like she would never get away from this video, but in recent years it seems to have actually disappeared. It doesn't pop back up. Hasn't for a long time, and even if it does again, it's been some good years of relief for her.
- deleted 5y ago[deleted]