5 ms·

You can virtualize a PC (or replicate the filesystem like Dropbox), but it is harder to virtualize a distributed system like Facebook's or Google's. And most mo

by herf 12y ago

You can virtualize a PC (or replicate the filesystem like Dropbox), but it is harder to virtualize a distributed system like Facebook's or Google's. And most mobile platforms are DoA without their vendor signing and cloud services. It's always been hard to migrate out of social systems--convincing a well-connected user of a photo service to move away from the place where they've accumulated comments and tags is really hard. That metadata is not portable because identity is not yet portable, and it's what we're spending time on (lots more time than we spend making spreadsheets). I think we might manage to keep "JPG as file" alive for 30-50 years, but there is lots more to manage.

vidarh 12y ago
This is the crux of the problem. We are getting better at archiving files. And thanks to a combination of people dedicated to emulation and the rise of virtual machines, there are few popular pieces of hardware we can't emulate in excruciating detail. But many of the services I used a decade ago are already gone. And pretty much all the services I used two decades ago have completely disappeared, with some very few notable examples. And with them, vast amounts of data. Some of it the Internet Archive have at least captured static snapshots of (and they really should have magnitudes more funding), but ten times that - or more - was data in walled gardens, behind logins or otherwise restricted in ways that means it is lost forever unless we're lucky and it turns out some admin held onto backup tapes they weren't really meant to keep. And the problem with there is not to create a snapshot of a single server, but as you say that distributed systems are far harder. Even recent. Twice I've been contracted to help companies take over infrastructure that involved systems I'd worked on, and try to "package it up", and it was incredibly hard, because no matter how much you try to tear down and bring up individual servers or groups of servers and automate deployment, very few places running complex services ever try - or could afford to try - to tear down and bring up a full copy of their entire infrastructure. Suddenly all kinds of nasty interdependencies and bootstrap problems nobody had needed to think about shows up.

by herf 12y ago

vidarh 12y ago

This is the crux of the problem. We are getting better at archiving files. And thanks to a combination of people dedicated to emulation and the rise of virtual machines, there are few popular pieces of hardware we can't emulate in excruciating detail. But many of the services I used a decade ago are already gone. And pretty much all the services I used two decades ago have completely disappeared, with some very few notable examples. And with them, vast amounts of data. Some of it the Internet Archive have at least captured static snapshots of (and they really should have magnitudes more funding), but ten times that - or more - was data in walled gardens, behind logins or otherwise restricted in ways that means it is lost forever unless we're lucky and it turns out some admin held onto backup tapes they weren't really meant to keep. And the problem with there is not to create a snapshot of a single server, but as you say that distributed systems are far harder. Even recent. Twice I've been contracted to help companies take over infrastructure that involved systems I'd worked on, and try to "package it up", and it was incredibly hard, because no matter how much you try to tear down and bring up individual servers or groups of servers and automate deployment, very few places running complex services ever try - or could afford to try - to tear down and bring up a full copy of their entire infrastructure. Suddenly all kinds of nasty interdependencies and bootstrap problems nobody had needed to think about shows up.