5 ms·

What innovations in file storage and manipulation would characterize a progression to the Bronze Age?

vinceguidry 9y ago
I would say when files become easily storable and accessible. Right now untold amounts of human labor are destroyed whenever the computer systems those humans are using fail. When I think about how most people use computers, I get the distinct feeling like I'm watching an episode of Primitive Technologies. Systems like Dropbox are a good first step, but the archival of files is still far too complicated. This stuff needs to be easy enough for your 80 year old grandma to master. I'd say we're probably 20-50 years out. We're eventually going to figure out a basic file format that all applications will use and extend. Then it'll become necessary for all applications to support it. It'll have solved the encoding problem, and it'll have built-in error correction. Sort of a Unicode for file formats. Once that's in place, then cloud storage and backup solutions will finally become useful.
- Splines 9y ago
  IMO one of the problems with cloud storage in general is how complicated it is. There are so many moving parts and so many things that could go wrong that there are inevitably many states a file could be in and asking them to troubleshoot these is impossible. For example, my wife is syncing several GBs of data up to Google Photos as we speak. Periodically, the sync process will complain about something and stop. There is nothing we can do except ask it to exit and we try again. What went wrong? Who knows. We certainly don't. The app UI isn't telling us anything. So we cross our fingers, restart the sync app and hope it works. This isn't the future. "A file" needs to be have the qualities we need as a paper-replacement as an intrinsic property of itself, not conceptual software design properties that are merely abstractions.
- mjevans 9y ago
  I conjecture that error detection (and recovery) are layers beneath the file format it's self. They belong in the filing system and storage layers within/under that. Application code should never need to worry about error recovery. We're kind of close to this for text files, if only Windows and related programs would join the rest of the sane world. Stop adding 'BOM Marks' and just assume everything is UTF-8 (at least for any txt file created after 2000 and all application stdin/out/errs). However the biggest barrier here is a complete disconnect between interested parties. Adobe wants to make money and keeps 'pdf' locked away. It's also bad because paper size is enforced, instead of having an aspect ratio, and positioning for different elements. Even HTML rendering isn't that simple, and NO designer is working using those tools. Additionally that complexity is FAR above what the average user can handle and WAY above what they want to deal with even if they can. Any format which /is/ user friendly will have to support things like 'drawings' / sketches which may or may not be auto OCRed for searching. I'm of the opinion that this won't really be solved until we have //strong// AI to do transcription and formatting for the lazy humans.
- nradov 9y ago
  Dropbox is fine until they go bankrupt or get acquired by another company which isn't interested in acting as your archive.
- fragmede 9y ago
  > We're eventually going to figure out a basic file format that all applications will use and extend. The general trend seems to skip that and is moving towards renting hosted software applications. Google docs is already pervasive in certain corners, and not only can I not tell you what version of Google docs I'm using, but I can't really tell you what its file formats are, or even where it's even actually stored other than "in the cloud". Assuming that trend continues, what, then, even are file formats?
  - vinceguidry 9y ago
    The hiding of file formats behind the application isn't new, but that doesn't make them go away. Should the app ever die, necessitating a migration, then the file formats will suddenly become painfully relevant again. We need a universal file format similar to plaintext but has standards-based extensions for storing data in that all applications can extract to. That way software can finally achieve paper's universality. It's the first step on the road to making information technology truly worthy of the name. The next would be easy-to-use programming platforms. Like Excel or Visual Basic but not shitty.
cryptoz 9y ago
Not the OP, but 98% of the time that someone sends me a .key file for a Mac Keynote presentation, my Keynote application is out of date and cannot open the file. Additionally, it sometimes cannot update either, unless other components are updated, such as the entire OS itself. A bare minimum requirement for me would be, "a computer can be reasonably expected to open the file". Yes, a computer without the correct application installed could not reasonably be expected to open the file. But here is a computer, perfectly capable of displaying the file, but choosing not to do so. Note that these are not files that are created with the new Keynote that I may not have installed. These are normal Keynote files created in the same version I have - but now my version isn't good enough to open files any more, it would seem. A piece of paper would never do that.
- kqr 9y ago
  > Yes, a computer without the correct application installed could not reasonably be expected to open the file. I disagree. I write my presentations in Org. Perfectly readable even to peooole without Emacs. I do my accounting using hledger. Perfectly readable etc. I do my $task using ${plain text format}, peefectly readble even to people without ${tool that helps me format the document nicely}. Etc.
  - gkya 9y ago
    You're part of a minority among a minority. And many institutions require you use some certain application software and file formats.
brendyn 9y ago
I'll cut in rudely with my own reply. Sharing files is unbelievably inconvenient. Sharing files to my mum in the other room involves copying files to a USB drive and moving them manually, after I reformat a USB to ntfs or fat32. It doesn't matter that both of our computers are on a LAN because by default we have no pre-existing convenient ways to share files. With her being on Windows and me being on GNU/Linux, I'm not even sure which program I would pick if I could install one. For anyone that isn't in USB drive proximity, I have to brainstorm a solution every single time I want to share a file for every person I want to give it too. This is all despite the fact that theoretically, there are a million different ways that we could share a file.
- lsadam0 9y ago
  There are numerous better options than a usb drive: Email, cloud storage, almost any messaging client, etc.
  - nradov 9y ago
    Yes well that's kind of the problem isn't it? Too many options and no default cross-platform standard.
    - lsadam0 9y ago
      All of the options I listed are cross platform. Why must a standard exist in this case? FTP is standard and cross platform. This is nothing more than a UX problem.
      - nradov 9y ago
        It's not really practical to run an FTP server on a mobile device.
  - 2muchcoffeeman 9y ago
    Most alternatives are really inadequate, especially for large files. When you are in the same room or even city, I should be able to just move large amounts of data from one computer to a physically adjacent computer without having to go through the internet. For long distance transfers, I agree with various cloud storage. Upload some files and create a share link. But even this is an annoyance for a large file.

I would say when files become easily storable and accessible. Right now untold amounts of human labor are destroyed whenever the computer systems those humans are using fail. When I think about how most people use computers, I get the distinct feeling like I'm watching an episode of Primitive Technologies. Systems like Dropbox are a good first step, but the archival of files is still far too complicated. This stuff needs to be easy enough for your 80 year old grandma to master. I'd say we're probably 20-50 years out. We're eventually going to figure out a basic file format that all applications will use and extend. Then it'll become necessary for all applications to support it. It'll have solved the encoding problem, and it'll have built-in error correction. Sort of a Unicode for file formats. Once that's in place, then cloud storage and backup solutions will finally become useful.

Splines 9y ago
IMO one of the problems with cloud storage in general is how complicated it is. There are so many moving parts and so many things that could go wrong that there are inevitably many states a file could be in and asking them to troubleshoot these is impossible. For example, my wife is syncing several GBs of data up to Google Photos as we speak. Periodically, the sync process will complain about something and stop. There is nothing we can do except ask it to exit and we try again. What went wrong? Who knows. We certainly don't. The app UI isn't telling us anything. So we cross our fingers, restart the sync app and hope it works. This isn't the future. "A file" needs to be have the qualities we need as a paper-replacement as an intrinsic property of itself, not conceptual software design properties that are merely abstractions.
mjevans 9y ago
I conjecture that error detection (and recovery) are layers beneath the file format it's self. They belong in the filing system and storage layers within/under that. Application code should never need to worry about error recovery. We're kind of close to this for text files, if only Windows and related programs would join the rest of the sane world. Stop adding 'BOM Marks' and just assume everything is UTF-8 (at least for any txt file created after 2000 and all application stdin/out/errs). However the biggest barrier here is a complete disconnect between interested parties. Adobe wants to make money and keeps 'pdf' locked away. It's also bad because paper size is enforced, instead of having an aspect ratio, and positioning for different elements. Even HTML rendering isn't that simple, and NO designer is working using those tools. Additionally that complexity is FAR above what the average user can handle and WAY above what they want to deal with even if they can. Any format which /is/ user friendly will have to support things like 'drawings' / sketches which may or may not be auto OCRed for searching. I'm of the opinion that this won't really be solved until we have //strong// AI to do transcription and formatting for the lazy humans.
nradov 9y ago
Dropbox is fine until they go bankrupt or get acquired by another company which isn't interested in acting as your archive.
fragmede 9y ago
> We're eventually going to figure out a basic file format that all applications will use and extend. The general trend seems to skip that and is moving towards renting hosted software applications. Google docs is already pervasive in certain corners, and not only can I not tell you what version of Google docs I'm using, but I can't really tell you what its file formats are, or even where it's even actually stored other than "in the cloud". Assuming that trend continues, what, then, even are file formats?
- vinceguidry 9y ago
  The hiding of file formats behind the application isn't new, but that doesn't make them go away. Should the app ever die, necessitating a migration, then the file formats will suddenly become painfully relevant again. We need a universal file format similar to plaintext but has standards-based extensions for storing data in that all applications can extract to. That way software can finally achieve paper's universality. It's the first step on the road to making information technology truly worthy of the name. The next would be easy-to-use programming platforms. Like Excel or Visual Basic but not shitty.

cryptoz 9y ago

Not the OP, but 98% of the time that someone sends me a .key file for a Mac Keynote presentation, my Keynote application is out of date and cannot open the file. Additionally, it sometimes cannot update either, unless other components are updated, such as the entire OS itself. A bare minimum requirement for me would be, "a computer can be reasonably expected to open the file". Yes, a computer without the correct application installed could not reasonably be expected to open the file. But here is a computer, perfectly capable of displaying the file, but choosing not to do so. Note that these are not files that are created with the new Keynote that I may not have installed. These are normal Keynote files created in the same version I have - but now my version isn't good enough to open files any more, it would seem. A piece of paper would never do that.

kqr 9y ago
> Yes, a computer without the correct application installed could not reasonably be expected to open the file. I disagree. I write my presentations in Org. Perfectly readable even to peooole without Emacs. I do my accounting using hledger. Perfectly readable etc. I do my $task using ${plain text format}, peefectly readble even to people without ${tool that helps me format the document nicely}. Etc.
- gkya 9y ago
  You're part of a minority among a minority. And many institutions require you use some certain application software and file formats.

brendyn 9y ago

I'll cut in rudely with my own reply. Sharing files is unbelievably inconvenient. Sharing files to my mum in the other room involves copying files to a USB drive and moving them manually, after I reformat a USB to ntfs or fat32. It doesn't matter that both of our computers are on a LAN because by default we have no pre-existing convenient ways to share files. With her being on Windows and me being on GNU/Linux, I'm not even sure which program I would pick if I could install one. For anyone that isn't in USB drive proximity, I have to brainstorm a solution every single time I want to share a file for every person I want to give it too. This is all despite the fact that theoretically, there are a million different ways that we could share a file.

lsadam0 9y ago
There are numerous better options than a usb drive: Email, cloud storage, almost any messaging client, etc.
- nradov 9y ago
  Yes well that's kind of the problem isn't it? Too many options and no default cross-platform standard.
  - lsadam0 9y ago
    All of the options I listed are cross platform. Why must a standard exist in this case? FTP is standard and cross platform. This is nothing more than a UX problem.
    - nradov 9y ago
      It's not really practical to run an FTP server on a mobile device.
- 2muchcoffeeman 9y ago
  Most alternatives are really inadequate, especially for large files. When you are in the same room or even city, I should be able to just move large amounts of data from one computer to a physically adjacent computer without having to go through the internet. For long distance transfers, I agree with various cloud storage. Upload some files and create a share link. But even this is an annoyance for a large file.