13 ms·

Your real problem isn't the data format, it's the host machine - you need to be sure that the computer where your files live is still around/working in 30 years

by jdormit 10y ago

Your real problem isn't the data format, it's the host machine - you need to be sure that the computer where your files live is still around/working in 30 years. Either make frequent backups to multiple machines, or choose a cloud provider that will be around for a while, e.g. AWS or similar (and even then, still do frequent backups to other machines). The data format you choose should depend on the relationships present in the data. If there are relational connections between data points (e.g. data1 HAS A data2, data3 HAS MANY data4s), then SQLite is a fine choice. If all you want to do is keep the data around, there is nothing wrong with text files - perhaps the directory structure could represent the organizational structure you require.

avh02 10y ago
oh yeah of course, it was a very theoretical "if i dig up this file in 30 years" - for the most part i access and update these files on the regular and have multiple backups. - but not much will beat stability of a format when things do go very wrong. I can't find it now, but I remember reading about an organisation trying to archive work that had been on floppies and even when the data was read, the file formats were obsolete. so I wanted to draw the question on that kind of parallel. as for the relationship format - definitely with visas/passports, a passport will have several visas on it, and will have taken journeys on it, etc. - again, i haven't done this _yet_ but it's been on my mind for a good year now and am just trying to take down my barriers to doing it.
- jdormit 10y ago
  Ah understood. In that case, I'll echo the comments below - I would choose SQLite to preserve data relationships with automated dumps to plain text (maybe JSON? Something parseable, anyway), since I can't imagine utf8-encoded text going obselete to the point where no programs exist that can read in the foreseeable future.

by jdormit 10y ago

avh02 10y ago

oh yeah of course, it was a very theoretical "if i dig up this file in 30 years" - for the most part i access and update these files on the regular and have multiple backups. - but not much will beat stability of a format when things do go very wrong. I can't find it now, but I remember reading about an organisation trying to archive work that had been on floppies and even when the data was read, the file formats were obsolete. so I wanted to draw the question on that kind of parallel. as for the relationship format - definitely with visas/passports, a passport will have several visas on it, and will have taken journeys on it, etc. - again, i haven't done this _yet_ but it's been on my mind for a good year now and am just trying to take down my barriers to doing it.

jdormit 10y ago
Ah understood. In that case, I'll echo the comments below - I would choose SQLite to preserve data relationships with automated dumps to plain text (maybe JSON? Something parseable, anyway), since I can't imagine utf8-encoded text going obselete to the point where no programs exist that can read in the foreseeable future.