4 ms·
Ask HN: What application file format to use for a desktop app?
I'm working on a desktop app and I'm trying to figure out the application file format.
I think that ideally, the application file format would internally be a directory but I want the directory to be hidden from the user.
Ideally there would be something like cross platform macOS bundles (https://en.wikipedia.org/wiki/Bundle_(macOS))
Sketch does this by zipping up the directory but I would like to avoid zipping things.
I don't want to use sqlite.
- PaulHoule 5y agoAlmost any modern application format is "a box that contains other boxes that themselves contain boxes". PDF is an interesting case study, so are the old Microsoft Office formats (e.g. a "doc" file) The new Microsoft Office (e.g "docx") format does what almost anybody would do today which is pack everything into a ZIP file, they just don't put a ZIP extension in it. It is not hard at all to treat a ZIP file as a filesystem or create it dynamically (look at the stdlib ZIP support in Python or Java) so you don't need to "zip up a directory". In fact the ZIP file format was designed to be editable in place, with the slight awkwardness that you have to rewrite the directory at the end of the file if you extend the file. You can definitely make a "hole" at the end of the file that you can write new items at the beginning of and write new directory entries at the end of. You can "delete" a file by trashing its directory entries and at some point "garbage collect" the file removing all the holes.
- pwg 5y ago> I don't want to use sqlite. Why not? If you read this page here: https://sqlite.org/appfileformat.html https://sqlite.org/appfileformat.html You'll find that your requirement of "internally be a directory" can be done using Sqlite with this one, single, simple table: CREATE TABLE files(filename TEXT PRIMARY KEY, content BLOB); I.e., you do not have to create a traditional relational DB schema if you do not want to do so. Of course, if you later do want to add additional relational data, if you are already using Sqlite then doing so becomes trivial. Plus you mention "cross platform macOS bundles", Sqlite provides cross platform compatibility automatically (even to Linux or Windows platforms). And, presumably, your focus is on the desktop app rather than a file format. Using Sqlite then removes your need to distract your focus to a file format.
- adamnemecek 5y agoSQLite does too much copying. I might have large binary assets and copying back and forth sounds terrible.
- pwg 5y agoAnd how is your custom format going to avoid copying? If you use the standard open()/[read()|write()]/close() then every read() or write() is a copy operation. Either copying from memory out to storage, or copying from storage into memory. If you instead mmap() the data then the copying is just hidden behind the VM page swapping routines in the kernel. But ultimately there will be a copy from storage into memory or a copy from memory into storage. And, have you bench-marked using Sqlite (in the simple form presented in the web page linked to above) to know whether the copying is an issue? If you have not bench-marked it, then you are just going on assumptions, and assumptions often have a very high probability of being wrong.
- pestatije 5y agoYou could zip without compression. Doing so lets you not only edit the zip file in-place, but also the files within the zip. If one of the files grows in size, you can also extend the whole zip by just appending the overgrown file.
- yuppie_scum 5y agoDocker image.