3 ms·

What's wrong with zip? It's fine. Maybe more than fine if it managed to be this popular for so many applications. From my understanding it's very good on floppy

What's wrong with zip? It's fine. Maybe more than fine if it managed to be this popular for so many applications. From my understanding it's very good on floppy disk too

tgbugs 6y ago
Consider a use case where you need high concurrent write performance and you don't want the things you are writing to be accidentally dissociated from the file. If you have to write to a zip file there will be overhead every time you append a new "file" because you have to rewrite the central registry at the end of the file. Basically there is no way to say to you operating system "hey, can I get a no-nonsense, don't what to have to come up with some other conventions that will be broken, way to have some additional storage and file system paths that are just for this file so I don't have to worry about name collisions?" I already have a perfectly good unique name under which I could stash information, but the posix standard basically prevents me from using that name as a way to safely namespace things that I need per file. However, to your original point I have partially implemented this [0] using zip files on the read side. I treat the internal paths inside the zip file as if they are rooted at the zip file itself. I haven't implemented rewriting though. 0. https://github.com/tgbugs/augpathlib/blob/master/augpathlib/zip.py https://github.com/tgbugs/augpathlib/blob/master/augpathlib/...
- helloycombinat 6y ago
  This legit makes no sense to me. Why would you need concurrent write on a zip file? How would you even know where to write when you depend on compressed sizes that haven't been done on other cores? From what I remember you basically can create a zip by putting a header than adding compressed files back to back (no idea why you couldn't use multiple cores to compress each and add when your ready) then add the filelist at the end. Like I have no idea what your issue is

What's wrong with zip? It's fine. Maybe more than fine if it managed to be this popular for so many applications. From my understanding it's very good on floppy disk too

tgbugs 6y ago

Consider a use case where you need high concurrent write performance and you don't want the things you are writing to be accidentally dissociated from the file. If you have to write to a zip file there will be overhead every time you append a new "file" because you have to rewrite the central registry at the end of the file. Basically there is no way to say to you operating system "hey, can I get a no-nonsense, don't what to have to come up with some other conventions that will be broken, way to have some additional storage and file system paths that are just for this file so I don't have to worry about name collisions?" I already have a perfectly good unique name under which I could stash information, but the posix standard basically prevents me from using that name as a way to safely namespace things that I need per file. However, to your original point I have partially implemented this [0] using zip files on the read side. I treat the internal paths inside the zip file as if they are rooted at the zip file itself. I haven't implemented rewriting though. 0. https://github.com/tgbugs/augpathlib/blob/master/augpathlib/zip.py https://github.com/tgbugs/augpathlib/blob/master/augpathlib/...

helloycombinat 6y ago
This legit makes no sense to me. Why would you need concurrent write on a zip file? How would you even know where to write when you depend on compressed sizes that haven't been done on other cores? From what I remember you basically can create a zip by putting a header than adding compressed files back to back (no idea why you couldn't use multiple cores to compress each and add when your ready) then add the filelist at the end. Like I have no idea what your issue is