3 ms·

Do you not still pay a significant performance penalty by reverifying the container upon each application load? Especially considering that, if the container is

Do you not still pay a significant performance penalty by reverifying the container upon each application load? Especially considering that, if the container is signed, you need to verify the signature itself before trusting the container, and full signature verification - including checking whether the signature has been revoked - involves expensive network calls? If your operational and security model really frowns on trusting your extraction cache, then perhaps a different workflow is more appropriate - download the container, verify the container, extract, bake the OS plus extracted apps into an image, sign the image, verify the image upon each boot and mount apps read-only. Then you don't need to re-verify anything upon each launch, instead trusting that your image creation process is routinely updating and re-verifying the software in your current images.

theamk 8y ago
Verification of a single file is much faster than walking entire tree, especially when there are lots of small files, for example when there is a compiler or large python project inside. A simple example: my /usr/include is 33037 files, 356M uncompressed. On SSD with cold cache, it takes 6.7 sec to read each file individually, or 0.7 sec to checksum a single 356M archive, a 10x difference. The difference in the backup time is even more dramatic -- the backup program has to call stat() either 33K times, or just once, a 3,330,000% improvement! The other filesystem tools (What takes all the space? What has changed in the last X hours? Please sync this directory elsewhere.) will have similarly high speed improvements. So if I had a choice, I would love my dev environment to come in mountable form. Similarly, I don't understand why container runtimes (like docker) don't use loop mounts more -- it seems like many advantages and very few disadvantages. As for signature verification -- I don't care about 3rd party signature and revocation, I just want to ensure that I am running the same code every time. There are many ways one can damage extraction cache, especially if it is owned by the same user as application (like the topicstarter post described) -- sysadmin errors (`sudo find / -name app-old -delete`), application errors (create cache file in bin dir), disk errors (silent corruption), transfer errors (one file did not get transferred to a new computer). Loop mounting makes disk errors easier to detect, and eliminates other classes of error entirely.

theamk 8y ago

Verification of a single file is much faster than walking entire tree, especially when there are lots of small files, for example when there is a compiler or large python project inside. A simple example: my /usr/include is 33037 files, 356M uncompressed. On SSD with cold cache, it takes 6.7 sec to read each file individually, or 0.7 sec to checksum a single 356M archive, a 10x difference. The difference in the backup time is even more dramatic -- the backup program has to call stat() either 33K times, or just once, a 3,330,000% improvement! The other filesystem tools (What takes all the space? What has changed in the last X hours? Please sync this directory elsewhere.) will have similarly high speed improvements. So if I had a choice, I would love my dev environment to come in mountable form. Similarly, I don't understand why container runtimes (like docker) don't use loop mounts more -- it seems like many advantages and very few disadvantages. As for signature verification -- I don't care about 3rd party signature and revocation, I just want to ensure that I am running the same code every time. There are many ways one can damage extraction cache, especially if it is owned by the same user as application (like the topicstarter post described) -- sysadmin errors (`sudo find / -name app-old -delete`), application errors (create cache file in bin dir), disk errors (silent corruption), transfer errors (one file did not get transferred to a new computer). Loop mounting makes disk errors easier to detect, and eliminates other classes of error entirely.