3 ms·

Is there a good reason for not allowing parallel uploads in the spec?

by cpa 2y ago

wofo 2y ago
No idea... I asked the same question here (https://news.ycombinator.com/item?id=40943480 https://news.ycombinator.com/item?id=40943480) and am hoping we'll have a classic HN moment where someone who was involved in the design of the spec will chime in.
benterix 2y ago
I believe that even if there was one then, it's probably no longer valid and it's now just a performance limitation.
- wofo 2y ago
  Other than backwards-compatibility, I can imagine simplicity being a reason. For instance, sequential pushing makes it easier to calculate the sha256 hash of the layer as it's being uploaded, without having to do it after-the-fact when the uploaded chunks are assembled.
  - jtmarmon 2y ago
    I’m no expert on docker but I thought the hashes for each layer would already be computed if your image is built
    - wofo 2y ago
      That's true, but I'd assume the server would like to double-check that the hashes are valid (for robustness / consistency)... That's something my little experiment doesn't do, obviously.
    - cpuguy83 2y ago
      It's complicated. If you are using the containerd backed image store (opt-in still) OR if you push with "build --push" then yes. The default storage backend does not keep compressed layers, so those need to be recreated and digested on push. With the new store all that stuff is kept and reused.
  - catlifeonmars 2y ago
    That does not make any sense; as the network usually is a much bigger bottleneck than compute, even with disk reads. You’re paying quite a lot for “simplicity” if that were the case
  - amluto 2y ago
    The fact that layers are hashed with SHA256 is IMO a mistake. Layers are large, and using SHA256 means that you can’t incrementally verify the layer as you download it, which means that extreme care would be needed to start unpacking a layer while downloading it. And SHA256 is fast but not that fast, whereas if you really feel like downloading in parallel, a hash tree can be verified in parallel. A hash tree would have been nicer, and parallel uploads would have been an extra bonus.
    - cpuguy83 2y ago
      sha256 has been around a long time and is highly compatible. blake3 support has been proposed both in the OCI spec and in the runtimes, which at least for runtimes I expect to happen soon. I tend to think gzip is the bigger problem, though.
      - amluto 2y ago
        > sha256 has been around a long time and is highly compatible. Sure, and one can construct a perfectly nice tree hash from SHA256. (AWS Glacier did this, but their construction should not be emulated.)
        cpuguy83 2y ago
        But then every single client needs to support this. sha256 support is already ubiquitous.
        amluto 2y ago
        Every single client already had to implement enough of the OCI distribution spec to be able to parse and download OCI images. Implementing a more appropriate hash, which could be done using SHA-256 as a primitive, would have been a rather small complication. A better compression algorithm (zstd?) is far more complex.
        cpuguy83 2y ago

Is there a good reason for not allowing parallel uploads in the spec?

by cpa 2y ago

Is there a good reason for not allowing parallel uploads in the spec?

wofo 2y ago

No idea... I asked the same question here (https://news.ycombinator.com/item?id=40943480 https://news.ycombinator.com/item?id=40943480) and am hoping we'll have a classic HN moment where someone who was involved in the design of the spec will chime in.

benterix 2y ago

I believe that even if there was one then, it's probably no longer valid and it's now just a performance limitation.

wofo 2y ago
Other than backwards-compatibility, I can imagine simplicity being a reason. For instance, sequential pushing makes it easier to calculate the sha256 hash of the layer as it's being uploaded, without having to do it after-the-fact when the uploaded chunks are assembled.
- jtmarmon 2y ago
  I’m no expert on docker but I thought the hashes for each layer would already be computed if your image is built
  - wofo 2y ago
    That's true, but I'd assume the server would like to double-check that the hashes are valid (for robustness / consistency)... That's something my little experiment doesn't do, obviously.
  - cpuguy83 2y ago
    It's complicated. If you are using the containerd backed image store (opt-in still) OR if you push with "build --push" then yes. The default storage backend does not keep compressed layers, so those need to be recreated and digested on push. With the new store all that stuff is kept and reused.
- catlifeonmars 2y ago
  That does not make any sense; as the network usually is a much bigger bottleneck than compute, even with disk reads. You’re paying quite a lot for “simplicity” if that were the case
- amluto 2y ago
  The fact that layers are hashed with SHA256 is IMO a mistake. Layers are large, and using SHA256 means that you can’t incrementally verify the layer as you download it, which means that extreme care would be needed to start unpacking a layer while downloading it. And SHA256 is fast but not that fast, whereas if you really feel like downloading in parallel, a hash tree can be verified in parallel. A hash tree would have been nicer, and parallel uploads would have been an extra bonus.
  - cpuguy83 2y ago
    sha256 has been around a long time and is highly compatible. blake3 support has been proposed both in the OCI spec and in the runtimes, which at least for runtimes I expect to happen soon. I tend to think gzip is the bigger problem, though.
    - amluto 2y ago
      > sha256 has been around a long time and is highly compatible. Sure, and one can construct a perfectly nice tree hash from SHA256. (AWS Glacier did this, but their construction should not be emulated.)
      - cpuguy83 2y ago
        But then every single client needs to support this. sha256 support is already ubiquitous.
        amluto 2y ago
        Every single client already had to implement enough of the OCI distribution spec to be able to parse and download OCI images. Implementing a more appropriate hash, which could be done using SHA-256 as a primitive, would have been a rather small complication. A better compression algorithm (zstd?) is far more complex.
        cpuguy83 2y ago