5 ms·
As more of these emerge, we need a Registry V3 API with some key changes: images and layers should be (1) expressible as URNs, not URLs and (2) entirely content
by jacques_chester 6y ago
As more of these emerge, we need a Registry V3 API with some key changes: images and layers should be (1) expressible as URNs, not URLs and (2) entirely content-addressable. A nice bonus would be to (3) register a URI scheme to make image references more easily identified in other places (such as YAML, YAML oh and YAML).
Right now (1) causes the most pain. Suppose you have a reference to 'ubuntu'. That actually expands to `index.docker.io/library/ubuntu:latest`. If you decide to switch to a private registry or alternative public registry, you now need to find every instance of `ubuntu` and change it to `registry.example.com/library/ubuntu:latest`. This is called the "relocation problem".
You might think this is easy: just find references and update them. Firstly, what a colossal hack. Secondly, "just find"ing the references turns out to be a game of whack-a-mole. People stuff image references into all sorts of whacky places and at many different levels of detail. Any such solution is stringly typed and relies on a pile of wobbly heuristics. We need to do better.
So what do I mean by "URNs, not URLs"? I mean that an ideal scheme is agnostic to the registry's domain. Maven gets this basically correct: every package has its "coordinates" triplet of groupid:artifactid:version. These represent a universal name that can be submitted to any conforming Maven server for resolution into a pile of bits. Maybe they're there on the server, maybe they aren't, but you won't have to change all your references when you switch servers.
This leads to (2), content addressability. Perhaps you are confused -- doesn't that already exist? Yes, but no. You can add `@sha256:abcdef123...` to your references. But this resolves only within that domain and only for that repository name. I can wind up with a variety of names like `ubuntu@sha256:123abc...`, `library/ubuntu@sha256:123abc...` and so on. These are resolved to the same bits, but I am still stuck with the entire nightmare of image relocation.
It gets worse with layers. These are addressed by digest, but scoped to repositories. You can't ask the registry "do you have layer `10a9b4...`?", just as you can't ask "do you have an existing image of any name with the digest `bc4d9ee...?`". You must include the repository, such as `library/ubuntu`, in the query. That means you must know in advance what repos and images are already in the target registry, otherwise you can't even ask if they are in the target registry.
As for (3), it would be an alternative or addition to (1). If all image references started with `oci://`, then it would be tremendously easier to find them and modify them programmatically. There would still be jank and corner cases, but it would be less janky, with fewer corners.
- justincormack 6y agoThe reason layers are scoped is so that they can be authed to something. Public only registries don't have this restriction, but otherwise if you saw a reference to a layer then you would be able to access it. The alternatives are to encrypt everything, or shard by access control (potentially to larger chunks than they are now.
- jacques_chester 6y agoI almost mentioned the auth thing but left it out as a bit of an implementation detail. It would not be difficult to map from a digest to a list of repositories.
- justincormack 6y agoThats a information disclosure potentially.
- jacques_chester 6y agoI disagree. You can still apply rules to repo/image and have those descend to blobs, just as happens now. If you ask for a blob and have access to repo/image with that blob, you see it. Otherwise not.
- toomuchtodo 6y agoChampion such an updated standard.
- jacques_chester 6y agoHere I am!
- toomuchtodo 6y agoTouché :)