docker/schema2: support manifest lists

[thepwagner]

The image manifest v2 allows returning an "manifest list", containing references to multiple manifests for distinct platforms. This is how a single image can represent multiple architectures.

Voucher should eventually support manifest lists, but doing so is a large refactor: many Manifest and ImageConfig objects become []Manifest and []ImageConfig.

This PR is aiming to "support" manifest lists rather quickly, by discarding all but a single target architecture (by default: linux/amd64, but it can be overridden by environment variables, I did not wire the configuration file through 🤢). This allows voucher to vouch for the linux/amd64 platform of multi-arch images.

A similar change will be required for any systems that resolve these images to vouched digests at deploy time. Until that happens, these images will be vouch-able but not deploy-able. Other systems will query by image label, and get fallback behaviour from the registry.

⚠️ We do not intend to send this functionality to https://github.com/grafeas/voucher ; we'll use this fork until full manifest list support is added (probably 1-2months)

Related

• Multi-arch image vouching fails: failed to load manifest with status 404 Not Found grafeas/voucher#47

[kimbilida]

I don't feel confident enough in this repo to give a good review. @ChrisBr or @shanesmith ?

[shanesmith]

Sorry I don't feel any more confident.

I'd like to make sure I understand what are the outcomes of this PR though. After it's merged we'll be able to push multi-arch images, only one platform (amd64 by default) will be vouched for, but the multi-arch image will still be available in gcr.io to be pulled down. Is that correct?

[thepwagner]

I'd like to make sure I understand what are the outcomes of this PR though. After it's merged we'll be able to push multi-arch images, only one platform (amd64 by default) will be vouched for, but the multi-arch image will still be available in gcr.io to be pulled down. Is that correct?

That's correct! For the worked example in grafeas#47 -

• sha256:fbb75455e35366f89fe310e88420a1a35a15ee54bc86aeef36fd561de68ce5ae would be vouched
• sha256:f587207cab008ab273d77a3b1c9f9c983888fa659a46e1284c1984272e4dc208 and sha256:1512682275109406d73565ec11b8bd3aeaf53bfac83e8e12202a3effd0ae8ea4 would not be vouched
• sha256:XXX (the manifest list) would not be vouched

All of the above would be available to pull, but only sha256:fbb75455e35366f89fe310e88420a1a35a15ee54bc86aeef36fd561de68ce5ae could be deployed to environments with binary authorization enforced.

[thepwagner]

A similar change will be required for any systems that resolve these images to vouched digests at deploy time. Until that happens, these images will be vouch-able but not deploy-able.

This part is a lie. From Image Manifest V2 Schema 2 (emphasis mine):

If the manifest being requested uses the new format, and the appropriate media type is not present in an Accept header, the registry will assume that the client cannot handle the manifest as-is, and rewrite it on the fly into the old format. If the object that would otherwise be returned is a manifest list, the registry will look up the appropriate manifest for the amd64 platform and linux OS, rewrite that manifest into the old format if necessary, and return the result to the client. If no suitable manifest is found in the manifest list, the registry will return a 404 error.

The aforementioned systems that resolve images to digests will rely on this functionality and work without changes.

Voucher can't use this functionality, as voucher operates on the canonical (aka digested) url. In the instance of grafeas#47 - the canonical URL is a manifest list.
Querying for the manifest by digest means the registry is unable to return linux/amd64 manifest on the fly - it won't have the expected digest.

This reaffirms the viable approaches:

• This PR, which adds the on-the-fly mapping to voucher
• Having clients only request the linux/amd64 platform's manifest be vouched, not the entire manifest list - perhaps via "pre-flight" mapping - docker manifest inspect ubuntu:impish-20211102@sha256:cc8f713078bfddfe9ace41e29eb73298f52b2c958ccacd1b376b9378e20906ef | jq -r '.manifests[] | select(.platform.architecture=="amd64") | .digest'