Add support for Docker credential helpers

[ldx]

This PR adds the ability to use existing Docker credential helpers (like docker-credential-ecr-login, docker-credential-gcr, etc.) for registry authentication. This enables Keel to leverage the same credential management tools that users already have configured in their Docker environments, instead of re-implementing registry credential helpers in Keel itself.

I used the same helper registration method as in the aws and gcr helpers. The only input the new helper needs is the name of an executable that implements the Docker credential protocol (registry name is supplied via stdin, and a command line argument get is added when executing the program). The executable name is passed in via the DOCKER_CREDENTIALS_HELPER environment variable (and can be just the name of the executable without the full path).

The main use case for me was adding Azure ACR support without hacks or implementing an ACR-specific extension, and leveraging https://github.com/chrismellard/docker-credential-acr-env instead, together with Keel polling the registry.

[ldx]

@david-garcia-garcia not sure if you're the right person, but it looks like you've been active on this repo. Any chance you can look at this PR or point me to the right person?

[david-garcia-garcia]

@ldx I took a look at the implementation. Hope you can help with some questions.

[1] How are you supposed to get the credential helper binaries into the keel image?

[2] The helper you are sharing (https://github.com/chrismellard/docker-credential-acr-env) seems to be using the identity from within the container itself. I guess to make the wiring work in Azure you'll need to setup workload identity on the keel pod which is not currently supported in the Helm chart.

I am mainly using Azure and was trying to figure out the actual setup details needed to use this to have keel authorize private registries without having to explicitly add ACR credentials to all containers (which I've been doing so that keel can see private registries when polling).

[ldx]

@ldx I took a look at the implementation. Hope you can help with some questions.

[1] How are you supposed to get the credential helper binaries into the keel image?

There are a couple of ways. What I did is to build my own image, installing the credential helper. If one wants to use the official keel image, an initcontainer with the helper, sharing the helper binary with the main container via an emptyDir volume is also an option.

[2] The helper you are sharing (https://github.com/chrismellard/docker-credential-acr-env) seems to be using the identity from within the container itself. I guess to make the wiring work in Azure you'll need to setup workload identity on the keel pod which is not currently supported in the Helm chart.

Yes, that is correct, but an MSI should also work (i.e. one can use the kubelet identity in AKS as well). But a pod workload identity is the best option IMO.

I am mainly using Azure and was trying to figure out the actual setup details needed to use this to have keel authorize private registries without having to explicitly add ACR credentials to all containers (which I've been doing so that keel can see private registries when polling).

Exactly, I did not want to deal with ACR credentials. Adding an ACR-specific token auth flow like it is implemented for AWS and GCR felt like reinventing the wheel, given that this is exactly the goal of Docker credential helpers. But I understand it's a tradeoff between managing an external dependency vs maintaining code for each managed registry, thus some people might prefer a self-container ACR extension in Keel. Personally I'm in favor of reusing existing Docker credential helpers.

[david-garcia-garcia]

With all that feedback this is what comes to my mind:

• We might just get rid of GCR and AWS custom implementations, and add to the container binaries for AWS, GCR and Azure. It is convenient that this works out of the box for major cloud providers (Aws and Azure). You can always add your own manually.
• As there is no default support in the helm chart for custom labels which are needed to support workload identity in azure, we should add something flexible in the form of "podLabels" (to use a pattern similar to what's being used now for podAnnotations).

[ldx]

With all that feedback this is what comes to my mind:

* We might just get rid of GCR and AWS custom implementations, and add to the container binaries for AWS, GCR and Azure. It is convenient that this works out of the box for major cloud providers (Aws and Azure). You can always add your own manually.

* As there is no default support in the helm chart for custom labels which are needed to support workload identity in azure, we should add something flexible in the form of "podLabels" (to use a pattern similar to what's being used now for podAnnotations).

This sounds good to me. Anything you wanted to include or change in this PR?