Skip to content

Commit

Permalink
Added documentation for App sync decoupling feature
Browse files Browse the repository at this point in the history
Signed-off-by: anandf <anjoseph@redhat.com>
  • Loading branch information
anandf committed Apr 10, 2024
1 parent f263b64 commit 8bc9d5f
Show file tree
Hide file tree
Showing 2 changed files with 137 additions and 26 deletions.
111 changes: 111 additions & 0 deletions docs/operator-manual/app-sync-using-impersonation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Application Sync using impersonation

!!! warning
Please read this documentation carefully before you enable this feature. Misconfiguration could lead to potential security issues.

## Introduction

As of version 2.10, Argo CD supports syncing `Application` resources using the same service account used for its control plane operations. This feature enables users to decouple service account used for application sync from the service account used for control plane operations.

Application syncs in Argo CD have the same privileges as the Argo CD control plane. As a consequence, in a multi-tenant setup, the Argo CD control plane privileges needs to match the tenant that needs the highest privileges. As an example, if an Argo CD instance has 10 Applications and only one of them requires admin privileges, then the Argo CD control plane must have admin privileges in order to be able to sync that one Application. Argo CD provides a multi-tenancy model to restrict what each Application can do using `AppProjects`, even though the control plane has higher privileges, however that creates a large attack surface since if Argo CD is compromised, attackers would have cluster-admin access to the cluster.

Some manual steps will need to be performed by the Argo CD administrator in order to enable this feature.

!!! note
This feature is considered beta as of now. Some of the implementation details may change over the course of time until it is promoted to a stable status. We will be happy if early adopters use this feature and provide us with bug reports and feedback.

### What is Impersonation

Impersonation is a feature in Kubernetes and enabled in the `kubectl` CLI client, using which, a user can act as another user through impersonation headers. For example, an admin could use this feature to debug an authorization policy by temporarily impersonating another user and seeing if a request was denied.

Impersonation requests first authenticate as the requesting user, then switch to the impersonated user info.

```shell
kubectl --as <user-to-impersonate> ...
kubectl --as <user-to-impersonate> --as-group <group-to-impersonate> ...
```
## Prerequisites
- In a multi team/multi tenant environment, an application team is typically granted access to a namespace to self-manage their Applications in a declarative way.
- The tenant namespace and the service account to be used for creating the resources in that namespace is created.
- Create a Role to manage kubernetes resources in the tenant namespace
- Create a RoleBinding to map the service account to the role created in the previous step.

## Implementation details

### Overview

In order for an application to use a different service account for the application sync operation, the following steps needs to be performed:

1. The impersonation feature flag should be enabled by setting the value of key `application.impersonation.enabled` to `true` in the `argocd-cmd-params-cm` ConfigMap as below:
```yaml
data:
application.impersonation.enabled: true
```
2. The `AppProject` referenced by the `.spec.project` field of the `Application` must have the `DestinationServiceAccounts` mapping the destination server and namespace to a service account to be used for the sync operation.

`DestinationServiceAccounts` associated to a `AppProject` can be created and managed, either declaratively or through the Argo CD API (e.g. using the CLI, the web UI, the REST API, etc).


### Enable application sync with impersonation feature

In order to enable this feature, the Argo CD administrator must reconfigure the `application.impersonation.enabled` settings in the `argocd-cmd-params-cm` ConfigMap as below:

```yaml
data:
application.impersonation.enabled: true
```

## Configuring destination service accounts

### Declaratively

For declaratively configuring destination service accounts, in the `AppProject`, add a section `.spec.destinationServiceAccounts`. Specify the target destination `server` and `namespace` and the provide the service account to be used for the sync operation using `defaultServiceAccount` field. Applications that refer this `AppProject` would use the corresponding service account configured for its destination. If there are multiple matches, then the first valid match would be considered.

```yaml
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: my-project
namespace: argocd
spec:
description: Example Project
# Allow manifests to deploy from any Git repos
sourceRepos:
- '*'
destinations:
- *
destinationServiceAccounts:
- server: https://kubernetes.default.svc
namespace: guestbook
defaultServiceAccount: guestbook-deployer
- server: https://kubernetes.default.svc
namespace: guestbook-dev
defaultServiceAccount: guestbook-dev-deployer
- server: https://kubernetes.default.svc
namespace: guestbook-stage
defaultServiceAccount: guestbook-stage-deployer
- server: https://kubernetes.default.svc
namespace: '*'
defaultServiceAccount: default
```

### Using the CLI

You can use all existing Argo CD CLI commands for adding destination service account

For example, to add a destination service account for `in-cluster` and `guestbook` namespace, you can use the following CLI command:

```shell
argocd proj add-destination-service-account my-project https://kubernetes.default.svc guestbook guestbook-sa
```

Likewise, to remove the destination service account from an `AppProject`, you can use the following CLI command:

```shell
argocd proj remove-destination-service-account my-project https://kubernetes.default.svc guestbook
```

### Using the UI

Similar to the CLI, you can add destination service account when creating an `AppProject` from the UI
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Impersonation is a feature in Kubernetes and enabled in the `kubectl` CLI client

Impersonation requests first authenticate as the requesting user, then switch to the impersonated user info.

```
```shell
kubectl --as <user-to-impersonate> ...
kubectl --as <user-to-impersonate> --as-group <group-to-impersonate> ...
```
Expand Down Expand Up @@ -83,7 +83,7 @@ When applications gets synced, based on its destination (target cluster and name

We would be introducing a new element `destinationServiceAccounts` in `AppProject.spec`. This element is used for the sole purpose of specifying the impersonation configuration. The `defaultServiceAccount` configured for the `AppProject` would be used for the sync operation for a particular destination cluster and namespace. If impersonation feature is enabled and no specific service account is provided in the `AppProject` CR, then the `default` service account in the destination namespace would be used for impersonation.

```
```yaml
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
Expand Down Expand Up @@ -189,12 +189,12 @@ Set `ARGOCD_APPLICATION_CONTROLLER_ENABLE_IMPERSONATION=true` in the Application
In this specific scenario, service account name `generic-deployer` will get used for the application sync as the namespace `guestbook` matches the glob pattern `*`.

- Install ArgoCD in the `argocd` namespace.
```
```shell
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/install.yaml -n argocd
```

- Enable the impersonation feature in ArgoCD.
```
```shell
kubectl set env statefulset/argocd-application-controller ARGOCD_APPLICATION_CONTROLLER_ENABLE_IMPERSONATION=true
```

Expand All @@ -205,13 +205,13 @@ kubectl create serviceaccount guestbook-deployer
```

- Create Role and RoleBindings and configure RBAC access for creating `Service` and `Deployment` objects in namespace `guestbook` for service account `guestbook-deployer`.
```
```shell
kubectl create role guestbook-deployer-role --verb get,list,update,delete --resource pods,deployment,service
kubectl create rolebinding guestbook-deployer-rb --serviceaccount guestbook-deployer --role guestbook-deployer-role
```

- Create the `Application` in the `argocd` namespace and the required `AppProject` as below
```
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
Expand Down Expand Up @@ -253,28 +253,28 @@ spec:
In this specific scenario, service account name `guestbook-deployer` will get used for the application sync as the namespace `guestbook` matches the target namespace `guestbook`.

- Install ArgoCD in the `argocd` namespace.
```
```shell
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/install.yaml -n argocd
```

- Enable the impersonation feature in ArgoCD.
```
```shell
kubectl set env statefulset/argocd-application-controller ARGOCD_APPLICATION_CONTROLLER_ENABLE_IMPERSONATION=true
```

- Create a namespace called `guestbook` and a service account called `guestbook-deployer`.
```
```shell
kubectl create namespace guestbook
kubectl create serviceaccount guestbook-deployer
```
- Create Role and RoleBindings and configure RBAC access for creating `Service` and `Deployment` objects in namespace `guestbook` for service account `guestbook-deployer`.
```
```shell
kubectl create role guestbook-deployer-role --verb get,list,update,delete --resource pods,deployment,service
kubectl create rolebinding guestbook-deployer-rb --serviceaccount guestbook-deployer --role guestbook-deployer-role
```

In this specific scenario, service account name `guestbook-deployer` will get used as it matches to the specific namespace `guestbook`.
```
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
Expand Down Expand Up @@ -321,38 +321,38 @@ spec:
**Note**: In this example, we are relying on the default service account `argocd-manager` with `cluster-admin` privileges which gets created when adding a remote cluster destination using the ArgoCD CLI.

- Install ArgoCD in the `argocd` namespace.
```
```shell
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/install.yaml -n argocd
```

- Enable the impersonation feature in ArgoCD.
```
```shell
kubectl set env statefulset/argocd-application-controller ARGOCD_APPLICATION_CONTROLLER_ENABLE_IMPERSONATION=true
```

- Add the remote cluster as a destination to argocd
```
```shell
argocd cluster add remote-cluster --name remote-cluster
```
**Note:** The above command would create a service account named `argocd-manager` in `kube-system` namespace and `ClusterRole` named `argocd-manager-role` with full cluster admin access and a `ClusterRoleBinding` named `argocd-manager-role-binding` mapping the `argocd-manager-role` to the service account `remote-cluster`

- In the remote cluster, create a namespace called `guestbook` and a service account called `guestbook-deployer`.
```
```shell
kubectl ctx remote-cluster
kubectl create namespace guestbook
kubectl create serviceaccount guestbook-deployer
```

- In the remote cluster, create `Role` and `RoleBindings` and configure RBAC access for creating `Service` and `Deployment` objects in namespace `guestbook` for service account `guestbook-deployer`.

```
```shell
kubectl ctx remote-cluster
kubectl create role guestbook-deployer-role --verb get,list,update,delete --resource pods,deployment,service
kubectl create rolebinding guestbook-deployer-rb --serviceaccount guestbook-deployer --role guestbook-deployer-role
```

- Create the `Application` and `AppProject` for the `guestbook` application.
```
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
Expand Down Expand Up @@ -394,17 +394,17 @@ spec:
**Note**: In this example, we are relying on a non default service account `guestbook` created in the target cluster and namespace for the sync operation. This use case is for handling scenarios where the remote cluster is managed by a different administrator and providing a service account with `cluster-admin` level access is not feasible.

- Install ArgoCD in the `argocd` namespace.
```
```shell
kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/install.yaml -n argocd
```

- Enable the impersonation feature in ArgoCD.
```
```shell
kubectl set env statefulset/argocd-application-controller ARGOCD_APPLICATION_CONTROLLER_ENABLE_IMPERSONATION=true
```

- In the remote cluster, create a service account called `argocd-admin`
```
```shell
kubectl ctx remote-cluster
kubectl create serviceaccount argocd-admin
kubectl create clusterrole argocd-admin-role --verb=impersonate --resource="users,groups,serviceaccounts"
Expand All @@ -414,20 +414,20 @@ kubectl create clusterrolebinding argocd-admin-access-review-role-binding --serv
```

- In the remote cluster, create a namespace called `guestbook` and a service account called `guestbook-deployer`.
```
```shell
kubectl ctx remote-cluster
kubectl create namespace guestbook
kubectl create serviceaccount guestbook-deployer
```

- In the remote cluster, create `Role` and `RoleBindings` and configure RBAC access for creating `Service` and `Deployment` objects in namespace `guestbook` for service account `guestbook-deployer`.
```
```shell
kubectl create role guestbook-deployer-role --verb get,list,update,delete --resource pods,deployment,service
kubectl create rolebinding guestbook-deployer-rb --serviceaccount guestbook-deployer --role guestbook-deployer-role
```

In this specific scenario, service account name `guestbook-deployer` will get used as it matches to the specific namespace `guestbook`.
```
```yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
Expand Down Expand Up @@ -475,7 +475,7 @@ spec:

By default, the service account would be looked up in the Application's destination namespace configured through `Application.Spec.Destination.Namespace` field. If the service account is in a different namespace, then users can provide the namespace of the service account explicitly in the format <namespace>:<service_account_name>
eg:
```
```yaml
...
destinationServiceAccounts:
- server: https://kubernetes.default.svc
Expand All @@ -490,7 +490,7 @@ If there are multiple matches for a given destination, the first valid match in

eg:
Lets assume that the `AppProject` has the below `destinationServiceAccounts` configured.
```
```yaml
...
destinationServiceAccounts:
- server: https://kubernetes.default.svc
Expand Down Expand Up @@ -552,7 +552,7 @@ Consider the following in developing an upgrade/downgrade strategy for this enha
### Option 1
Allow all options available in the `ImpersonationConfig` available to the user through the `AppProject` CRs.

```
```yaml
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
Expand Down

0 comments on commit 8bc9d5f

Please sign in to comment.