Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy mise as a frontend subchart #1424

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Deploy mise as a frontend subchart #1424

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
633d69e
deploy mise as a dependent subchart of frontend
tony-schndr Feb 19, 2025
114ba33
istio shared cm per version and update svc cluster rollout order
tony-schndr Feb 28, 2025
9e3655d
only allow ingress traffic when mise is deployed
tony-schndr Feb 28, 2025
69cd5a1
fix schema / variable mapping
tony-schndr Mar 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .yamllint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ yaml-files:
ignore:
- 'acm/deploy/helm/clc-state-metrics/'
- 'acrpull/deploy/helm/acrpull/templates/deployment.yaml'
- 'frontend/deploy/helm/frontend/templates/ext-authz.authorizationpolicy.yaml'
- 'frontend/deploy/helm/frontend/templates/allow-ingress.authorizationpolicy.yaml'
- 'istio/deploy/helm/istio/templates/istio-shared-configmap.yml'

rules:
brackets: enable
Expand Down
13 changes: 13 additions & 0 deletions config/config.msft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,19 @@ defaults:
private: true
zoneRedundantMode: 'Auto'

# Mise
mise:
deploy: true
azureAdInstance: https://login.microsoftonline.com/
azureAdClientId: ""
armInstance: https://management.core.windows.net/
validAppId0: ""
validAppId1: ""
image:
registry: arohcpsvcint.azurecr.io
repository: mise
digest: sha256:ad3f7efeeb6691c25bf31d46d7b879e06093ec2ff43c05ad32b5bc5315ab96a7

# Maestro
maestro:
server:
Expand Down
30 changes: 30 additions & 0 deletions config/config.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -461,6 +461,36 @@
"serviceTag"
]
},
"mise":{
"properties": {
"deploy" :{
"type": "boolean"
},
"azureAdInstance":{
"type":"string"
},
"azureAdClientId":{
"type":"string"
},
"validAppId0":{
"type":"string"
},
"validAppId1":{
"type":"string"
},
"image":{
"$ref": "#/definitions/containerImage"
}
},
"required" : [
"deploy",
"image",
"azureAdInstance",
"azureAdClientId",
"validAppId0",
"validAppId1"
]
},
"global": {
"type": "object",
"properties": {
Expand Down
13 changes: 13 additions & 0 deletions config/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,19 @@ defaults:
name: frontend-cert-{{ .ctx.regionShort }}
issuer: Self

# Mise
mise:
deploy: false
azureAdInstance: ""
azureAdClientId: ""
armInstance: ""
validAppId0: ""
validAppId1: ""
image:
registry: ""
repository: ""
digest: ""

# Maestro
maestro:
server:
Expand Down
13 changes: 13 additions & 0 deletions config/public-cloud-cs-pr.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,19 @@
"miMockCertName": "msiMockCert2",
"miMockClientId": "e8723db7-9b9e-46a4-9f7d-64d75c3534f0",
"miMockPrincipalId": "d6b62dfa-87f5-49b3-bbcb-4a687c4faa96",
"mise": {
"armInstance": "",
"azureAdClientId": "",
"azureAdInstance": "",
"deploy": false,
"image": {
"digest": "",
"registry": "",
"repository": ""
},
"validAppId0": "",
"validAppId1": ""
},
"monitoring": {
"grafanaAdminGroupPrincipalId": "6b6d3adf-8476-4727-9812-20ffdef2b85c",
"grafanaName": "arohcp-dev",
Expand Down
13 changes: 13 additions & 0 deletions config/public-cloud-dev.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,19 @@
"miMockCertName": "msiMockCert2",
"miMockClientId": "e8723db7-9b9e-46a4-9f7d-64d75c3534f0",
"miMockPrincipalId": "d6b62dfa-87f5-49b3-bbcb-4a687c4faa96",
"mise": {
"armInstance": "",
"azureAdClientId": "",
"azureAdInstance": "",
"deploy": false,
"image": {
"digest": "",
"registry": "",
"repository": ""
},
"validAppId0": "",
"validAppId1": ""
},
"monitoring": {
"grafanaAdminGroupPrincipalId": "6b6d3adf-8476-4727-9812-20ffdef2b85c",
"grafanaName": "arohcp-dev",
Expand Down
13 changes: 13 additions & 0 deletions config/public-cloud-msft-int.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,19 @@
"miMockCertName": "msiMockCert2",
"miMockClientId": "e8723db7-9b9e-46a4-9f7d-64d75c3534f0",
"miMockPrincipalId": "d6b62dfa-87f5-49b3-bbcb-4a687c4faa96",
"mise": {
"armInstance": "https://management.core.windows.net/",
"azureAdClientId": "",
"azureAdInstance": "https://login.microsoftonline.com/",
"deploy": true,
"image": {
"digest": "sha256:ad3f7efeeb6691c25bf31d46d7b879e06093ec2ff43c05ad32b5bc5315ab96a7",
"registry": "arohcpsvcint.azurecr.io",
"repository": "mise"
},
"validAppId0": "",
"validAppId1": ""
},
"monitoring": {
"grafanaAdminGroupPrincipalId": "2fdb57d4-3fd3-415d-b604-1d0e37a188fe",
"grafanaName": "arohcp-int",
Expand Down
13 changes: 13 additions & 0 deletions config/public-cloud-personal-dev.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,19 @@
"miMockCertName": "msiMockCert2",
"miMockClientId": "e8723db7-9b9e-46a4-9f7d-64d75c3534f0",
"miMockPrincipalId": "d6b62dfa-87f5-49b3-bbcb-4a687c4faa96",
"mise": {
"armInstance": "",
"azureAdClientId": "",
"azureAdInstance": "",
"deploy": false,
"image": {
"digest": "",
"registry": "",
"repository": ""
},
"validAppId0": "",
"validAppId1": ""
},
"monitoring": {
"grafanaAdminGroupPrincipalId": "6b6d3adf-8476-4727-9812-20ffdef2b85c",
"grafanaName": "arohcp-dev",
Expand Down
20 changes: 10 additions & 10 deletions dev-infrastructure/svc-pipeline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,16 @@ resourceGroups:
subscription: {{ .svc.subscription }}
aksCluster: {{ .svc.aks.name }}
steps:
# configure istio
- name: istio-config
action: Shell
command: make -C ../istio deploy
dryRun:
variables:
- name: DRY_RUN
value: "true"
dependsOn:
- istio-upgrade
# - updates workload to use istio on version svc.istio.targetVersion
# - configures istio IP tag usage
- name: istio-upgrade
Expand All @@ -120,16 +130,6 @@ resourceGroups:
configRef: svc.rg
dependsOn:
- svc
# configure istio
- name: istio-config
action: Shell
command: make -C ../istio deploy
dryRun:
variables:
- name: DRY_RUN
value: "true"
dependsOn:
- istio-upgrade
# Install ACRpull
- name: acrpull
action: Shell
Expand Down
13 changes: 13 additions & 0 deletions frontend/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ deploy:
DB_URL=$$(az cosmosdb show -n ${DB_NAME} -g ${RESOURCEGROUP} --query documentEndpoint -o tsv) && \
kubectl create namespace aro-hcp --dry-run=client -o json | kubectl apply -f - && \
kubectl label namespace aro-hcp "istio.io/rev=${ISTO_TAG}" --overwrite=true && \
kubectl create namespace mise --dry-run=client -o json | kubectl apply -f - && \
kubectl label namespace mise "istio.io/rev=${ISTO_TAG}" --overwrite=true && \
${HELM_CMD} aro-hcp-frontend-dev \
deploy/helm/frontend/ \
--set azure.clientId=$${SECRET_STORE_MI_CLIENT_ID} \
Expand All @@ -85,6 +87,17 @@ deploy:
--set pullBinding.scope=repository:${ARO_HCP_IMAGE_REPOSITORY}:pull \
--set clusterService.namespace=${CS_NAMESPACE} \
--set clusterService.serviceAccount=${CS_SERVICE_ACCOUNT_NAME} \
--set deployMise=${DEPLOY_MISE} \
--set mise.namespace=mise \
--set mise.imageRegistry=${MISE_IMAGE_REGISTRY} \
--set mise.imageRepository=${MISE_IMAGE_REPOSITORY} \
--set mise.imageDigest=${MISE_IMAGE_DIGEST} \
--set mise.tenantId=$${TENANT_ID} \
--set mise.azureAdInstance=${MISE_AZURE_AD_INSTANCE} \
--set mise.azureAdClientId=${MISE_AZURE_AD_CLIENT_ID} \
--set mise.armInstance=${MISE_ARM_INSTANCE} \
--set mise.validAppId0=${MISE_VALID_APP_ID_0} \
--set mise.validAppId1=${MISE_VALID_APP_ID_1} \
--namespace aro-hcp
.PHONY: deploy

Expand Down
6 changes: 6 additions & 0 deletions frontend/deploy/helm/frontend/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,9 @@ type: application

version: 0.1.0
appVersion: "1.0.0"

dependencies:
- name: mise
version: 0.1.0
repository: "file://charts/mise"
condition: deployMise
7 changes: 7 additions & 0 deletions frontend/deploy/helm/frontend/charts/mise/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v2
name: mise
description: A Helm chart for mise
type: application

version: 0.1.0
appVersion: "1.0.0"
51 changes: 51 additions & 0 deletions frontend/deploy/helm/frontend/charts/mise/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: mise
namespace: {{ .Values.namespace }}
spec:
replicas: 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: probably best to expose this in values.yml

selector:
Copy link
Collaborator

@SudoBrendan SudoBrendan Feb 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it's a wider-reaching issue across our repo, and you're following the convention we have here, so I wouldn't block merge - but we should really really consider using the Helm standard Selectors/Labels for multiple Helm lifecycling reasons. Instead of writing these up ourselves, we can simply inject ChartName.selectorLabels. If you do a raw helm init <chartName> you'll see a mature implementation in their default scaffolding to follow, but here's the jist:

On the Deployment (and any resource in the chart) itself, that looks like:

kind: Deployment
metadata:
  labels:
    {{- include "mise.labels" . | nindent 4 }}

Here in the Pod selector, we use:

  selector:
    matchLabels:
      {{- include "mise.selectorLabels" . | nindent 6 }}

and below in the Pod template, it would be:

  template:
    metadata:
      labels:
        {{- include "mise.labels" . | nindent 8 }}

As I understand, properly labeling our resources in a way Helm understands allows us to do full lifecyling (including delete) on resources in our charts. Without standard Labels for Helm to select on, it's impossible for Helm to understand what resources are part of a chart inside a cluster, so it can't find them after initial installation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I observed Helm adds their own labels to keep track what is managed by helm for lifecycling (including deletion)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevertheless consistent labels are nice. We should decide on a scheme and implement it across our repo. I would not block merging though

matchLabels:
app: mise
template:
metadata:
labels:
app: mise
spec:
containers:
- name: mise
image: "{{ .Values.imageRegistry }}/{{ .Values.imageRepository }}@{{ .Values.imageDigest }}"
ports:
- containerPort: 8080
livenessProbe:
httpGet:
path: /healthz
port: 8080
readinessProbe:
httpGet:
path: /readyz
port: 8080
env:
- name: AzureAd__Instance
value: {{ .Values.azureAdInstance }}
- name: AzureAd__ClientId
value: {{ .Values.azureAdClientId }}
- name: AzureAd__TenantId
value: {{ .Values.tenantId }}
- name: AzureAd__InboundPolicies__0__Label
value: "ARM Policy"
- name: AzureAd__InboundPolicies__0__Authority
value: "{{ .Values.azureAdInstance }}{{ .Values.tenantId }}"
- name: AzureAd__InboundPolicies__0__AuthenticationSchemes__0
value: "Bearer"
- name: AzureAd__InboundPolicies__0__ValidAudiences__0
value: {{ .Values.armInstance }}
- name: AzureAd__InboundPolicies__0__ValidApplicationIds__0
value: {{ .Values.validAppId0 }}
- name: AzureAd__InboundPolicies__0__ValidApplicationIds__1
value: {{ .Values.validAppId1 }}
- name: AllowedHosts
value: "*"
- name: Kestrel__Endpoints__Http__Url
value: "http://0.0.0.0:8080"
12 changes: 12 additions & 0 deletions frontend/deploy/helm/frontend/charts/mise/templates/service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Service
metadata:
name: mise
namespace: {{ .Values.namespace }}
spec:
selector:
app: mise
ports:
- protocol: TCP
port: 8080
targetPort: 8080
9 changes: 9 additions & 0 deletions frontend/deploy/helm/frontend/charts/mise/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
imageRegistry: ""
imageRepository: ""
imageDigest: ""
tenantId: ""
adInstance: ""
armInstance: ""
validAppId0: ""
validAppId1: ""
namespace: ""
1 change: 1 addition & 0 deletions frontend/deploy/helm/frontend/templates/acrpullbinding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apiVersion: acrpull.microsoft.com/v1beta2
kind: AcrPullBinding
metadata:
name: pull-binding
namespace: {{ .Release.namespace }}
spec:
acr:
environment: PublicCloud
Expand Down
6 changes: 4 additions & 2 deletions frontend/deploy/helm/frontend/templates/allow-ingress.authorizationpolicy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
{{- if eq .Values.deployMise true }}
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: allow-istio-ingress
namespace: aro-hcp
namespace: {{ .Release.namespace }}
spec:
action: ALLOW
rules:
Expand All @@ -11,6 +12,7 @@ spec:
namespaces: ["aks-istio-ingress"]
to:
- operation:
methods: ["GET"]
methods: ["GET", "PUT", "POST","PATCH", "DELETE"]
ports:
- "8443"
{{- end }}
1 change: 1 addition & 0 deletions frontend/deploy/helm/frontend/templates/allow-metrics.authorizationpolicy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: allow-metrics-frontend
namespace: {{ .Release.namespace }}
spec:
action: "ALLOW"
rules:
Expand Down
1 change: 1 addition & 0 deletions frontend/deploy/helm/frontend/templates/authorizationpolicy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
name: allow-nothing
namespace: {{ .Release.namespace }}
spec: {}
15 changes: 15 additions & 0 deletions frontend/deploy/helm/frontend/templates/ext-authz.authorizationpolicy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{{- if eq .Values.deployMise true }}
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: ext-authz
namespace: {{ .Release.namespace }}
spec:
action: CUSTOM
provider:
name: ext-authz
rules:
- to:
- operation:
paths: ["/*"]
{{- end }}
1 change: 1 addition & 0 deletions frontend/deploy/helm/frontend/templates/frontend.configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apiVersion: v1
kind: ConfigMap
metadata:
name: frontend-config
namespace: {{ .Release.namespace }}
data:
DB_NAME: '{{ .Values.configMap.databaseName }}'
DB_URL: '{{ .Values.configMap.databaseUrl }}'
Expand Down
1 change: 1 addition & 0 deletions frontend/deploy/helm/frontend/templates/frontend.deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ metadata:
labels:
app: aro-hcp-frontend
name: aro-hcp-frontend
namespace: {{ .Release.namespace }}
spec:
progressDeadlineSeconds: 600
replicas: 2
Expand Down
Loading