KRO fails to generate ResourceGraphDefinition when OpenAPI Schema type is not set

[jknutson]

Description

I am encountering an error when creating a ResourceGraphDefinition that includes a CRD with openAPIV3Schema fields that do not have type set.

Initially I created a bug on the FluxCD Helm Controller; the Flux maintainers recommended that KRO handle x-kubernetes-preserve-unknown-fields

Observed Behavior:

ResourceGraphDefinition does not Sync successfully with reason: Faulty Graph

Expected Behavior:

RGD syncs succesfully

Reproduction Steps (Please include ResourceGraphDefinition and Instances files):

ResourceGraphDefinition:

---
apiVersion: kro.run/v1alpha1
kind: ResourceGraphDefinition
metadata:
  name: gharunnerscaleset
spec:
  schema:
    apiVersion: v1alpha1
    kind: GhaRunnerScaleSet
    spec:
      name: string | default="example-gha-runner-scale-set"
      image: string | default="ghcr.io/actions/actions-runner:0.2.1"
      chartSourceName: string | default="ghcr-arc-charts"
      fluxNamespace: string | default="cluster-config"
      interval: string | default="1h0m0s"  # increase to e.g. 1h0m0s when not testing
      githubConfigUrl: string | default="https://github.com/MY_ORG/MY_REPO"
      githubConfigSecret: string | default="github-token"
      workloadClientId: string | default="<YOUR_WORKLOAD_CLIENT_ID>"
      nodeSelector: 'map[string]string | default={"MySelectorKey": "MySelectorValue"}'
      azAppConfigEndpoint: string | default="https://myappconfig.azconfig.io"
      azAppConfigLabelFilter: string | default="kubernetes" description="fetch values matching this label filter"

  # Define the resources this API will manage.
  resources:
    - id: resourceNamespace
      template:
        apiVersion: v1
        kind: Namespace
        metadata:
          name: ${schema.spec.name}

    - id: serviceAccount
      template:
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: ${schema.spec.name}-workload-id
          namespace: ${schema.spec.name}
          annotations:
            azure.workload.identity/client-id: ${schema.spec.workloadClientId}

    - id: azAppConfigProvider
      template:
        apiVersion: azconfig.io/v1
        kind: AzureAppConfigurationProvider
        metadata:
          name: ${schema.spec.name}
          namespace: ${schema.spec.name}
        spec:
          endpoint: ${schema.spec.azAppConfigEndpoint}
          target:
            configMapName: ${schema.spec.name}-az-app-config
          auth:
            workloadIdentity:
              serviceAccountName: ${serviceAccount.metadata.name}
          secret:
            target:
              secretName: ${schema.spec.name}-az-app-config
            auth:
              workloadIdentity:
                serviceAccountName: ${serviceAccount.metadata.name}
          configuration:
            selectors:
              - keyFilter: github_token
                labelFilter: ${schema.spec.azAppConfigLabelFilter}

    # this may fail due to the openapi schema for this CRD not having the `type` set for some properties:
    # `values`, `kustomize`, `postRenderers`, `patchesStrategicMerge`
    # if this occurs, you can manually edit the CRD to have `type: object` for each offending property
    - id: helmRelease
      template:
        apiVersion: helm.toolkit.fluxcd.io/v2
        kind: HelmRelease
        metadata:
          name: ${schema.spec.name}
          namespace: ${schema.spec.fluxNamespace}
        spec:
          targetNamespace: ${schema.spec.name}
          releaseName: ${schema.spec.name}
          chart:
            spec:
              chart: gha-runner-scale-set
              sourceRef:
                kind: HelmRepository
                name: ${schema.spec.chartSourceName}
                namespace: ${schema.spec.fluxNamespace}
          interval: "2m0s"
          install:
            remediation:
              retries: 3
          # https://github.com/actions/actions-runner-controller/tree/master/charts/gha-runner-scale-set/values.yaml
          values:
            githubConfigUrl: ${schema.spec.githubConfigUrl}
            githubConfigSecret: ${azAppConfigProvider.spec.secret.target.secretName}
            listenerTemplate:
              spec:
                nodeSelector: ${schema.spec.nodeSelector}
                containers:
                  - name: listener
            template:
              metadata:
                labels:
                  azure.workload.identity/use: "true"
              spec:
                nodeSelector: ${schema.spec.nodeSelector}
                serviceAccountName: ${serviceAccount.metadata.name}
                initContainers:
                  - name: init-dind-externals
                    image: ${schema.spec.image}
                    command: ["cp", "-r", "-v", "/home/runner/externals/.", "/home/runner/tmpDir/"]
                    volumeMounts:
                      - name: dind-externals
                        mountPath: /home/runner/tmpDir
                containers:
                  - name: runner
                    image: ${schema.spec.image}
                    command: ["/home/runner/run.sh"]
                    env:
                      - name: DOCKER_HOST
                        value: unix:///var/run/docker.sock
                    volumeMounts:
                      - name: work
                        mountPath: /home/runner/_work
                      - name: dind-sock
                        mountPath: /var/run
                  - name: dind
                    image: docker:dind
                    args:
                      - dockerd
                      - --host=unix:///var/run/docker.sock
                      - --group=$(DOCKER_GROUP_GID)
                    env:
                      - name: DOCKER_GROUP_GID
                        value: "123"
                    securityContext:
                      privileged: true
                    volumeMounts:
                      - name: work
                        mountPath: /home/runner/_work
                      - name: dind-sock
                        mountPath: /var/run
                      - name: dind-externals
                        mountPath: /home/runner/externals
                volumes:
                  - name: work
                    emptyDir: {}
                  - name: dind-sock
                    emptyDir: {}
                  - name: dind-externals
                    emptyDir: {}

No instance yaml needed, the error is on the RGD itself, not an instance of the RGD.

Versions:

• kro version: 0.2.1
• Kubernetes Version (kubectl version):

Client Version: v1.32.0
Kustomize Version: v5.5.0
Server Version: v1.31.5

Involved Controllers:

• Controller URLs and Versions (if applicable):

Error Logs (if applicable)**:

failed to build resource "helmRelease": failed to generate dummy CR for resource helmRelease: error generating field spec: error generating field values: schema type is empty and has no properties

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Which option describes the most your issue?

No response

[a-hilaly]

Indeed it looks like we need to handle x-kubernetes-preserve-unknwon for the resource emulator https://github.com/kro-run/kro/blob/main/pkg/graph/emulator/emulator.go#L103 - for now i'd say we can skip it

[jknutson]

@a-hilaly thanks for the quick reply!

I am interested in working on this issue, I think I generally understand what needs to happen.

I'll post in the slack if/when I have questions or something worth a cursory glance.

[jknutson]

I've got a PR ready, I welcome any feedback: #325

I will cross-post to slack as well.