Eirini

Experimental SCF Eirini on GKE (with load balancers)

1. Get a cluster and configure your gcloud and kubectl CLI, and make sure you have jq installed.

2. Install helm. Keep in mind that RBAC is a thing, and init with an appropriate service account.

3. Install UAA.

• Write a values.yaml file that looks like this:

  env:
    DOMAIN: YOURDOMAIN.COM
    EIRINI_REGISTRY_ADDRESS: registry.YOURDOMAIN.COM:5800
    UAA_HOST: uaa.YOURDOMAIN.COM
    UAA_PORT: 2793
    EIRINI_KUBE_CONFIG: 'JSON OF YOUR KUBECONFIG'
  
  secrets:
    CLUSTER_ADMIN_PASSWORD: changeme
    UAA_ADMIN_CLIENT_SECRET: uaa-admin-client-secret
  
  services:
    loadbalanced: true
  
  kube:
    storage_class:
      persistent: "standard"
      shared: "standard"
  
  sizing:
    cc_uploader:
      capabilities: ["ALL"]
    nats:
      capabilities: ["ALL"]
    routing_api:
      capabilities: ["ALL"]
    router:
      capabilities: ["ALL"]
    locket:
      capabilities: ["ALL"]

  To get a JSON of your kube config, you can run:

  kubectl config view --flatten -o json | jq . -c -M
  

• Install it

  helm upgrade --install uaa https://s3.amazonaws.com/cap-experiments/uaa-eirini.tgz \
      --namespace uaa \
      --values values.yaml \
      --wait
  

• Wait for it to be ready

• Use kubectl get service -n uaa uaa-uaa-public to find the public IP of the load balancer

• Point uaa.YOURDOMAIN.COM and *.uaa.YOURDOMAIN.COM to this IP

4. Install SCF.

• In your terminal, grab UAA's CA certificate:

  SECRET=$(kubectl get pods --namespace uaa -o jsonpath='{.items[?(.metadata.name=="uaa-0")].spec.containers[?(.name=="uaa")].env[?(.name=="INTERNAL_CA_CERT")].valueFrom.secretKeyRef.name}')
  
  CA_CERT="$(kubectl get secret $SECRET --namespace uaa -o jsonpath="{.data['internal-ca-cert']}" | base64 --decode -)"
  

• Install SCF

  helm upgrade --install cf https://s3.amazonaws.com/cap-experiments/scf-eirini.tgz \
      --namespace cf \
      --values values.yaml \
      --wait \
      --set "secrets.UAA_CA_CERT=${CA_CERT}"
  

• Point your domain to the External IP addresses of the following kube services:

  *.YOURDOMAIN.COM        -> router-gorouter-public
  tcp.YOURDOMAIN.COM      -> tcp-router-tcp-router-public
  registry.YOURDOMAIN.COM -> eirini-opi-public
  

  To look up the external IPs:

  kubectl get service -n cf router-gorouter-public
  kubectl get service -n cf tcp-router-tcp-router-public
  kubectl get service -n cf eirini-opi-public
  

• Trust the CA of our cluster by running the following script (it's needed so Kube can pull images from the Eirini registry). Make sure to set the first 3 vars.

  CLUSTER_NAME="MYCLUSTER"
  CLUSTER_ZONE="us-east4-c"
  CF_NAMESPACE="cf"
  
  EIRINI_REGISTRY=$(kubectl exec -n $CF_NAMESPACE eirini-0 -- bash -c 'echo -e "$EIRINI_REGISTRY_ADDRESS"')
  EIRINI_REGISTRY_CA_CERT=$(kubectl exec -n $CF_NAMESPACE api-0 -- bash -c 'echo -e "$INTERNAL_CA_CERT"')
  DOCKER_REGISTRY_PATH=/etc/docker/certs.d/${EIRINI_REGISTRY}
  
  instance_names=$(gcloud compute instances list --filter=name~$CLUSTER_NAME --format json | jq --raw-output '.[].name')
  echo "$instance_names" | xargs -i{} gcloud compute ssh {} -- \
    "sudo mkdir -p ${DOCKER_REGISTRY_PATH}; echo '${EIRINI_REGISTRY_CA_CERT}' | sudo tee ${DOCKER_REGISTRY_PATH}/ca.crt > /dev/null;"
  

Push an app and enjoy!