Deployment on Alibaba Cloud

• Cluster creation
  • Update nodes to support swap accounting, namespaces and a newer kernel
• Install helm
• Install SCF and UAA
  • Install UAA
  • Install SCF

Cluster creation

IMPORTANT: You need to create a "Managed Kubernetes" cluster.

Update nodes to support swap accounting, namespaces and a newer kernel

You need to run the following commands on all nodes of your cluster.

# Enable swap accounting and user namespaces
grubby --update-kernel=ALL --args='swapaccount=1'
grubby --update-kernel=ALL --args='user_namespace.enable=1'
grubby --update-kernel=ALL --args='namespace.unpriv_enable=1'

# Update all packages
yum -y update

# Install the latest supported kernel
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
yum --enablerepo=elrepo-kernel -y install kernel-ml

# The next line sets the default kernel to be the first in the list
# that you'd see by running the commented line below:
#   awk -F\' '$1=="menuentry " {print $2}' /etc/grub2.cfg
grub2-set-default 0

# Persist grub changes
grub2-mkconfig -o /boot/grub2/grub.cfg

# Reboot so all changes take effect
reboot

Install helm

Once all nodes are up and running, you can setup Helm's tiller.

Save the following to a file named helm-sa.yaml.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: helm
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: helm
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: helm
    namespace: kube-system

Then, create the service account and install helm:

kubectl create -f helm-sa.yaml
helm init --service-account helm

Install SCF and UAA

Add the SUSE helm repo.

helm repo add suse https://kubernetes-charts.suse.com/

Install UAA

Use a values.yaml file similar to this:

secrets:
    # Create a password for your CAP cluster
    CLUSTER_ADMIN_PASSWORD: changeme

    # Create a password for your UAA client secret
    UAA_ADMIN_CLIENT_SECRET: changeme

env:
    # Enter the domain you created for your CAP cluster
    DOMAIN: MYDOMAIN.FOO

    # UAA host and port
    UAA_HOST: uaa.MYDOMAIN.FOO
    UAA_PORT: 2793
    GARDEN_ROOTFS_DRIVER: overlay-xfs
    GARDEN_APPARMOR_PROFILE: ""
kube:
    # The IP address assigned to the kube node pointed to by the domain.
    external_ips: []

    # Run kubectl get storageclasses
    # to view your available storage classes
    storage_class:
        persistent: "alicloud-disk-ssd"
        shared: "shared"
    # The registry the images will be fetched from. No values below should work for
    # a default installation of opensuse based scf containers from dockerhub. If you
    # are going to deploy sle based cap containers, comment out the next five lines.
#    registry:
#      hostname: "registry.suse.com"
#      username: ""
#      password: ""
#    organization: "cap"
    auth: rbac

sizing:
  cc_uploader:
    capabilities: ["ALL"]
  nats:
    capabilities: ["ALL"]
  routing_api:
    capabilities: ["ALL"]
  router:
    capabilities: ["ALL"]
    count: 1
  diego_locket:
    capabilities: ["ALL"]
  diego_access:
    capabilities: ["ALL"]
  diego_brain:
    capabilities: ["ALL"]
  diego_api:
    capabilities: ["ALL"]
  diego_cell:
    count: 1
    disk_sizes:
      grootfs_data: 50
  blobstore:
    disk_sizes:
      blobstore_data: 50
  api:
    count: 1
  mysql:
    disk_sizes:
      mysql_data: 30
  postgres:
    disk_sizes:
      postgres_data: 30
  autoscaler_postgres:
    disk_sizes:
      postgres_data: 30

services:
  loadbalanced: true

• Install the uaa chart

helm install suse/uaa \
    --namespace uaa \
    --values values.yaml \
    --name uaa

• Wait for it to be ready
• Use kubectl get service -n uaa uaa-uaa-public to find the public IP of the load balancer
• Point uaa.YOURDOMAIN.COM and *.uaa.YOURDOMAIN.COM to this IP

Install SCF

• In your terminal, grab UAA's CA certificate:

SECRET=$(kubectl get pods --namespace uaa -o jsonpath='{.items[?(.metadata.name=="uaa-0")].spec.containers[?(.name=="uaa")].env[?(.name=="INTERNAL_CA_CERT")].valueFrom.secretKeyRef.name}')

CA_CERT="$(kubectl get secret $SECRET --namespace uaa -o jsonpath="{.data['internal-ca-cert']}" | base64 --decode -)"

• Install the cf chart

helm install suse/cf \
    --namespace cf \
    --values values.yaml \
    --set "secrets.UAA_CA_CERT=${CA_CERT}"
    --name cf

• Point your domain to the External IP addresses of the following kube services:

*.YOURDOMAIN.COM        -> router-gorouter-public
tcp.YOURDOMAIN.COM      -> tcp-router-tcp-router-public
registry.YOURDOMAIN.COM -> eirini-opi-public