Merge pull request #1 from fabriziopandini/first-implementation

first POC, try it!
fabriziopandini · Feb 12, 2022 · 8e18bbc · 8e18bbc
2 parents f3eb6c8 + 914d265
commit 8e18bbc
Show file tree

Hide file tree

Showing 30 changed files with 3,833 additions and 1 deletion.
diff --git a/.gitignore b/.gitignore
@@ -13,3 +13,18 @@
 
 # Dependency directories (remove the comment below to include it)
 # vendor/
+
+# Tmp folder wher runtime data are stored
+.tmp
+
+# macOS
+.DS_Store
+
+# files generated by editors
+.idea/
+*.iml
+.vscode/
+*.swp
+*.sublime-project
+*.sublime-workspace
+*~
diff --git a/README.md b/README.md
@@ -1,2 +1,94 @@
 # kBB-8
-playing around Cluster API bootstrap sequence
+
+This repository contains an experiment for making the [Cluster API](https://github.com/kubernetes-sigs/cluster-api/) bootstrap sequence faster and simpler.
+
+It is an early POC, but it bootstraps a minimal Kubernetes Cluster in 8s, then it bootstraps Cluster providers 
+in another 8s, so I have named it kBB-8 :smile:
+
+## Try it!
+(only on darwin/amd64 with Docker installed)
+
+Clone the project locally, open a terminal in the project folder and then run:
+
+````shell
+# Preliminary step to download all the packages with Kubernetes and Cluster API (performances can be improved!).
+$ test/prepare-packages.sh 
+...
+
+# Start kBB-8 and get a Cluster API bootstrap cluster in few seconds!
+$ go run kBB-8.go
+
+ ✓ kBB-8 started!
+ ✓ Cluster API with CABPK, KCP, CAPD, CAPI Ready!
+
+Set kubectl context to "kBB-8-bootstrap"
+You can now use your bootstrap cluster with:
+
+ kubectl cluster-info 
+
+Enjoy Cluster API with kBB-8! 😊
+````
+
+Now that your Cluster API bootstrap cluster is up (it is fast!), you can test it actually works by creating
+your first Workload Cluster; from another terminal window
+
+```sh
+# Create a Cluster Class
+$  k apply -f test/templates/clusterclass1.yaml 
+clusterclass.cluster.x-k8s.io/clusterclass1 created
+dockerclustertemplate.infrastructure.cluster.x-k8s.io/clusterclass1-infrastructure-cluster-template created
+kubeadmcontrolplanetemplate.controlplane.cluster.x-k8s.io/clusterclass1-controlplane-template created
+dockermachinetemplate.infrastructure.cluster.x-k8s.io/clusterclass1-controlplane-machinetemplate created
+kubeadmconfigtemplate.bootstrap.cluster.x-k8s.io/clusterclass1-md-class-1-bootstraptemplate created
+dockermachinetemplate.infrastructure.cluster.x-k8s.io/clusterclass1-md-class-1-machinetemplate created
+
+# Create a ClusterResourceSet so a CNI will be automatically applied to new clusters
+$  k apply -f test/templates/crs.yaml 
+configmap/kindnet created
+clusterresourceset.addons.cluster.x-k8s.io/cni created
+
+# Create the first cluster
+$  k apply -f test/templates/cluster1.yaml 
+cluster.cluster.x-k8s.io/my-cluster1 created
+```
+After the last command the Workload Cluster provisioning starts, and given that we are using CAPD it creates a Kubernetes 
+Cluster running in docker containers on your local machine:a
+
+
+```sh
+# Wait for machines in the the new Cluster to be provisioned, it takes ~1m (1 control-plane, 1 worker)
+$ watch kubectl get machines
+Every 2.0s: kubectl get machines                                                                                                                                                                                                                                                                                             fpandini-a01.vmware.com: Sat Feb 12 16:47:17 2022
+
+NAME                                     CLUSTER       NODENAME                                 PROVIDERID                                          PHASE     AGE     VERSION
+my-cluster1-kn22x-ghp8r                  my-cluster1   my-cluster1-kn22x-ghp8r                  docker:////my-cluster1-kn22x-ghp8r                  Running   2m47s   v1.21.2
+my-cluster1-md1-spr2f-78686b44bd-rltbv   my-cluster1   my-cluster1-md1-spr2f-78686b44bd-rltbv   docker:////my-cluster1-md1-spr2f-78686b44bd-rltbv   Running   2m50s   v1.21.2
+
+# After machine has been provisioned, you can check the containers hosting the CAPD machines actually exists.
+$ docker ps
+CONTAINER ID   IMAGE                                COMMAND                  CREATED              STATUS              PORTS                                  NAMES
+5797f7e6e79c   kindest/node:v1.21.2                 "/usr/local/bin/entr…"   About a minute ago   Up About a minute                                          my-cluster1-md1-spr2f-78686b44bd-rltbv
+8ca99305fcbb   kindest/node:v1.21.2                 "/usr/local/bin/entr…"   2 minutes ago        Up 2 minutes        49364/tcp, 127.0.0.1:49364->6443/tcp   my-cluster1-kn22x-ghp8r
+d1266dc93a98   kindest/haproxy:v20210715-a6da3463   "haproxy -sf 7 -W -d…"   3 minutes ago        Up 2 minutes        49359/tcp, 0.0.0.0:49359->6443/tcp     my-cluster1-lb
+```
+
+## How kBB-8 it works
+
+1. it downloads bootstrap packages from Cluster API/providers (not implemented yet, Cluster API/providers are not building those artifacts so we are using a local copy fetched form a GCS bucket :stuck_out_tongue_winking_eye:).
+2. it creates CAs & Certificates, it runs API server and etcd, it runs the providers as out-of-cluster processes controllers and then connects everything to get a minimal Cluster API bootstrap environment.
+
+Important!:
+- kBB-8 mimics the [kind](https://github.com/kubernetes-sigs/kind) CLI, but the intent is to move it into clusterctl (it is not a kind replacement).
+- kBB-8 is heavily inspired by [controller-runtime](https://github.com/kubernetes-sigs/controller-runtime) 's [envtest](https://github.com/kubernetes-sigs/controller-runtime/tree/v0.11.0/pkg/envtest), All the credits to the awesome contributors who created this code :heart: :pray:  :rocket: :rainbow:
+- kBB-8 does not create a compliant/fully working Kubernetes cluster e.g. no scheduler, no controller manager also, there is no cert-manager;
+  there is only the Kubernetes bits required to run Cluster API components out of cluster.
+- the prototype works, you can create your first workload cluster, but there is still a lot to do (e.g pivot, idempotence etc)"
+
+## Cleanup
+
+You can stop kBB-8 with CTRL+c, and cleanup all the docker containers with:
+
+```shell
+$ docker ps | grep my-cluster1- | awk '{ print $1; }' | xargs docker rm -f
+```
+
diff --git a/go.mod b/go.mod
@@ -0,0 +1,65 @@
+module github.com/fabriziopandini/kBB-8
+
+go 1.17
+
+require (
+	github.com/briandowns/spinner v1.18.1
+	github.com/onsi/ginkgo v1.16.5
+	github.com/onsi/gomega v1.18.1
+	golang.org/x/sys v0.0.0-20220209214540-3681064d5158
+	k8s.io/api v0.23.0
+	k8s.io/apiextensions-apiserver v0.23.0
+	k8s.io/apimachinery v0.23.0
+	k8s.io/client-go v0.23.0
+	k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b
+	sigs.k8s.io/controller-runtime v0.11.0
+	sigs.k8s.io/yaml v1.3.0
+)
+
+require (
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cespare/xxhash/v2 v2.1.1 // indirect
+	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/evanphx/json-patch v4.12.0+incompatible // indirect
+	github.com/fatih/color v1.7.0 // indirect
+	github.com/fsnotify/fsnotify v1.5.1 // indirect
+	github.com/go-logr/logr v1.2.0 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
+	github.com/golang/protobuf v1.5.2 // indirect
+	github.com/google/go-cmp v0.5.5 // indirect
+	github.com/google/gofuzz v1.1.0 // indirect
+	github.com/google/uuid v1.1.2 // indirect
+	github.com/googleapis/gnostic v0.5.5 // indirect
+	github.com/imdario/mergo v0.3.12 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/mattn/go-colorable v0.1.2 // indirect
+	github.com/mattn/go-isatty v0.0.8 // indirect
+	github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
+	github.com/nxadm/tail v1.4.8 // indirect
+	github.com/pkg/errors v0.9.1 // indirect
+	github.com/prometheus/client_golang v1.11.0 // indirect
+	github.com/prometheus/client_model v0.2.0 // indirect
+	github.com/prometheus/common v0.28.0 // indirect
+	github.com/prometheus/procfs v0.6.0 // indirect
+	github.com/spf13/pflag v1.0.5 // indirect
+	golang.org/x/net v0.0.0-20210825183410-e898025ed96a // indirect
+	golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f // indirect
+	golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b // indirect
+	golang.org/x/text v0.3.7 // indirect
+	golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect
+	gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
+	google.golang.org/appengine v1.6.7 // indirect
+	google.golang.org/protobuf v1.27.1 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
+	k8s.io/component-base v0.23.0 // indirect
+	k8s.io/klog/v2 v2.30.0 // indirect
+	k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 // indirect
+	sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.2.0 // indirect
+)