Merge pull request #2021 from weaveworks/release/1.12.2

Release Flux v1.12.2

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Suggest a new feature for Weave Flux
+title: ''
+labels: [blocked-needs-validation, enhancement]
+assignees: ''
+
+---
+
+**Describe the feature**
+A clear and concise description of the feature.
+
+**What would the new user story look like?**
+How would the new interaction with Flux look like? E.g.
+0. What are the prerequisites for this?
+1. User starts up Flux using the --add-frobnication flag
+2. New commits in git branch happen
+3. Deployments happen, which include frobnication
+
+Feel free to add a diagram if that helps explaining things.
+
+**Expected behavior**
+A clear and concise description of what you expect to happen.

CHANGELOG.md

@@ -1,6 +1,55 @@
 This is the changelog for the Flux daemon; the changelog for the Helm
 operator is in [./CHANGELOG-helmop.md](./CHANGELOG-helmop.md).
 
+## 1.12.2 (2019-05-08)
+
+This is a patch release.
+
+### Fixes
+
+- Fix error shadowing when parsing YAML manifests
+  [weaveworks/flux#1994][]
+- Fix 'workspace' -> 'workload' typo in deprecated controller flag
+  [weaveworks/flux#1987][] [weaveworks/flux#1996][]
+- Improve internal Kubernetes error logging, by removing the duplicate
+  timestamp and providing a full path to the Kubernetes file emitting
+  the error
+  [weaveworks/flux#2000][]
+- Improve `fluxctl` auto portforward connection error, by better
+  guiding the user about what could be wrong
+  [weaveworks/flux#2001][]
+- Ignore discovery errors for metrics resources, to prevent syncs from
+  failing when the metrics API is misconfigured
+  [weaveworks/flux#2009][]
+- Fix `(Flux)HelmRelease` cluster lookups, before this change, the
+  same resource ID would be reported for all `HelmRelease`s with e.g.
+  `fluctl list-workloads`
+  [weaveworks/flux#2018][]
+
+
+### Maintenance and documentation
+
+- Replace deprecated `--controller` flag in documentation with
+  `--workload`
+  [weaveworks/flux#1985][]
+- Update `MAINTAINERS` and include email addresses
+  [weaveworks/flux#1995][]
+
+### Thanks
+
+Thanks to @2opremio, @cdenneen, @hiddeco, @jan-schumacher, @squaremo,
+@stefanprodan for contributions.
+
+[weaveworks/flux#1985]: https://github.com/weaveworks/flux/pull/1985
+[weaveworks/flux#1987]: https://github.com/weaveworks/flux/pull/1987
+[weaveworks/flux#1994]: https://github.com/weaveworks/flux/pull/1994
+[weaveworks/flux#1995]: https://github.com/weaveworks/flux/pull/1995
+[weaveworks/flux#1996]: https://github.com/weaveworks/flux/pull/1996
+[weaveworks/flux#2000]: https://github.com/weaveworks/flux/pull/2000
+[weaveworks/flux#2001]: https://github.com/weaveworks/flux/pull/2001
+[weaveworks/flux#2009]: https://github.com/weaveworks/flux/pull/2009
+[weaveworks/flux#2018]: https://github.com/weaveworks/flux/pull/2018
+
 ## 1.12.1 (2019-04-25)
 
 This is a patch release.
@@ -29,7 +78,7 @@ This is a patch release.
   [weaveworks/flux#1932][]
 - Move images to DockerHub and have a separate pre-releases image repo
   [weaveworks/flux#1949][], [weaveworks/flux#1956][]
-- Suppoort `arm` and `arm64` builds [weaveworks/flux#1950][]
+- Support `arm` and `arm64` builds [weaveworks/flux#1950][]
 - Refactor the core image metadata fetching func
   [weaveworks/flux#1935][]
 - Update client-go to v1.11 [weaveworks/flux#1929][]

MAINTAINERS

@@ -4,7 +4,9 @@ at https://slack.weave.works/).
 
 In alphabetical order:
 
-Hidde Beydals (github: @hiddeco, slack: hidde)
-Justin Barrick (github: @justinbarrick, slack: Justin Barrick)
-Michael Bridgen (github: @squaremo, slack: mbridgen)
-Nick Cabatoff (github: @ncabatoff, slack: Nick Cabatoff)
+Alfonso Acosta, Weaveworks <fons@weave.works> (github: @2opremio, slack: fons)
+Hidde Beydals, Weaveworks <hidde@weave.works> (github: @hiddeco, slack: hidde)
+Justin Barrick, independent <justin.m.barrick@gmail.com> (github: @justinbarrick, slack: Justin Barrick)
+Michael Bridgen, Weaveworks <michael@weave.works> (github: @squaremo, slack: mbridgen)
+Nick Cabatoff, independent <nick.cabatoff@gmail.com> (github: @ncabatoff, slack: Nick Cabatoff)
+Stefan Prodan, Weaveworks <stefan@weave.works> (github: @stefanprodan, slack: stefan)

README.md

@@ -57,6 +57,7 @@ Its major features are:
 
 ### Who is using Flux in production
 
+- [APPUiO](https://appuio.ch)
 - [Control Plane](https://control-plane.io)
 - [Crowd Riff](https://crowdriff.com) 
 - [College of William & Mary](https://www.wm.edu)
@@ -67,7 +68,9 @@ Its major features are:
 - [Qordoba](https://qordoba.com)
 - [Troii](https://troii.com/)
 - [Under Armour](https://www.underarmour.com)
-- [Weave Cloud](https://cloud.weave.works)  
+- [VSHN](https://vshn.ch)
+- [Weave Cloud](https://cloud.weave.works)
+- [Babylon Health](https://www.babylonhealth.com/)
 
 If you too are using Flux in production; please submit a PR to add your organization to the list!
 

chart/flux/CHANGELOG.md

@@ -1,3 +1,10 @@
+## 0.9.1 (2019-04-29)
+
+### Improvements
+
+ - Updated Flux to `1.12.1`
+   [weaveworks/flux#1993](https://github.com/weaveworks/flux/pull/1993)
+
 ## 0.9.1 (2019-04-17)
 
 ### Improvements

chart/flux/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
-appVersion: "1.12.0"
-version: 0.9.1
+appVersion: "1.12.1"
+version: 0.9.2
 kubeVersion: ">=1.9.0-0"
 name: flux
 description: Flux is a tool that automatically ensures that the state of a cluster matches what is specified in version control

chart/flux/README.md

@@ -213,6 +213,9 @@ The following tables lists the configurable parameters of the Weave Flux chart a
 | `git.pollInterval`                                | `5m`                                                 | Period at which to poll git repo for new commits
 | `git.timeout`                                     | `20s`                                                | Duration after which git operations time out
 | `git.secretName`                                  | `None`                                               | Kubernetes secret with the SSH private key. Superceded by `helmOperator.git.secretName` if set.
+| `git.config.enabled`                              | `false`                                              | Mount `$HOME/.gitconfig` via Secret into the Flux and HelmOperator Pods, allowing for custom global Git configuration
+| `git.config.secretName`                           | `Computed`                                           | Kubernetes secret with the global Git configuration
+| `git.config.data`                                 | `None`                                               | Global Git configuration per [git-config](https://git-scm.com/docs/git-config)
 | `gpgKeys.secretName`                              | `None`                                               | Kubernetes secret with GPG keys the Flux daemon should import
 | `ssh.known_hosts`                                 | `None`                                               | The contents of an SSH `known_hosts` file, if you need to supply host key(s)
 | `registry.pollInterval`                           | `5m`                                                 | Period at which to check for updated images
@@ -273,6 +276,7 @@ The following tables lists the configurable parameters of the Weave Flux chart a
 | `helmOperator.affinity`                           | `{}`                                                 | Affinity properties for the helmOperator deployment
 | `kube.config`                                     | [See values.yaml](/chart/flux/values.yaml#L151-L165) | Override for kubectl default config in the Flux pod(s).
 | `prometheus.enabled`                              | `false`                                              | If enabled, adds prometheus annotations to Flux and helmOperator pod(s)
+| `syncGarbageCollection.enabled`                   | `false`                                              | If enabled, fluxd will delete resources that it created, but are no longer present in git (experimental, see [garbage collection](/site/garbagecollection.md))
 
 Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example:
 

chart/flux/templates/_helpers.tpl

@@ -60,3 +60,16 @@ repositories:
   username: "{{ .username | default "" }}"
 {{- end }}
 {{- end -}}
+
+{{/*
+Create the name of the Git config Secret.
+*/}}
+{{- define "git.config.secretName" -}}
+{{- if .Values.git.config.enabled }}
+    {{- if .Values.git.config.secretName -}}
+        {{ default "default" .Values.git.config.secretName }}
+    {{- else -}}
+        {{ default (printf "%s-git-config" (include "flux.fullname" .)) }}
+{{- end -}}
+{{- end }}
+{{- end }}

chart/flux/templates/deployment.yaml

@@ -41,6 +41,12 @@ spec:
           name: {{ template "flux.fullname" . }}-ssh-config
           defaultMode: 0600
       {{- end }}
+      {{- if .Values.git.config.enabled }}
+      - name: git-config
+        secret:
+          secretName: {{ include "git.config.secretName" . }}
+          defaultMode: 0400
+      {{- end }}
       - name: git-key
         secret:
           {{- if .Values.git.secretName }}
@@ -95,6 +101,12 @@ spec:
             mountPath: /root/.ssh
             readOnly: true
           {{- end }}
+          {{- if .Values.git.config.enabled }}
+          - name: git-config
+            mountPath: /root/.gitconfig
+            subPath: gitconfig
+            readOnly: true
+          {{- end }}
           - name: git-key
             mountPath: /etc/fluxd/ssh
             readOnly: true
@@ -179,6 +191,9 @@ spec:
           - --connect=wss://cloud.weave.works/api/flux
           - --token={{ .Values.token }}
           {{- end }}
+          {{- if .Values.syncGarbageCollection.enabled }}
+          - --sync-garbage-collection={{ .Values.syncGarbageCollection.enabled }}
+          {{- end }}
           {{- if .Values.additionalArgs }}
 {{ toYaml .Values.additionalArgs | indent 10 }}
           {{- end }}

chart/flux/templates/gitconfig.yaml

@@ -0,0 +1,9 @@
+{{- if .Values.git.config.enabled -}}
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ include "git.config.secretName" . }}
+type: Opaque
+data:
+  gitconfig: {{ .Values.git.config.data | b64enc }}
+{{- end -}}

chart/flux/templates/helm-operator-deployment.yaml

@@ -41,6 +41,12 @@ spec:
           name: {{ template "flux.fullname" . }}-ssh-config
           defaultMode: 0600
       {{- end }}
+      {{- if .Values.git.config.enabled }}
+      - name: git-config
+        secret:
+          secretName: {{ include "git.config.secretName" . }}
+          defaultMode: 0400
+      {{- end }}
       - name: git-key
         secret:
           {{- if .Values.helmOperator.git.secretName }}
@@ -84,6 +90,12 @@ spec:
           subPath: known_hosts
           readOnly: true
         {{- end }}
+        {{- if .Values.git.config.enabled }}
+        - name: git-config
+          mountPath: /root/.gitconfig
+          subPath: gitconfig
+          readOnly: true
+        {{- end }}
         - name: git-key
           mountPath: /etc/fluxd/ssh
           readOnly: true

chart/flux/values.yaml

@@ -7,7 +7,7 @@ replicaCount: 1
 
 image:
   repository: docker.io/weaveworks/flux
-  tag: 1.12.0
+  tag: 1.12.1
   pullPolicy: IfNotPresent
   pullSecret:
 
@@ -137,6 +137,14 @@ git:
   # add ./identity.pub as a deployment key with write access in your Git repo
   # set the secret name (flux-ssh) below
   secretName: ""
+  # Global Git configuration See https://git-scm.com/docs/git-config for more details.
+  config:
+    enabled: false
+    secretName: ""
+    data: ""
+    # data: |
+    #   [credential "https://github.com"]
+    #           username = foo
 
 registry:
   # Period at which to check for updated images
@@ -221,6 +229,9 @@ extraEnvs: []
 prometheus:
   enabled: false
 
+syncGarbageCollection:
+  enabled: false
+
 # Add your own init container or uncomment and modify the given example.
 initContainers: {}
 #   flux-init:  # <- will be used as container name

cluster/kubernetes/resource/load.go

@@ -140,7 +140,7 @@ func ParseMultidoc(multidoc []byte, source string) (map[string]KubeManifest, err
 		// NOTE: gopkg.in/yaml.v3 supports round tripping comments
 		//       by using `gopkg.in/yaml.v3.Node`.
 		var val interface{}
-		if err := decoder.Decode(&val); err != nil {
+		if err = decoder.Decode(&val); err != nil {
 			break
 		}
 		bytes, err := yaml.Marshal(val)

cluster/kubernetes/resource/load_test.go

@@ -142,6 +142,18 @@ metadata:
 	assert.Len(t, resources, 1)
 }
 
+func TestParseError(t *testing.T) {
+	doc := `---
+kind: ConfigMap
+metadata:
+	name: bigmap # contains a tab at the beginning
+`
+	buffer := bytes.NewBufferString(doc)
+
+	_, err := ParseMultidoc(buffer.Bytes(), "test")
+	assert.Error(t, err)
+}
+
 func TestParseCronJob(t *testing.T) {
 	doc := `---
 apiVersion: batch/v1beta1

cluster/kubernetes/resourcekinds.go

@@ -397,8 +397,8 @@ func (fhr *fluxHelmReleaseKind) getWorkloads(c *Cluster, namespace string) ([]wo
 	}
 
 	var workloads []workload
-	for _, f := range fluxHelmReleases.Items {
-		workloads = append(workloads, makeFluxHelmReleaseWorkload(&f))
+	for i, _ := range fluxHelmReleases.Items {
+		workloads = append(workloads, makeFluxHelmReleaseWorkload(&fluxHelmReleases.Items[i]))
 	}
 
 	return workloads, nil
@@ -459,8 +459,8 @@ func (hr *helmReleaseKind) getWorkloads(c *Cluster, namespace string) ([]workloa
 	}
 
 	var workloads []workload
-	for _, f := range helmReleases.Items {
-		workloads = append(workloads, makeHelmReleaseWorkload(&f))
+	for i, _ := range helmReleases.Items {
+		workloads = append(workloads, makeHelmReleaseWorkload(&helmReleases.Items[i]))
 	}
 
 	return workloads, nil

cluster/kubernetes/sync.go

@@ -206,6 +206,11 @@ func (c *Cluster) getAllowedResourcesBySelector(selector string) (map[string]*ku
 			return nil, err
 		}
 		for gv, e := range discErr.Groups {
+			if strings.HasSuffix(gv.Group, "metrics.k8s.io") {
+				// The Metrics API tends to be misconfigured, causing errors.
+				// We just ignore them, since it doesn't make sense to sync metrics anyways.
+				continue
+			}
 			// Tolerate empty GroupVersions due to e.g. misconfigured custom metrics
 			if e.Error() != fmt.Sprintf("Got empty response for: %v", gv) {
 				return nil, err

cluster/kubernetes/sync_test.go

@@ -266,7 +266,7 @@ func TestSyncTolerateEmptyGroupVersion(t *testing.T) {
 
 	// Add a GroupVersion without API Resources
 	fakeClient := kube.client.coreClient.(*corefake.Clientset)
-	fakeClient.Resources = append(fakeClient.Resources, &metav1.APIResourceList{GroupVersion: "custom.metrics.k8s.io/v1beta1"})
+	fakeClient.Resources = append(fakeClient.Resources, &metav1.APIResourceList{GroupVersion: "foo.bar/v1"})
 
 	// We should tolerate the error caused in the cache due to the
 	// GroupVersion being empty
@@ -278,6 +278,38 @@ func TestSyncTolerateEmptyGroupVersion(t *testing.T) {
 	assert.NoError(t, err)
 }
 
+type failingDiscoveryClient struct {
+	discovery.DiscoveryInterface
+}
+
+func (d *failingDiscoveryClient) ServerResourcesForGroupVersion(groupVersion string) (*metav1.APIResourceList, error) {
+	return nil, errors.NewServiceUnavailable("")
+}
+
+func TestSyncTolerateMetricsErrors(t *testing.T) {
+	kube, _, cancel := setup(t)
+
+	// Replace the discovery client by one returning errors when asking for resources
+	cancel()
+	crdClient := crdfake.NewSimpleClientset()
+	shutdown := make(chan struct{})
+	defer close(shutdown)
+	newDiscoveryClient := &failingDiscoveryClient{kube.client.coreClient.Discovery()}
+	kube.client.discoveryClient = MakeCachedDiscovery(newDiscoveryClient, crdClient, shutdown)
+
+	// Check that syncing results in an error for groups other than metrics
+	fakeClient := kube.client.coreClient.(*corefake.Clientset)
+	fakeClient.Resources = []*metav1.APIResourceList{{GroupVersion: "foo.bar/v1"}}
+	err := kube.Sync(cluster.SyncSet{})
+	assert.Error(t, err)
+
+	// Check that syncing doesn't result in an error for a metrics group
+	kube.client.discoveryClient.(*cachedDiscovery).CachedDiscoveryInterface.Invalidate()
+	fakeClient.Resources = []*metav1.APIResourceList{{GroupVersion: "custom.metrics.k8s.io/v1"}}
+	err = kube.Sync(cluster.SyncSet{})
+	assert.NoError(t, err)
+}
+
 func TestSync(t *testing.T) {
 	const ns1 = `---
 apiVersion: v1