From e7051201458c5bb9369401b94058e50744cf05cf Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 21 Jun 2019 13:26:43 +0200 Subject: [PATCH 001/158] List --registry-use-labels flag in daemon docs --- site/daemon.md | 95 +++++++++++++++++++++++++------------------------- 1 file changed, 48 insertions(+), 47 deletions(-) diff --git a/site/daemon.md b/site/daemon.md index 968b9d5c4..fdf67137d 100644 --- a/site/daemon.md +++ b/site/daemon.md @@ -42,58 +42,59 @@ document. fluxd requires setup and offers customization though a multitude of flags. -| Flag | Default | Purpose -| ------------------------------------------------ | ------------------------ | --- -| --listen -l | `:3030` | listen address where /metrics and API will be served -| --listen-metrics | | listen address for /metrics endpoint -| --kubernetes-kubectl | | optional, explicit path to kubectl tool -| --version | false | output the version number and exit +| Flag | Default | Purpose +| ------------------------------------------------ | ---------------------------------- | --- +| --listen -l | `:3030` | listen address where /metrics and API will be served +| --listen-metrics | | listen address for /metrics endpoint +| --kubernetes-kubectl | | optional, explicit path to kubectl tool +| --version | false | output the version number and exit | **Git repo & key etc.** -| --git-url | | URL of git repo with Kubernetes manifests; e.g., `git@github.com:weaveworks/flux-get-started` -| --git-branch | `master` | branch of git repo to use for Kubernetes manifests -| --git-ci-skip | false | when set, fluxd will append `\n\n[ci skip]` to its commit messages -| --git-ci-skip-message | `""` | if provided, fluxd will append this to commit messages (overrides --git-ci-skip`) -| --git-path | | path within git repo to locate Kubernetes manifests (relative path) -| --git-user | `Weave Flux` | username to use as git committer -| --git-email | `support@weave.works` | email to use as git committer -| --git-set-author | false | if set, the author of git commits will reflect the user who initiated the commit and will differ from the git committer -| --git-gpg-key-import | | if set, fluxd will attempt to import the gpg key(s) found on the given path -| --git-signing-key | | if set, commits made by fluxd to the user git repo will be signed with the provided GPG key. See [Git commit signing](git-commit-signing.md) to learn how to use this feature -| --git-label | | label to keep track of sync progress; overrides both --git-sync-tag and --git-notes-ref -| --git-sync-tag | `flux-sync` | tag to use to mark sync progress for this cluster (old config, still used if --git-label is not supplied) -| --git-notes-ref | `flux` | ref to use for keeping commit annotations in git notes -| --git-poll-interval | `5m` | period at which to fetch any new commits from the git repo -| --git-timeout | `20s` | duration after which git operations time out -| --git-secret | false | if set, git-secret will be run on every git checkout. A gpg key must be imported using --git-gpg-key-import or by mounting a keyring containing it directly +| --git-url | | URL of git repo with Kubernetes manifests; e.g., `git@github.com:weaveworks/flux-get-started` +| --git-branch | `master` | branch of git repo to use for Kubernetes manifests +| --git-ci-skip | false | when set, fluxd will append `\n\n[ci skip]` to its commit messages +| --git-ci-skip-message | `""` | if provided, fluxd will append this to commit messages (overrides --git-ci-skip`) +| --git-path | | path within git repo to locate Kubernetes manifests (relative path) +| --git-user | `Weave Flux` | username to use as git committer +| --git-email | `support@weave.works` | email to use as git committer +| --git-set-author | false | if set, the author of git commits will reflect the user who initiated the commit and will differ from the git committer +| --git-gpg-key-import | | if set, fluxd will attempt to import the gpg key(s) found on the given path +| --git-signing-key | | if set, commits made by fluxd to the user git repo will be signed with the provided GPG key. See [Git commit signing](git-commit-signing.md) to learn how to use this feature +| --git-label | | label to keep track of sync progress; overrides both --git-sync-tag and --git-notes-ref +| --git-sync-tag | `flux-sync` | tag to use to mark sync progress for this cluster (old config, still used if --git-label is not supplied) +| --git-notes-ref | `flux` | ref to use for keeping commit annotations in git notes +| --git-poll-interval | `5m` | period at which to fetch any new commits from the git repo +| --git-timeout | `20s` | duration after which git operations time out +| --git-secret | false | if set, git-secret will be run on every git checkout. A gpg key must be imported using --git-gpg-key-import or by mounting a keyring containing it directly | **syncing:** control over how config is applied to the cluster -| --sync-interval | `5m` | apply the git config to the cluster at least this often. New commits may provoke more frequent syncs -| --sync-garbage-collection | `false` | experimental: when set, fluxd will delete resources that it created, but are no longer present in git (see [garbage collection](./garbagecollection.md)) +| --sync-interval | `5m` | apply the git config to the cluster at least this often. New commits may provoke more frequent syncs +| --sync-garbage-collection | `false` | experimental: when set, fluxd will delete resources that it created, but are no longer present in git (see [garbage collection](./garbagecollection.md)) | **registry cache:** (none of these need overriding, usually) -| --memcached-hostname | `memcached` | hostname for memcached service to use for caching image metadata -| --memcached-timeout | `1s` | maximum time to wait before giving up on memcached requests -| --memcached-service | `memcached` | SRV service used to discover memcache servers -| --registry-cache-expiry | `1h` | Duration to keep cached registry tag info. Must be < 1 month. -| --registry-poll-interval | `5m` | period at which to poll registry for new images -| --registry-rps | `200` | maximum registry requests per second per host -| --registry-burst | `125` | maximum number of warmer connections to remote and memcache -| --registry-insecure-host | [] | registry hosts to use HTTP for (instead of HTTPS) -| --registry-exclude-image | `["k8s.gcr.io/*"]` | do not scan images that match these glob expressions -| --docker-config | `""` | path to a Docker config file with default image registry credentials -| --registry-ecr-region | `[]` | Allow these AWS regions when scanning images from ECR (multiple values allowed); defaults to the detected cluster region -| --registry-ecr-include-id | `[]` | Include these AWS account ID(s) when scanning images in ECR (multiple values allowed); empty means allow all, unless excluded -| --registry-ecr-exclude-id | `[]` | Exclude these AWS account ID(s) when scanning ECR (multiple values allowed); defaults to the EKS system account, so system images will not be scanned -| --registry-require | `[]` | exit with an error if the given services are not available. Useful for escalating misconfiguration or outages that might otherwise go undetected. Presently supported values: {`ecr`} | +| --memcached-hostname | `memcached` | hostname for memcached service to use for caching image metadata +| --memcached-timeout | `1s` | maximum time to wait before giving up on memcached requests +| --memcached-service | `memcached` | SRV service used to discover memcache servers +| --registry-cache-expiry | `1h` | Duration to keep cached registry tag info. Must be < 1 month. +| --registry-poll-interval | `5m` | period at which to poll registry for new images +| --registry-rps | `200` | maximum registry requests per second per host +| --registry-burst | `125` | maximum number of warmer connections to remote and memcache +| --registry-insecure-host | [] | registry hosts to use HTTP for (instead of HTTPS) +| --registry-exclude-image | `["k8s.gcr.io/*"]` | do not scan images that match these glob expressions +| --registry-use-labels | `["index.docker.io/weaveworks/*"]` | use the timestamp (RFC3339) from labels for (canonical) image refs that match these glob expressions +| --docker-config | `""` | path to a Docker config file with default image registry credentials +| --registry-ecr-region | `[]` | allow these AWS regions when scanning images from ECR (multiple values allowed); defaults to the detected cluster region +| --registry-ecr-include-id | `[]` | include these AWS account ID(s) when scanning images in ECR (multiple values allowed); empty means allow all, unless excluded +| --registry-ecr-exclude-id | `[]` | exclude these AWS account ID(s) when scanning ECR (multiple values allowed); defaults to the EKS system account, so system images will not be scanned +| --registry-require | `[]` | exit with an error if the given services are not available. Useful for escalating misconfiguration or outages that might otherwise go undetected. Presently supported values: {`ecr`} | | **k8s-secret backed ssh keyring configuration** -| --k8s-secret-name | `flux-git-deploy` | name of the k8s secret used to store the private SSH key -| --k8s-secret-volume-mount-path | `/etc/fluxd/ssh` | mount location of the k8s secret storing the private SSH key -| --k8s-secret-data-key | `identity` | data key holding the private SSH key within the k8s secret +| --k8s-secret-name | `flux-git-deploy` | name of the k8s secret used to store the private SSH key +| --k8s-secret-volume-mount-path | `/etc/fluxd/ssh` | mount location of the k8s secret storing the private SSH key +| --k8s-secret-data-key | `identity` | data key holding the private SSH key within the k8s secret | **k8s configuration** -| --k8s-allow-namespace | | experimental: restrict all operations to the provided namespaces +| --k8s-allow-namespace | | experimental: restrict all operations to the provided namespaces | **upstream service** -| --connect | | connect to an upstream service e.g., Weave Cloud, at this base address -| --token | | authentication token for upstream service +| --connect | | connect to an upstream service e.g., Weave Cloud, at this base address +| --token | | authentication token for upstream service | **SSH key generation** -| --ssh-keygen-bits | | -b argument to ssh-keygen (default unspecified) -| --ssh-keygen-type | | -t argument to ssh-keygen (default unspecified) +| --ssh-keygen-bits | | -b argument to ssh-keygen (default unspecified) +| --ssh-keygen-type | | -t argument to ssh-keygen (default unspecified) | **manifest generation** -| --manifest-generation | false | experimental; search for .flux.yaml files to generate manifests +| --manifest-generation | false | experimental; search for .flux.yaml files to generate manifests From 2a39f87ce37ef98fba95795160e4ff82aa6c501c Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 21 Jun 2019 13:30:55 +0200 Subject: [PATCH 002/158] Implement label whitelist in Helm chart --- chart/flux/README.md | 1 + chart/flux/templates/deployment.yaml | 3 +++ chart/flux/values.yaml | 2 ++ 3 files changed, 6 insertions(+) diff --git a/chart/flux/README.md b/chart/flux/README.md index 7257d1b10..94d3fa78d 100755 --- a/chart/flux/README.md +++ b/chart/flux/README.md @@ -234,6 +234,7 @@ The following tables lists the configurable parameters of the Flux chart and the | `registry.insecureHosts` | `None` | Use HTTP rather than HTTPS for the image registry domains | `registry.cacheExpiry` | `None` | Duration to keep cached image info (deprecated) | `registry.excludeImage` | `None` | Do not scan images that match these glob expressions; if empty, 'k8s.gcr.io/*' images are excluded +| `registry.useTimestampLabels` | `None` | Allow usage of (RFC3339) timestamp labels from (canonical) image refs that match these glob expressions; if empty, 'index.docker.io/weaveworks/*' images are allowed | `registry.ecr.region` | `None` | Restrict ECR scanning to these AWS regions; if empty, only the cluster's region will be scanned | `registry.ecr.includeId` | `None` | Restrict ECR scanning to these AWS account IDs; if empty, all account IDs that aren't excluded may be scanned | `registry.ecr.excludeId` | `602401143452` | Do not scan ECR for images in these AWS account IDs; the default is to exclude the EKS system account diff --git a/chart/flux/templates/deployment.yaml b/chart/flux/templates/deployment.yaml index 4000938f2..5b44f4778 100644 --- a/chart/flux/templates/deployment.yaml +++ b/chart/flux/templates/deployment.yaml @@ -199,6 +199,9 @@ spec: {{- if .Values.registry.excludeImage }} - --registry-exclude-image={{ .Values.registry.excludeImage }} {{- end }} + {{- if .Values.registry.useTimestampLabels }} + - --registry-use-labels={{ .Values.registry.useTimestampLabels }} + {{- end }} {{- if .Values.registry.ecr.region }} - --registry-ecr-region={{ .Values.registry.ecr.region }} {{- end }} diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index 9389c8dad..285c669ab 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -184,6 +184,8 @@ registry: cacheExpiry: # Do not scan images that match these glob expressions excludeImage: + # Allow usage of (RFC3339) timestamp labels from (canonical) image refs that match these glob expressions + useTimestampLabels: # AWS ECR settings ecr: region: From ec9bbdcafe73baafcc1adb799502dfccba3695e0 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 8 Jul 2019 19:12:48 +0200 Subject: [PATCH 003/158] Whitelist label ts using registry cache decorator We did not take the inheritance of labels from base images into account while developing the timestamps from image labels feature. As a result, users started experiencing issues with automated image updates, due to the (moderately old) timestamp from the base image being picked up and prioritized over the timestamp from the registry. This commits adds the ability to decorate `ImageRepository` structures before they are returned from cache by `GetImageRepositoryMetadata`. The sole decorator for now is `TimestampLabelWhitelist`, which replaces the `CreatedAt` field on the image info with the timestamp specified in one of the labels if the canonical name of the image matches one of the glob patterns on the whitelist. As a result, the labels from images will only be used if explicitly instructed by the user, dealing with the problem described at the top of this comment. --- cmd/fluxd/main.go | 4 ++ registry/cache/memcached/integration_test.go | 2 +- registry/cache/registry.go | 49 ++++++++++++- registry/cache/registry_test.go | 74 ++++++++++++++++++++ 4 files changed, 127 insertions(+), 2 deletions(-) create mode 100644 registry/cache/registry_test.go diff --git a/cmd/fluxd/main.go b/cmd/fluxd/main.go index 5d558767f..6fefd9bd5 100644 --- a/cmd/fluxd/main.go +++ b/cmd/fluxd/main.go @@ -146,6 +146,7 @@ func main() { registryTrace = fs.Bool("registry-trace", false, "output trace of image registry requests to log") registryInsecure = fs.StringSlice("registry-insecure-host", []string{}, "let these registry hosts skip TLS host verification and fall back to using HTTP instead of HTTPS; this allows man-in-the-middle attacks, so use with extreme caution") registryExcludeImage = fs.StringSlice("registry-exclude-image", []string{"k8s.gcr.io/*"}, "do not scan images that match these glob expressions; the default is to exclude the 'k8s.gcr.io/*' images") + registryUseLabels = fs.StringSlice("registry-use-labels", []string{"index.docker.io/weaveworks/*"}, "use the timestamp (RFC3339) from labels for (canonical) image refs that match these glob expression") // AWS authentication registryAWSRegions = fs.StringSlice("registry-ecr-region", nil, "include just these AWS regions when scanning images in ECR; when not supplied, the cluster's region will included if it can be detected through the AWS API") @@ -498,6 +499,9 @@ func main() { cacheRegistry = &cache.Cache{ Reader: cacheClient, + Decorators: []cache.Decorator{ + cache.TimestampLabelWhitelist(*registryUseLabels), + }, } cacheRegistry = registry.NewInstrumentedRegistry(cacheRegistry) diff --git a/registry/cache/memcached/integration_test.go b/registry/cache/memcached/integration_test.go index cfbd529ec..f25c0e82d 100644 --- a/registry/cache/memcached/integration_test.go +++ b/registry/cache/memcached/integration_test.go @@ -48,7 +48,7 @@ func TestWarming_WarmerWriteCacheRead(t *testing.T) { Trace: true, } - r := &cache.Cache{mc} + r := &cache.Cache{Reader: mc} w, _ := cache.NewWarmer(remote, mc, 125) shutdown := make(chan struct{}) diff --git a/registry/cache/registry.go b/registry/cache/registry.go index 967bae3ec..d238dc5e9 100644 --- a/registry/cache/registry.go +++ b/registry/cache/registry.go @@ -5,6 +5,7 @@ import ( "time" "github.com/pkg/errors" + "github.com/ryanuber/go-glob" fluxerr "github.com/weaveworks/flux/errors" "github.com/weaveworks/flux/image" @@ -28,7 +29,48 @@ repository. // Cache is a local cache of image metadata. type Cache struct { - Reader Reader + Reader Reader + Decorators []Decorator +} + +// Decorator is for decorating an ImageRepository before it is returned. +type Decorator interface { + apply(*ImageRepository) +} + +// TimestampLabelWhitelist contains a string slice of glob patterns. Any +// canonical image reference that matches one of the glob patterns will +// prefer creation timestamps from labels over the one it received from +// the registry. +type TimestampLabelWhitelist []string + +// apply checks if any of the canonical image references from the +// repository matches a glob pattern from the list. If it does, and the +// image record has a valid timestamp label, it will replace the Created +// field with the value from the label for all images in the repository. +func (l TimestampLabelWhitelist) apply(r *ImageRepository) { + var match bool + for k, i := range r.Images { + if !match { + for _, exp := range l { + if glob.Glob(exp, i.ID.CanonicalName().String()) { + match = true + break + } + } + if !match { + return + } + } + + switch { + case !i.Labels.Created.IsZero(): + i.CreatedAt = i.Labels.Created + case !i.Labels.BuildDate.IsZero(): + i.CreatedAt = i.Labels.BuildDate + } + r.Images[k] = i + } } // GetImageRepositoryMetadata returns the metadata from an image @@ -53,6 +95,11 @@ func (c *Cache) GetImageRepositoryMetadata(id image.Name) (image.RepositoryMetad return image.RepositoryMetadata{}, ErrNotCached } + // (Maybe) decorate the image repository. + for _, d := range c.Decorators { + d.apply(&repo) + } + return repo.RepositoryMetadata, nil } diff --git a/registry/cache/registry_test.go b/registry/cache/registry_test.go new file mode 100644 index 000000000..84cd15949 --- /dev/null +++ b/registry/cache/registry_test.go @@ -0,0 +1,74 @@ +package cache + +import ( + "encoding/json" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/weaveworks/flux/image" +) + +// mockStorage holds a fixed ImageRepository item. +type mockStorage struct { + Item ImageRepository +} + +// GetKey will always return the same item from the storage, +// and does not care about the key it receives. +func (m *mockStorage) GetKey(k Keyer) ([]byte, time.Time, error) { + b, err := json.Marshal(m.Item) + if err != nil { + return []byte{}, time.Time{}, err + } + return b, time.Time{}, nil +} + +// appendImage adds an image to the mocked storage item. +func (m *mockStorage) appendImage(i image.Info) { + tag := i.ID.Tag + + m.Item.Images[tag] = i + m.Item.Tags = append(m.Item.Tags, tag) +} + +func mockReader() *mockStorage { + return &mockStorage{ + Item: ImageRepository{ + RepositoryMetadata: image.RepositoryMetadata{ + Tags: []string{}, + Images: map[string]image.Info{}, + }, + LastUpdate: time.Now(), + }, + } +} + +func Test_WhitelabelDecorator(t *testing.T) { + r := mockReader() + + // Image with no timestamp label + r.appendImage(mustMakeInfo("docker.io/fluxcd/flux:equal", time.Time{}, time.Now().UTC())) + // Image with a timestamp label + r.appendImage(mustMakeInfo("docker.io/fluxcd/flux:label", time.Now().Add(-10*time.Second).UTC(), time.Now().UTC())) + + c := Cache{r, []Decorator{TimestampLabelWhitelist{"index.docker.io/fluxcd/*"}}} + + rm, err := c.GetImageRepositoryMetadata(image.Name{}) + assert.NoError(t, err) + + assert.Equal(t, r.Item.Images["equal"].CreatedAt, rm.Images["equal"].CreatedAt) + assert.Equal(t, r.Item.Images["label"].Labels.Created, rm.Images["label"].CreatedAt) +} + +func mustMakeInfo(ref string, label time.Time, created time.Time) image.Info { + r, err := image.ParseRef(ref) + if err != nil { + panic(err) + } + var labels image.Labels + if !label.IsZero() { + labels.Created = label + } + return image.Info{ID: r, Labels: labels, CreatedAt: created} +} From 6a3835ae55ce7ff6489391c29571b4b295b07e03 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Wed, 10 Jul 2019 11:13:42 +0200 Subject: [PATCH 004/158] Add FluxCD DockerHub organization to whitelist --- cmd/fluxd/main.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmd/fluxd/main.go b/cmd/fluxd/main.go index 6fefd9bd5..81a29befc 100644 --- a/cmd/fluxd/main.go +++ b/cmd/fluxd/main.go @@ -146,7 +146,7 @@ func main() { registryTrace = fs.Bool("registry-trace", false, "output trace of image registry requests to log") registryInsecure = fs.StringSlice("registry-insecure-host", []string{}, "let these registry hosts skip TLS host verification and fall back to using HTTP instead of HTTPS; this allows man-in-the-middle attacks, so use with extreme caution") registryExcludeImage = fs.StringSlice("registry-exclude-image", []string{"k8s.gcr.io/*"}, "do not scan images that match these glob expressions; the default is to exclude the 'k8s.gcr.io/*' images") - registryUseLabels = fs.StringSlice("registry-use-labels", []string{"index.docker.io/weaveworks/*"}, "use the timestamp (RFC3339) from labels for (canonical) image refs that match these glob expression") + registryUseLabels = fs.StringSlice("registry-use-labels", []string{"index.docker.io/weaveworks/*", "index.docker.io/fluxcd/*"}, "use the timestamp (RFC3339) from labels for (canonical) image refs that match these glob expression") // AWS authentication registryAWSRegions = fs.StringSlice("registry-ecr-region", nil, "include just these AWS regions when scanning images in ECR; when not supplied, the cluster's region will included if it can be detected through the AWS API") From aed61613e2279e625ecba177a4919d4a5ed0c520 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Wed, 3 Jul 2019 14:50:54 +0200 Subject: [PATCH 005/158] Push images to DockerHub fluxcd org (#2213) Push images to DockerHub fluxcd org --- .circleci/config.yml | 28 +++++++++++++++++++++++++--- Makefile | 2 +- test/bin/test-flux | 2 +- test/e2e/run.sh | 6 ++++-- test/flux-deploy-all.yaml | 2 +- 5 files changed, 32 insertions(+), 8 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 4b6e3bd51..26f198c24 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -61,13 +61,22 @@ jobs: name: Maybe push prerelease images command: | if [ -z "${CIRCLE_TAG}" -a "${CIRCLE_BRANCH}" == "master" ]; then + # Push to weaveworks org echo "$DOCKER_REGISTRY_PASSWORD" | docker login --username "$DOCKER_REGISTRY_USER" --password-stdin - docker tag "docker.io/weaveworks/flux:$(docker/image-tag)" "docker.io/weaveworks/flux-prerelease:$(docker/image-tag)" + docker tag "docker.io/fluxcd/flux:$(docker/image-tag)" "docker.io/weaveworks/flux-prerelease:$(docker/image-tag)" docker push "docker.io/weaveworks/flux-prerelease:$(docker/image-tag)" - docker tag "docker.io/weaveworks/helm-operator:$(docker/image-tag)" "docker.io/weaveworks/helm-operator-prerelease:$(docker/image-tag)" + docker tag "docker.io/fluxcd/helm-operator:$(docker/image-tag)" "docker.io/weaveworks/helm-operator-prerelease:$(docker/image-tag)" docker push "docker.io/weaveworks/helm-operator-prerelease:$(docker/image-tag)" + + # Push to fluxcd org + echo "$DOCKER_FLUXCD_PASSWORD" | docker login --username "$DOCKER_FLUXCD_USER" --password-stdin + docker tag "docker.io/fluxcd/flux:$(docker/image-tag)" "docker.io/fluxcd/flux-prerelease:$(docker/image-tag)" + docker push "docker.io/fluxcd/flux-prerelease:$(docker/image-tag)" + + docker tag "docker.io/fluxcd/helm-operator:$(docker/image-tag)" "docker.io/fluxcd/helm-operator-prerelease:$(docker/image-tag)" + docker push "docker.io/fluxcd/helm-operator-prerelease:$(docker/image-tag)" fi - deploy: name: Maybe push release image and upload binaries @@ -76,13 +85,26 @@ jobs: go get github.com/weaveworks/github-release make release-bins bin/upload-binaries + # Push to weaveworks org echo "$DOCKER_REGISTRY_PASSWORD" | docker login --username "$DOCKER_REGISTRY_USER" --password-stdin + docker tag "docker.io/fluxcd/flux:${CIRCLE_TAG}" "docker.io/weaveworks/flux:${CIRCLE_TAG}" docker push "docker.io/weaveworks/flux:${CIRCLE_TAG}" + + # Push to fluxcd org + echo "$DOCKER_FLUXCD_PASSWORD" | docker login --username "$DOCKER_FLUXCD_USER" --password-stdin + docker push "docker.io/fluxcd/flux:${CIRCLE_TAG}" fi if echo "${CIRCLE_TAG}" | grep -Eq "helm-[0-9]+(\.[0-9]+)*(-[a-z]+)?$"; then - echo "$DOCKER_REGISTRY_PASSWORD" | docker login --username "$DOCKER_REGISTRY_USER" --password-stdin RELEASE_TAG=$(echo "$CIRCLE_TAG" | cut -c 6-) + + # Push to weaveworks org + echo "$DOCKER_REGISTRY_PASSWORD" | docker login --username "$DOCKER_REGISTRY_USER" --password-stdin + docker tag "docker.io/fluxcd/helm-operator:${RELEASE_TAG}" "docker.io/weaveworks/helm-operator:${RELEASE_TAG}" docker push "docker.io/weaveworks/helm-operator:${RELEASE_TAG}" + + # Push to fluxcd org + echo "$DOCKER_FLUXCD_PASSWORD" | docker login --username "$DOCKER_FLUXCD_USER" --password-stdin + docker push "docker.io/fluxcd/helm-operator:${RELEASE_TAG}" fi workflows: diff --git a/Makefile b/Makefile index b44de0b31..e9e2a3f01 100644 --- a/Makefile +++ b/Makefile @@ -63,7 +63,7 @@ e2e: test/bin/helm test/bin/kubectl build/.flux.done build/.helm-operator.done build/.%.done: docker/Dockerfile.% mkdir -p ./build/docker/$* cp $^ ./build/docker/$*/ - $(SUDO) docker build -t docker.io/weaveworks/$* -t docker.io/weaveworks/$*:$(IMAGE_TAG) \ + $(SUDO) docker build -t docker.io/fluxcd/$* -t docker.io/fluxcd/$*:$(IMAGE_TAG) \ --build-arg VCS_REF="$(VCS_REF)" \ --build-arg BUILD_DATE="$(BUILD_DATE)" \ -f build/docker/$*/Dockerfile.$* ./build/docker/$* diff --git a/test/bin/test-flux b/test/bin/test-flux index 0ba284995..3e2cf2c80 100755 --- a/test/bin/test-flux +++ b/test/bin/test-flux @@ -22,7 +22,7 @@ minikube start --profile "$PROFILE" --keep-context MINIKUBE_IP=$(minikube --profile "$PROFILE" ip) # Copy the latest Flux image into the minikube VM -docker save "docker.io/weaveworks/flux:latest" | (eval $(minikube --profile "$PROFILE" docker-env) && docker load) +docker save "docker.io/fluxcd/flux:latest" | (eval $(minikube --profile "$PROFILE" docker-env) && docker load) # Create a central git repo inside the minikube VM and get the host key for ssh access minikube --profile "$PROFILE" ssh -- git init --bare /home/docker/flux.git diff --git a/test/e2e/run.sh b/test/e2e/run.sh index b5f400a3b..19d3b49b7 100755 --- a/test/e2e/run.sh +++ b/test/e2e/run.sh @@ -76,8 +76,8 @@ kubectl -n "${FLUX_NAMESPACE}" rollout status deployment/gitsrv if [ "${USING_KIND}" = 'true' ]; then echo '>>> Loading images into the Kind cluster' - kind --name "${KIND_CLUSTER}" load docker-image 'docker.io/weaveworks/flux:latest' - kind --name "${KIND_CLUSTER}" load docker-image 'docker.io/weaveworks/helm-operator:latest' + kind --name "${KIND_CLUSTER}" load docker-image 'docker.io/fluxcd/flux:latest' + kind --name "${KIND_CLUSTER}" load docker-image 'docker.io/fluxcd/helm-operator:latest' fi echo '>>> Installing Flux with Helm' @@ -100,6 +100,7 @@ defer helm delete --purge flux > /dev/null 2>&1 helm install --name flux --wait \ --namespace "${FLUX_NAMESPACE}" \ +--set image.repository=docker.io/fluxcd/flux \ --set image.tag=latest \ --set git.url=ssh://git@gitsrv/git-server/repos/cluster.git \ --set git.secretName=ssh-git \ @@ -107,6 +108,7 @@ helm install --name flux --wait \ --set git.config.secretName=gitconfig \ --set git.config.enabled=true \ --set-string git.config.data="${GITCONFIG}" \ +--set helmOperator.repository=docker.io/fluxcd/helm-operator \ --set helmOperator.tag=latest \ --set helmOperator.create=true \ --set helmOperator.createCRD=true \ diff --git a/test/flux-deploy-all.yaml b/test/flux-deploy-all.yaml index cd94bb58f..8af30e1ef 100644 --- a/test/flux-deploy-all.yaml +++ b/test/flux-deploy-all.yaml @@ -76,7 +76,7 @@ spec: containers: - name: flux # Require locally built image - image: docker.io/weaveworks/flux:latest + image: docker.io/fluxcd/flux:latest imagePullPolicy: Never ports: - containerPort: 3030 # informational From 4a10ee9c4aca14b55e9d813d8aa464bcb7632978 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 1 Jul 2019 16:14:46 +0200 Subject: [PATCH 006/158] Only require image TS when it is used for sorting (#2175) Only require image TS when it is used for sorting --- daemon/images.go | 2 +- daemon/images_test.go | 29 +++++++++++++++++++++++++---- policy/pattern.go | 14 ++++++++++++++ 3 files changed, 40 insertions(+), 5 deletions(-) diff --git a/daemon/images.go b/daemon/images.go index 6404b825b..f32281b1e 100644 --- a/daemon/images.go +++ b/daemon/images.go @@ -103,7 +103,7 @@ func calculateChanges(logger log.Logger, candidateWorkloads resources, workloads continue containers } current := repoMetadata.FindImageWithRef(currentImageID) - if current.CreatedTS().IsZero() || latest.CreatedTS().IsZero() { + if pattern.RequiresTimestamp() && (current.CreatedTS().IsZero() || latest.CreatedTS().IsZero()) { logger.Log("warning", "image with zero created timestamp", "current", fmt.Sprintf("%s (%s)", current.ID, current.CreatedTS()), "latest", fmt.Sprintf("%s (%s)", latest.ID, latest.CreatedTS()), "action", "skip container") continue containers } diff --git a/daemon/images_test.go b/daemon/images_test.go index 2d2295f25..9fdcfcb5c 100644 --- a/daemon/images_test.go +++ b/daemon/images_test.go @@ -18,6 +18,7 @@ import ( const ( container1 = "container1" container2 = "container2" + container3 = "container3" currentContainer1Image = "container1/application:current" newContainer1Image = "container1/application:new" @@ -25,6 +26,9 @@ const ( currentContainer2Image = "container2/application:current" newContainer2Image = "container2/application:new" noTagContainer2Image = "container2/application" + + currentContainer3Image = "container3/application:1.0.0" + newContainer3Image = "container3/application:1.1.0" ) type candidate struct { @@ -152,6 +156,7 @@ func TestCalculateChanges_UntaggedImage(t *testing.T) { t.Errorf("Expected changed image to be %s, got %s", newContainer1Image, newImage) } } + func TestCalculateChanges_ZeroTimestamp(t *testing.T) { logger := log.NewNopLogger() resourceID := flux.MakeResourceID(ns, "deployment", "application") @@ -159,7 +164,8 @@ func TestCalculateChanges_ZeroTimestamp(t *testing.T) { resourceID: candidate{ resourceID: resourceID, policies: policy.Set{ - policy.Automated: "true", + policy.Automated: "true", + policy.TagPrefix(container3): "semver:^1.0", }, }, } @@ -176,6 +182,10 @@ func TestCalculateChanges_ZeroTimestamp(t *testing.T) { Name: container2, Image: mustParseImageRef(currentContainer2Image), }, + { + Name: container3, + Image: mustParseImageRef(currentContainer3Image), + }, }, }, }, @@ -184,14 +194,21 @@ func TestCalculateChanges_ZeroTimestamp(t *testing.T) { { current1 := makeImageInfo(currentContainer1Image, time.Now()) new1 := makeImageInfo(newContainer1Image, time.Now().Add(1*time.Second)) + zeroTimestampCurrent2 := image.Info{ID: mustParseImageRef(currentContainer2Image)} new2 := makeImageInfo(newContainer2Image, time.Now().Add(1*time.Second)) + + current3 := makeImageInfo(currentContainer3Image, time.Now()) + zeroTimestampNew3 := image.Info{ID: mustParseImageRef(newContainer3Image)} + imageRegistry = ®istryMock.Registry{ Images: []image.Info{ current1, new1, zeroTimestampCurrent2, new2, + current3, + zeroTimestampNew3, }, } } @@ -202,9 +219,13 @@ func TestCalculateChanges_ZeroTimestamp(t *testing.T) { changes := calculateChanges(logger, candidateWorkloads, workloads, imageRepos) - if len := len(changes.Changes); len != 1 { - t.Errorf("Expected exactly 1 change, got %d changes", len) - } else if newImage := changes.Changes[0].ImageID.String(); newImage != newContainer1Image { + if len := len(changes.Changes); len != 2 { + t.Fatalf("Expected exactly 2 changes, got %d changes: %v", len, changes.Changes) + } + if newImage := changes.Changes[0].ImageID.String(); newImage != newContainer1Image { t.Errorf("Expected changed image to be %s, got %s", newContainer1Image, newImage) } + if newImage := changes.Changes[1].ImageID.String(); newImage != newContainer3Image { + t.Errorf("Expected changed image to be %s, got %s", newContainer3Image, newImage) + } } diff --git a/policy/pattern.go b/policy/pattern.go index 56380ab05..5f0d317ba 100644 --- a/policy/pattern.go +++ b/policy/pattern.go @@ -32,6 +32,8 @@ type Pattern interface { Newer(a, b *image.Info) bool // Valid returns true if the pattern is considered valid. Valid() bool + // RequiresTimestamp returns true if the pattern orders based on timestamp data. + RequiresTimestamp() bool } type GlobPattern string @@ -87,6 +89,10 @@ func (g GlobPattern) Valid() bool { return true } +func (g GlobPattern) RequiresTimestamp() bool { + return true +} + func (s SemverPattern) Matches(tag string) bool { v, err := semver.NewVersion(tag) if err != nil { @@ -111,6 +117,10 @@ func (s SemverPattern) Valid() bool { return s.constraints != nil } +func (s SemverPattern) RequiresTimestamp() bool { + return false +} + func (r RegexpPattern) Matches(tag string) bool { if r.regexp == nil { // Invalid regexp match anything @@ -130,3 +140,7 @@ func (r RegexpPattern) Newer(a, b *image.Info) bool { func (r RegexpPattern) Valid() bool { return r.regexp != nil } + +func (r RegexpPattern) RequiresTimestamp() bool { + return true +} From 9c90dcbaa0f1223a97661fd6fe10a4bea72ea638 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Wed, 10 Jul 2019 12:25:44 +0200 Subject: [PATCH 007/158] Implement whitelist for usage of TS from labels (#2176) Implement whitelist for usage of TS from labels --- chart/flux/README.md | 1 + chart/flux/templates/deployment.yaml | 3 + chart/flux/values.yaml | 2 + cmd/fluxd/main.go | 4 + registry/cache/memcached/integration_test.go | 2 +- registry/cache/registry.go | 49 ++++++++++- registry/cache/registry_test.go | 74 ++++++++++++++++ site/daemon.md | 93 ++++++++++---------- 8 files changed, 180 insertions(+), 48 deletions(-) create mode 100644 registry/cache/registry_test.go diff --git a/chart/flux/README.md b/chart/flux/README.md index fc281dc4d..1353abe89 100755 --- a/chart/flux/README.md +++ b/chart/flux/README.md @@ -232,6 +232,7 @@ The following tables lists the configurable parameters of the Weave Flux chart a | `registry.insecureHosts` | `None` | Use HTTP rather than HTTPS for the image registry domains | `registry.cacheExpiry` | `None` | Duration to keep cached image info (deprecated) | `registry.excludeImage` | `None` | Do not scan images that match these glob expressions; if empty, 'k8s.gcr.io/*' images are excluded +| `registry.useTimestampLabels` | `None` | Allow usage of (RFC3339) timestamp labels from (canonical) image refs that match these glob expressions; if empty, 'index.docker.io/weaveworks/*' images are allowed | `registry.ecr.region` | `None` | Restrict ECR scanning to these AWS regions; if empty, only the cluster's region will be scanned | `registry.ecr.includeId` | `None` | Restrict ECR scanning to these AWS account IDs; if empty, all account IDs that aren't excluded may be scanned | `registry.ecr.excludeId` | `602401143452` | Do not scan ECR for images in these AWS account IDs; the default is to exclude the EKS system account diff --git a/chart/flux/templates/deployment.yaml b/chart/flux/templates/deployment.yaml index e292dbabe..737c2d2d2 100644 --- a/chart/flux/templates/deployment.yaml +++ b/chart/flux/templates/deployment.yaml @@ -196,6 +196,9 @@ spec: {{- if .Values.registry.excludeImage }} - --registry-exclude-image={{ .Values.registry.excludeImage }} {{- end }} + {{- if .Values.registry.useTimestampLabels }} + - --registry-use-labels={{ .Values.registry.useTimestampLabels }} + {{- end }} {{- if .Values.registry.ecr.region }} - --registry-ecr-region={{ .Values.registry.ecr.region }} {{- end }} diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index 344134e87..63fbd6668 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -177,6 +177,8 @@ registry: cacheExpiry: # Do not scan images that match these glob expressions excludeImage: + # Allow usage of (RFC3339) timestamp labels from (canonical) image refs that match these glob expressions + useTimestampLabels: # AWS ECR settings ecr: region: diff --git a/cmd/fluxd/main.go b/cmd/fluxd/main.go index 75979707e..481886993 100644 --- a/cmd/fluxd/main.go +++ b/cmd/fluxd/main.go @@ -145,6 +145,7 @@ func main() { registryTrace = fs.Bool("registry-trace", false, "output trace of image registry requests to log") registryInsecure = fs.StringSlice("registry-insecure-host", []string{}, "let these registry hosts skip TLS host verification and fall back to using HTTP instead of HTTPS; this allows man-in-the-middle attacks, so use with extreme caution") registryExcludeImage = fs.StringSlice("registry-exclude-image", []string{"k8s.gcr.io/*"}, "do not scan images that match these glob expressions; the default is to exclude the 'k8s.gcr.io/*' images") + registryUseLabels = fs.StringSlice("registry-use-labels", []string{"index.docker.io/weaveworks/*", "index.docker.io/fluxcd/*"}, "use the timestamp (RFC3339) from labels for (canonical) image refs that match these glob expression") // AWS authentication registryAWSRegions = fs.StringSlice("registry-ecr-region", nil, "include just these AWS regions when scanning images in ECR; when not supplied, the cluster's region will included if it can be detected through the AWS API") @@ -493,6 +494,9 @@ func main() { cacheRegistry = &cache.Cache{ Reader: cacheClient, + Decorators: []cache.Decorator{ + cache.TimestampLabelWhitelist(*registryUseLabels), + }, } cacheRegistry = registry.NewInstrumentedRegistry(cacheRegistry) diff --git a/registry/cache/memcached/integration_test.go b/registry/cache/memcached/integration_test.go index cfbd529ec..f25c0e82d 100644 --- a/registry/cache/memcached/integration_test.go +++ b/registry/cache/memcached/integration_test.go @@ -48,7 +48,7 @@ func TestWarming_WarmerWriteCacheRead(t *testing.T) { Trace: true, } - r := &cache.Cache{mc} + r := &cache.Cache{Reader: mc} w, _ := cache.NewWarmer(remote, mc, 125) shutdown := make(chan struct{}) diff --git a/registry/cache/registry.go b/registry/cache/registry.go index 967bae3ec..d238dc5e9 100644 --- a/registry/cache/registry.go +++ b/registry/cache/registry.go @@ -5,6 +5,7 @@ import ( "time" "github.com/pkg/errors" + "github.com/ryanuber/go-glob" fluxerr "github.com/weaveworks/flux/errors" "github.com/weaveworks/flux/image" @@ -28,7 +29,48 @@ repository. // Cache is a local cache of image metadata. type Cache struct { - Reader Reader + Reader Reader + Decorators []Decorator +} + +// Decorator is for decorating an ImageRepository before it is returned. +type Decorator interface { + apply(*ImageRepository) +} + +// TimestampLabelWhitelist contains a string slice of glob patterns. Any +// canonical image reference that matches one of the glob patterns will +// prefer creation timestamps from labels over the one it received from +// the registry. +type TimestampLabelWhitelist []string + +// apply checks if any of the canonical image references from the +// repository matches a glob pattern from the list. If it does, and the +// image record has a valid timestamp label, it will replace the Created +// field with the value from the label for all images in the repository. +func (l TimestampLabelWhitelist) apply(r *ImageRepository) { + var match bool + for k, i := range r.Images { + if !match { + for _, exp := range l { + if glob.Glob(exp, i.ID.CanonicalName().String()) { + match = true + break + } + } + if !match { + return + } + } + + switch { + case !i.Labels.Created.IsZero(): + i.CreatedAt = i.Labels.Created + case !i.Labels.BuildDate.IsZero(): + i.CreatedAt = i.Labels.BuildDate + } + r.Images[k] = i + } } // GetImageRepositoryMetadata returns the metadata from an image @@ -53,6 +95,11 @@ func (c *Cache) GetImageRepositoryMetadata(id image.Name) (image.RepositoryMetad return image.RepositoryMetadata{}, ErrNotCached } + // (Maybe) decorate the image repository. + for _, d := range c.Decorators { + d.apply(&repo) + } + return repo.RepositoryMetadata, nil } diff --git a/registry/cache/registry_test.go b/registry/cache/registry_test.go new file mode 100644 index 000000000..84cd15949 --- /dev/null +++ b/registry/cache/registry_test.go @@ -0,0 +1,74 @@ +package cache + +import ( + "encoding/json" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/weaveworks/flux/image" +) + +// mockStorage holds a fixed ImageRepository item. +type mockStorage struct { + Item ImageRepository +} + +// GetKey will always return the same item from the storage, +// and does not care about the key it receives. +func (m *mockStorage) GetKey(k Keyer) ([]byte, time.Time, error) { + b, err := json.Marshal(m.Item) + if err != nil { + return []byte{}, time.Time{}, err + } + return b, time.Time{}, nil +} + +// appendImage adds an image to the mocked storage item. +func (m *mockStorage) appendImage(i image.Info) { + tag := i.ID.Tag + + m.Item.Images[tag] = i + m.Item.Tags = append(m.Item.Tags, tag) +} + +func mockReader() *mockStorage { + return &mockStorage{ + Item: ImageRepository{ + RepositoryMetadata: image.RepositoryMetadata{ + Tags: []string{}, + Images: map[string]image.Info{}, + }, + LastUpdate: time.Now(), + }, + } +} + +func Test_WhitelabelDecorator(t *testing.T) { + r := mockReader() + + // Image with no timestamp label + r.appendImage(mustMakeInfo("docker.io/fluxcd/flux:equal", time.Time{}, time.Now().UTC())) + // Image with a timestamp label + r.appendImage(mustMakeInfo("docker.io/fluxcd/flux:label", time.Now().Add(-10*time.Second).UTC(), time.Now().UTC())) + + c := Cache{r, []Decorator{TimestampLabelWhitelist{"index.docker.io/fluxcd/*"}}} + + rm, err := c.GetImageRepositoryMetadata(image.Name{}) + assert.NoError(t, err) + + assert.Equal(t, r.Item.Images["equal"].CreatedAt, rm.Images["equal"].CreatedAt) + assert.Equal(t, r.Item.Images["label"].Labels.Created, rm.Images["label"].CreatedAt) +} + +func mustMakeInfo(ref string, label time.Time, created time.Time) image.Info { + r, err := image.ParseRef(ref) + if err != nil { + panic(err) + } + var labels image.Labels + if !label.IsZero() { + labels.Created = label + } + return image.Info{ID: r, Labels: labels, CreatedAt: created} +} diff --git a/site/daemon.md b/site/daemon.md index 15544f2a0..3e896d249 100644 --- a/site/daemon.md +++ b/site/daemon.md @@ -42,57 +42,58 @@ document. fluxd requires setup and offers customization though a multitude of flags. -| Flag | Default | Purpose -| ------------------------------------------------ | ------------------------ | --- -| --listen -l | `:3030` | listen address where /metrics and API will be served -| --listen-metrics | | listen address for /metrics endpoint -| --kubernetes-kubectl | | optional, explicit path to kubectl tool -| --version | false | output the version number and exit +| Flag | Default | Purpose +| ------------------------------------------------ | ---------------------------------- | --- +| --listen -l | `:3030` | listen address where /metrics and API will be served +| --listen-metrics | | listen address for /metrics endpoint +| --kubernetes-kubectl | | optional, explicit path to kubectl tool +| --version | false | output the version number and exit | **Git repo & key etc.** -| --git-url | | URL of git repo with Kubernetes manifests; e.g., `git@github.com:weaveworks/flux-get-started` -| --git-branch | `master` | branch of git repo to use for Kubernetes manifests -| --git-ci-skip | false | when set, fluxd will append `\n\n[ci skip]` to its commit messages -| --git-ci-skip-message | `""` | if provided, fluxd will append this to commit messages (overrides --git-ci-skip`) -| --git-path | | path within git repo to locate Kubernetes manifests (relative path) -| --git-user | `Weave Flux` | username to use as git committer -| --git-email | `support@weave.works` | email to use as git committer -| --git-set-author | false | if set, the author of git commits will reflect the user who initiated the commit and will differ from the git committer -| --git-gpg-key-import | | if set, fluxd will attempt to import the gpg key(s) found on the given path -| --git-signing-key | | if set, commits made by fluxd to the user git repo will be signed with the provided GPG key. See [Git commit signing](git-commit-signing.md) to learn how to use this feature -| --git-label | | label to keep track of sync progress; overrides both --git-sync-tag and --git-notes-ref -| --git-sync-tag | `flux-sync` | tag to use to mark sync progress for this cluster (old config, still used if --git-label is not supplied) -| --git-notes-ref | `flux` | ref to use for keeping commit annotations in git notes -| --git-poll-interval | `5m` | period at which to fetch any new commits from the git repo -| --git-timeout | `20s` | duration after which git operations time out +| --git-url | | URL of git repo with Kubernetes manifests; e.g., `git@github.com:weaveworks/flux-get-started` +| --git-branch | `master` | branch of git repo to use for Kubernetes manifests +| --git-ci-skip | false | when set, fluxd will append `\n\n[ci skip]` to its commit messages +| --git-ci-skip-message | `""` | if provided, fluxd will append this to commit messages (overrides --git-ci-skip`) +| --git-path | | path within git repo to locate Kubernetes manifests (relative path) +| --git-user | `Weave Flux` | username to use as git committer +| --git-email | `support@weave.works` | email to use as git committer +| --git-set-author | false | if set, the author of git commits will reflect the user who initiated the commit and will differ from the git committer +| --git-gpg-key-import | | if set, fluxd will attempt to import the gpg key(s) found on the given path +| --git-signing-key | | if set, commits made by fluxd to the user git repo will be signed with the provided GPG key. See [Git commit signing](git-commit-signing.md) to learn how to use this feature +| --git-label | | label to keep track of sync progress; overrides both --git-sync-tag and --git-notes-ref +| --git-sync-tag | `flux-sync` | tag to use to mark sync progress for this cluster (old config, still used if --git-label is not supplied) +| --git-notes-ref | `flux` | ref to use for keeping commit annotations in git notes +| --git-poll-interval | `5m` | period at which to fetch any new commits from the git repo +| --git-timeout | `20s` | duration after which git operations time out | **syncing:** control over how config is applied to the cluster -| --sync-interval | `5m` | apply the git config to the cluster at least this often. New commits may provoke more frequent syncs -| --sync-garbage-collection | `false` | experimental: when set, fluxd will delete resources that it created, but are no longer present in git (see [garbage collection](./garbagecollection.md)) +| --sync-interval | `5m` | apply the git config to the cluster at least this often. New commits may provoke more frequent syncs +| --sync-garbage-collection | `false` | experimental: when set, fluxd will delete resources that it created, but are no longer present in git (see [garbage collection](./garbagecollection.md)) | **registry cache:** (none of these need overriding, usually) -| --memcached-hostname | `memcached` | hostname for memcached service to use for caching image metadata -| --memcached-timeout | `1s` | maximum time to wait before giving up on memcached requests -| --memcached-service | `memcached` | SRV service used to discover memcache servers -| --registry-cache-expiry | `1h` | Duration to keep cached registry tag info. Must be < 1 month. -| --registry-poll-interval | `5m` | period at which to poll registry for new images -| --registry-rps | `200` | maximum registry requests per second per host -| --registry-burst | `125` | maximum number of warmer connections to remote and memcache -| --registry-insecure-host | [] | registry hosts to use HTTP for (instead of HTTPS) -| --registry-exclude-image | `["k8s.gcr.io/*"]` | do not scan images that match these glob expressions -| --docker-config | `""` | path to a Docker config file with default image registry credentials -| --registry-ecr-region | `[]` | Allow these AWS regions when scanning images from ECR (multiple values allowed); defaults to the detected cluster region -| --registry-ecr-include-id | `[]` | Include these AWS account ID(s) when scanning images in ECR (multiple values allowed); empty means allow all, unless excluded -| --registry-ecr-exclude-id | `[]` | Exclude these AWS account ID(s) when scanning ECR (multiple values allowed); defaults to the EKS system account, so system images will not be scanned -| --registry-require | `[]` | exit with an error if the given services are not available. Useful for escalating misconfiguration or outages that might otherwise go undetected. Presently supported values: {`ecr`} | +| --memcached-hostname | `memcached` | hostname for memcached service to use for caching image metadata +| --memcached-timeout | `1s` | maximum time to wait before giving up on memcached requests +| --memcached-service | `memcached` | SRV service used to discover memcache servers +| --registry-cache-expiry | `1h` | Duration to keep cached registry tag info. Must be < 1 month. +| --registry-poll-interval | `5m` | period at which to poll registry for new images +| --registry-rps | `200` | maximum registry requests per second per host +| --registry-burst | `125` | maximum number of warmer connections to remote and memcache +| --registry-insecure-host | [] | registry hosts to use HTTP for (instead of HTTPS) +| --registry-exclude-image | `["k8s.gcr.io/*"]` | do not scan images that match these glob expressions +| --registry-use-labels | `["index.docker.io/weaveworks/*", "index.docker.io/fluxcd/*"]` | use the timestamp (RFC3339) from labels for (canonical) image refs that match these glob expressions +| --docker-config | `""` | path to a Docker config file with default image registry credentials +| --registry-ecr-region | `[]` | allow these AWS regions when scanning images from ECR (multiple values allowed); defaults to the detected cluster region +| --registry-ecr-include-id | `[]` | include these AWS account ID(s) when scanning images in ECR (multiple values allowed); empty means allow all, unless excluded +| --registry-ecr-exclude-id | `[]` | exclude these AWS account ID(s) when scanning ECR (multiple values allowed); defaults to the EKS system account, so system images will not be scanned +| --registry-require | `[]` | exit with an error if the given services are not available. Useful for escalating misconfiguration or outages that might otherwise go undetected. Presently supported values: {`ecr`} | | **k8s-secret backed ssh keyring configuration** -| --k8s-secret-name | `flux-git-deploy` | name of the k8s secret used to store the private SSH key -| --k8s-secret-volume-mount-path | `/etc/fluxd/ssh` | mount location of the k8s secret storing the private SSH key -| --k8s-secret-data-key | `identity` | data key holding the private SSH key within the k8s secret +| --k8s-secret-name | `flux-git-deploy` | name of the k8s secret used to store the private SSH key +| --k8s-secret-volume-mount-path | `/etc/fluxd/ssh` | mount location of the k8s secret storing the private SSH key +| --k8s-secret-data-key | `identity` | data key holding the private SSH key within the k8s secret | **k8s configuration** -| --k8s-allow-namespace | | experimental: restrict all operations to the provided namespaces +| --k8s-allow-namespace | | experimental: restrict all operations to the provided namespaces | **upstream service** -| --connect | | connect to an upstream service e.g., Weave Cloud, at this base address -| --token | | authentication token for upstream service +| --connect | | connect to an upstream service e.g., Weave Cloud, at this base address +| --token | | authentication token for upstream service | **SSH key generation** -| --ssh-keygen-bits | | -b argument to ssh-keygen (default unspecified) -| --ssh-keygen-type | | -t argument to ssh-keygen (default unspecified) +| --ssh-keygen-bits | | -b argument to ssh-keygen (default unspecified) +| --ssh-keygen-type | | -t argument to ssh-keygen (default unspecified) | **manifest generation** -| --manifest-generation | false | experimental; search for .flux.yaml files to generate manifests +| --manifest-generation | false | experimental; search for .flux.yaml files to generate manifests From ede3e65a2f24f0e1eeffda0b51fa3a74399b8bf1 Mon Sep 17 00:00:00 2001 From: Alfonso Acosta Date: Wed, 26 Jun 2019 23:32:23 +0200 Subject: [PATCH 008/158] Merge pull request #2195 from 2opremio/bump-circleci-image Upgrade CircleCI image to ubuntu-1604:201903-01 --- .circleci/config.yml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 26f198c24..0bbb1ff91 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -2,7 +2,8 @@ version: 2 jobs: build: working_directory: ~/flux - machine: true + machine: + image: ubuntu-1604:201903-01 environment: GO_VERSION: 1.12.5 # We don't need a GOPATH but CircleCI defines it, so we override it @@ -22,10 +23,8 @@ jobs: - run: name: Update packages and Start Memcached command: | - # These repos fail and we don't need them: - sudo rm /etc/apt/sources.list.d/circleci_trusty.list /etc/apt/sources.list.d/google-chrome.list sudo apt-get update - sudo apt-get install -y git rng-tools docker-ce memcached + sudo apt-get install -y git rng-tools memcached git version docker version - restore_cache: From d6f7fd0f61c03ba33be1bcfcde295b6ae5133825 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Mon, 8 Jul 2019 12:15:15 +0100 Subject: [PATCH 009/158] Merge pull request #2222 from weaveworks/issue/2119-allow-google-cloudsdk-env Allow Google Cloud SDK env entries for git --- git/operations.go | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/git/operations.go b/git/operations.go index ef25c8211..2d6f2f52f 100644 --- a/git/operations.go +++ b/git/operations.go @@ -24,8 +24,16 @@ var exemptedTraceCommands = []string{ // "config", } -// Env vars that are allowed to be inherited from the os -var allowedEnvVars = []string{"http_proxy", "https_proxy", "no_proxy", "HOME", "GNUPGHOME"} +// Env vars that are allowed to be inherited from the OS +var allowedEnvVars = []string{ + // these are for people using (no) proxies + "http_proxy", "https_proxy", "no_proxy", + // these are needed for GPG to find its files + "HOME", "GNUPGHOME", + // these are for Google Cloud SDK to find its files (which will + // have to be mounted, if running in a container) + "CLOUDSDK_CONFIG", "CLOUDSDK_PYTHON", +} type gitCmdConfig struct { dir string From 779051f214a616aac6e1394db70edcb1e58043e7 Mon Sep 17 00:00:00 2001 From: Alfonso Acosta Date: Fri, 28 Jun 2019 13:12:46 +0200 Subject: [PATCH 010/158] Merge pull request #2207 from 2opremio/get-dependencies-faster Optimize make dependecy obtention --- Makefile | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index e9e2a3f01..578f1b974 100644 --- a/Makefile +++ b/Makefile @@ -22,11 +22,11 @@ GOBIN?=$(shell echo `go env GOPATH`/bin) # if you're testing out the Makefile with `-W` (pretend a file is # new); use the full path to the pretend-new file, e.g., # `make -W $PWD/registry/registry.go` -godeps=$(shell go list -f '{{join .Deps "\n"}}' $1 | grep -v /vendor/ | xargs go list -f '{{if not .Standard}}{{ $$dep := . }}{{range .GoFiles}}{{$$dep.Dir}}/{{.}} {{end}}{{end}}') +godeps=$(shell go list -deps -f '{{if not .Standard}}{{ $$dep := . }}{{range .GoFiles}}{{$$dep.Dir}}/{{.}} {{end}}{{end}}' $(1)) -FLUXD_DEPS:=$(call godeps,./cmd/fluxd) -FLUXCTL_DEPS:=$(call godeps,./cmd/fluxctl) -HELM_OPERATOR_DEPS:=$(call godeps,./cmd/helm-operator) +FLUXD_DEPS:=$(call godeps,./cmd/fluxd/...) +FLUXCTL_DEPS:=$(call godeps,./cmd/fluxctl/...) +HELM_OPERATOR_DEPS:=$(call godeps,./cmd/helm-operator/...) IMAGE_TAG:=$(shell ./docker/image-tag) VCS_REF:=$(shell git rev-parse HEAD) @@ -113,15 +113,12 @@ cache/%/helm-$(HELM_VERSION): docker/helm.version mv cache/$*/helm $@ $(GOBIN)/fluxctl: $(FLUXCTL_DEPS) -$(GOBIN)/fluxctl: ./cmd/fluxctl/*.go go install ./cmd/fluxctl $(GOBIN)/fluxd: $(FLUXD_DEPS) -$(GOBIN)/fluxd: cmd/fluxd/*.go go install ./cmd/fluxd $(GOBIN)/helm-operator: $(HELM_OPERATOR_DEPS) -$(GOBIN)/help-operator: cmd/helm-operator/*.go go install ./cmd/helm-operator integration-test: all From d7df66a5c48ed89856cafcab28499b43af9c1700 Mon Sep 17 00:00:00 2001 From: Stefan Prodan Date: Thu, 27 Jun 2019 16:59:26 +0300 Subject: [PATCH 011/158] Merge pull request #2202 from weaveworks/kind-0.4 Update Kubernetes Kind to v0.4.0 --- test/e2e/run.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/e2e/run.sh b/test/e2e/run.sh index 19d3b49b7..4ef3f8ebc 100755 --- a/test/e2e/run.sh +++ b/test/e2e/run.sh @@ -26,7 +26,7 @@ function defer() { REPO_ROOT=$(git rev-parse --show-toplevel) SCRIPT_DIR="${REPO_ROOT}/test/e2e" -KIND_VERSION=0.2.1 +KIND_VERSION="v0.4.0" CACHE_DIR="${REPO_ROOT}/cache/$CURRENT_OS_ARCH" KIND_CACHE_PATH="${CACHE_DIR}/kind-$KIND_VERSION" KIND_CLUSTER=flux-e2e From 553c0d7bc9fb1f1490f48ca55ec7e5a70005ec16 Mon Sep 17 00:00:00 2001 From: Stefan Prodan Date: Thu, 27 Jun 2019 19:40:24 +0300 Subject: [PATCH 012/158] Merge pull request #2201 from weaveworks/refac Move ResourceID from root to resource package --- api/v11/api.go | 4 +- api/v6/api.go | 8 +- cluster/cluster.go | 9 +- cluster/kubernetes/images.go | 4 +- cluster/kubernetes/kubernetes.go | 16 +- cluster/kubernetes/manifests.go | 3 +- cluster/kubernetes/patch.go | 22 +- cluster/kubernetes/policies.go | 4 +- cluster/kubernetes/policies_test.go | 48 ++-- cluster/kubernetes/resource/load_test.go | 13 +- cluster/kubernetes/resource/resource.go | 7 +- cluster/kubernetes/resourcekinds.go | 9 +- cluster/kubernetes/sync.go | 15 +- cluster/kubernetes/sync_test.go | 19 +- cluster/kubernetes/testfiles/data.go | 40 +-- cluster/kubernetes/update.go | 4 +- cluster/kubernetes/update_test.go | 4 +- cluster/mock/mock.go | 18 +- cluster/sync.go | 3 +- cmd/fluxctl/args.go | 5 +- cmd/fluxctl/args_test.go | 23 +- cmd/fluxctl/list_images_cmd.go | 4 +- cmd/fluxctl/policy_cmd.go | 13 +- cmd/fluxctl/release_cmd.go | 8 +- cmd/fluxctl/release_cmd_test.go | 8 +- daemon/daemon.go | 19 +- daemon/daemon_test.go | 31 ++- daemon/errors.go | 4 +- daemon/images.go | 7 +- daemon/images_test.go | 14 +- daemon/sync.go | 7 +- daemon/sync_test.go | 18 +- event/event.go | 9 +- flux.go | 240 ------------------ git/gittest/repo.go | 4 +- http/daemon/server.go | 13 +- image/image.go | 2 +- image/image_test.go | 4 +- .../apis/flux.weave.works/v1beta1/types.go | 6 +- integrations/helm/release/release.go | 14 +- manifests/configaware.go | 10 +- manifests/configaware_test.go | 9 +- manifests/configfile.go | 8 +- manifests/configfile_test.go | 6 +- manifests/manifests.go | 6 +- manifests/rawfiles.go | 8 +- manifests/store.go | 6 +- policy/policy.go | 9 - registry/client.go | 6 +- registry/credentials.go | 10 +- release/context.go | 7 +- release/releaser_test.go | 97 ++++--- remote/mock.go | 12 +- remote/rpc/clientV11.go | 2 +- remote/rpc/compat.go | 10 +- remote/rpc/server.go | 2 +- resource/id.go | 240 ++++++++++++++++++ resourceid_test.go => resource/id_test.go | 6 +- resource/policy.go | 10 +- resource/resource.go | 9 +- sync/mock.go | 6 +- update/automated.go | 16 +- update/filter.go | 6 +- update/menu.go | 14 +- update/print_test.go | 14 +- update/release_containers.go | 5 +- update/release_image.go | 20 +- update/result.go | 8 +- update/spec.go | 4 +- update/workload.go | 3 +- 70 files changed, 618 insertions(+), 654 deletions(-) delete mode 100644 flux.go create mode 100644 resource/id.go rename resourceid_test.go => resource/id_test.go (87%) diff --git a/api/v11/api.go b/api/v11/api.go index 1e0034204..5b591b9a1 100644 --- a/api/v11/api.go +++ b/api/v11/api.go @@ -4,14 +4,14 @@ package v11 import ( "context" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/api/v10" "github.com/weaveworks/flux/api/v6" + "github.com/weaveworks/flux/resource" ) type ListServicesOptions struct { Namespace string - Services []flux.ResourceID + Services []resource.ID } type Server interface { diff --git a/api/v6/api.go b/api/v6/api.go index a95f2a4f3..4b8e341d5 100644 --- a/api/v6/api.go +++ b/api/v6/api.go @@ -3,16 +3,16 @@ package v6 import ( "context" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/git" "github.com/weaveworks/flux/job" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/ssh" "github.com/weaveworks/flux/update" ) type ImageStatus struct { - ID flux.ResourceID + ID resource.ID Containers []Container } @@ -31,13 +31,13 @@ const ( ) type ControllerStatus struct { - ID flux.ResourceID + ID resource.ID Containers []Container ReadOnly ReadOnlyReason Status string Rollout cluster.RolloutStatus SyncError string - Antecedent flux.ResourceID + Antecedent resource.ID Labels map[string]string Automated bool Locked bool diff --git a/cluster/cluster.go b/cluster/cluster.go index 5fedba272..e8d203a67 100644 --- a/cluster/cluster.go +++ b/cluster/cluster.go @@ -4,7 +4,6 @@ import ( "context" "errors" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/ssh" @@ -27,8 +26,8 @@ const ( type Cluster interface { // Get all of the services (optionally, from a specific namespace), excluding those AllWorkloads(ctx context.Context, maybeNamespace string) ([]Workload, error) - SomeWorkloads(ctx context.Context, ids []flux.ResourceID) ([]Workload, error) - IsAllowedResource(flux.ResourceID) bool + SomeWorkloads(ctx context.Context, ids []resource.ID) ([]Workload, error) + IsAllowedResource(resource.ID) bool Ping() error Export(ctx context.Context) ([]byte, error) Sync(SyncSet) error @@ -60,7 +59,7 @@ type RolloutStatus struct { // Workload describes a cluster resource that declares versioned images. type Workload struct { - ID flux.ResourceID + ID resource.ID Status string // A status summary for display // Is the controller considered read-only because it's under the // control of the platform. In the case of Kubernetes, we simply @@ -70,7 +69,7 @@ type Workload struct { // resource through some mechanism (like an operator, or custom // resource controller), we try to record the ID of that resource // in this field. - Antecedent flux.ResourceID + Antecedent resource.ID Labels map[string]string Policies policy.Set Rollout RolloutStatus diff --git a/cluster/kubernetes/images.go b/cluster/kubernetes/images.go index e4fdcc912..89196fe10 100644 --- a/cluster/kubernetes/images.go +++ b/cluster/kubernetes/images.go @@ -11,9 +11,9 @@ import ( apierrors "k8s.io/apimachinery/pkg/api/errors" meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/registry" + "github.com/weaveworks/flux/resource" ) func mergeCredentials(log func(...interface{}) error, @@ -145,7 +145,7 @@ func (c *Cluster) ImagesToFetch() registry.ImageCreds { imageCreds := make(registry.ImageCreds) for _, workload := range workloads { - logger := log.With(c.logger, "resource", flux.MakeResourceID(ns.Name, kind, workload.GetName())) + logger := log.With(c.logger, "resource", resource.MakeID(ns.Name, kind, workload.GetName())) mergeCredentials(logger.Log, c.includeImage, c.client, ns.Name, workload.podTemplate, imageCreds, seenCreds) } diff --git a/cluster/kubernetes/kubernetes.go b/cluster/kubernetes/kubernetes.go index 188f6b0d2..6547f4b91 100644 --- a/cluster/kubernetes/kubernetes.go +++ b/cluster/kubernetes/kubernetes.go @@ -17,11 +17,11 @@ import ( k8sclientdynamic "k8s.io/client-go/dynamic" k8sclient "k8s.io/client-go/kubernetes" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" - "github.com/weaveworks/flux/cluster/kubernetes/resource" + kresource "github.com/weaveworks/flux/cluster/kubernetes/resource" fhrclient "github.com/weaveworks/flux/integrations/client/clientset/versioned" "github.com/weaveworks/flux/ssh" + "github.com/weaveworks/flux/resource" ) type coreClient k8sclient.Interface @@ -98,7 +98,7 @@ type Cluster struct { // syncErrors keeps a record of all per-resource errors during // the sync from Git repo to the cluster. - syncErrors map[flux.ResourceID]error + syncErrors map[resource.ID]error muSyncErrors sync.RWMutex allowedNamespaces []string @@ -128,7 +128,7 @@ func NewCluster(client ExtendedClient, applier Applier, sshKeyRing ssh.KeyRing, // SomeWorkloads returns the workloads named, missing out any that don't // exist in the cluster or aren't in an allowed namespace. // They do not necessarily have to be returned in the order requested. -func (c *Cluster) SomeWorkloads(ctx context.Context, ids []flux.ResourceID) (res []cluster.Workload, err error) { +func (c *Cluster) SomeWorkloads(ctx context.Context, ids []resource.ID) (res []cluster.Workload, err error) { var workloads []cluster.Workload for _, id := range ids { if !c.IsAllowedResource(id) { @@ -192,7 +192,7 @@ func (c *Cluster) AllWorkloads(ctx context.Context, namespace string) (res []clu for _, workload := range workloads { if !isAddon(workload) { - id := flux.MakeResourceID(ns.Name, kind, workload.GetName()) + id := resource.MakeID(ns.Name, kind, workload.GetName()) c.muSyncErrors.RLock() workload.syncError = c.syncErrors[id] c.muSyncErrors.RUnlock() @@ -208,7 +208,7 @@ func (c *Cluster) AllWorkloads(ctx context.Context, namespace string) (res []clu func (c *Cluster) setSyncErrors(errs cluster.SyncError) { c.muSyncErrors.Lock() defer c.muSyncErrors.Unlock() - c.syncErrors = make(map[flux.ResourceID]error) + c.syncErrors = make(map[resource.ID]error) for _, e := range errs { c.syncErrors[e.ResourceID] = e.Error } @@ -317,7 +317,7 @@ func (c *Cluster) getAllowedAndExistingNamespaces(ctx context.Context) ([]apiv1. return namespaces.Items, nil } -func (c *Cluster) IsAllowedResource(id flux.ResourceID) bool { +func (c *Cluster) IsAllowedResource(id resource.ID) bool { if len(c.allowedNamespaces) == 0 { // All resources are allowed when all namespaces are allowed return true @@ -326,7 +326,7 @@ func (c *Cluster) IsAllowedResource(id flux.ResourceID) bool { namespace, kind, name := id.Components() namespaceToCheck := namespace - if namespace == resource.ClusterScope { + if namespace == kresource.ClusterScope { // All cluster-scoped resources (not namespaced) are allowed ... if kind != "namespace" { return true diff --git a/cluster/kubernetes/manifests.go b/cluster/kubernetes/manifests.go index 0dcf28237..26abae33b 100644 --- a/cluster/kubernetes/manifests.go +++ b/cluster/kubernetes/manifests.go @@ -10,7 +10,6 @@ import ( "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1" "k8s.io/apimachinery/pkg/runtime/schema" - "github.com/weaveworks/flux" kresource "github.com/weaveworks/flux/cluster/kubernetes/resource" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/resource" @@ -133,7 +132,7 @@ func (m *manifests) ParseManifest(def []byte, source string) (map[string]resourc return result, nil } -func (m *manifests) SetWorkloadContainerImage(def []byte, id flux.ResourceID, container string, image image.Ref) ([]byte, error) { +func (m *manifests) SetWorkloadContainerImage(def []byte, id resource.ID, container string, image image.Ref) ([]byte, error) { return updateWorkload(def, id, container, image) } diff --git a/cluster/kubernetes/patch.go b/cluster/kubernetes/patch.go index 808f795f5..50307e665 100644 --- a/cluster/kubernetes/patch.go +++ b/cluster/kubernetes/patch.go @@ -15,17 +15,17 @@ import ( "k8s.io/apimachinery/pkg/util/strategicpatch" k8sscheme "k8s.io/client-go/kubernetes/scheme" - "github.com/weaveworks/flux" - "github.com/weaveworks/flux/cluster/kubernetes/resource" + kresource "github.com/weaveworks/flux/cluster/kubernetes/resource" + "github.com/weaveworks/flux/resource" ) func createManifestPatch(originalManifests, modifiedManifests []byte, originalSource, modifiedSource string) ([]byte, error) { - originalResources, err := resource.ParseMultidoc(originalManifests, originalSource) + originalResources, err := kresource.ParseMultidoc(originalManifests, originalSource) if err != nil { fmt.Errorf("cannot parse %s: %s", originalSource, err) } - modifiedResources, err := resource.ParseMultidoc(modifiedManifests, modifiedSource) + modifiedResources, err := kresource.ParseMultidoc(modifiedManifests, modifiedSource) if err != nil { fmt.Errorf("cannot parse %s: %s", modifiedSource, err) } @@ -61,12 +61,12 @@ func createManifestPatch(originalManifests, modifiedManifests []byte, originalSo } func applyManifestPatch(originalManifests, patchManifests []byte, originalSource, patchSource string) ([]byte, error) { - originalResources, err := resource.ParseMultidoc(originalManifests, originalSource) + originalResources, err := kresource.ParseMultidoc(originalManifests, originalSource) if err != nil { return nil, fmt.Errorf("cannot parse %s: %s", originalSource, err) } - patchResources, err := resource.ParseMultidoc(patchManifests, patchSource) + patchResources, err := kresource.ParseMultidoc(patchManifests, patchSource) if err != nil { return nil, fmt.Errorf("cannot parse %s: %s", patchSource, err) } @@ -120,7 +120,7 @@ func getFullScheme() *runtime.Scheme { return fullScheme } -func getPatch(originalManifest resource.KubeManifest, modifiedManifest resource.KubeManifest, scheme *runtime.Scheme) ([]byte, error) { +func getPatch(originalManifest kresource.KubeManifest, modifiedManifest kresource.KubeManifest, scheme *runtime.Scheme) ([]byte, error) { groupVersion, err := schema.ParseGroupVersion(originalManifest.GroupVersion()) if err != nil { return nil, fmt.Errorf("cannot parse groupVersion %q: %s", originalManifest.GroupVersion(), err) @@ -192,7 +192,7 @@ func addIdentifyingData(apiVersion string, kind string, name string, namespace s return obj, err } -func applyPatch(originalManifest, patchManifest resource.KubeManifest, scheme *runtime.Scheme) ([]byte, error) { +func applyPatch(originalManifest, patchManifest kresource.KubeManifest, scheme *runtime.Scheme) ([]byte, error) { groupVersion, err := schema.ParseGroupVersion(originalManifest.GroupVersion()) if err != nil { return nil, fmt.Errorf("cannot parse groupVersion %q: %s", originalManifest.GroupVersion(), err) @@ -227,8 +227,8 @@ func applyPatch(originalManifest, patchManifest resource.KubeManifest, scheme *r return patched, nil } -// resourceID works like Resource.ResourceID() but avoids namespaces, +// resourceID works like Resource.ID() but avoids namespaces, // since they may be incorrect -func resourceID(manifest resource.KubeManifest) flux.ResourceID { - return flux.MakeResourceID(manifest.GetNamespace(), manifest.GetKind(), manifest.GetKind()) +func resourceID(manifest kresource.KubeManifest) resource.ID { + return resource.MakeID(manifest.GetNamespace(), manifest.GetKind(), manifest.GetKind()) } diff --git a/cluster/kubernetes/policies.go b/cluster/kubernetes/policies.go index 0136ee072..5418035d2 100644 --- a/cluster/kubernetes/policies.go +++ b/cluster/kubernetes/policies.go @@ -5,13 +5,11 @@ import ( "github.com/pkg/errors" - "github.com/weaveworks/flux" kresource "github.com/weaveworks/flux/cluster/kubernetes/resource" - "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" ) -func (m *manifests) UpdateWorkloadPolicies(def []byte, id flux.ResourceID, update policy.Update) ([]byte, error) { +func (m *manifests) UpdateWorkloadPolicies(def []byte, id resource.ID, update resource.PolicyUpdate) ([]byte, error) { resources, err := m.ParseManifest(def, "stdin") if err != nil { return nil, err diff --git a/cluster/kubernetes/policies_test.go b/cluster/kubernetes/policies_test.go index 2438a21ec..edc33ebee 100644 --- a/cluster/kubernetes/policies_test.go +++ b/cluster/kubernetes/policies_test.go @@ -9,22 +9,22 @@ import ( "github.com/go-kit/kit/log" "github.com/stretchr/testify/assert" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/policy" + "github.com/weaveworks/flux/resource" ) func TestUpdatePolicies(t *testing.T) { for _, c := range []struct { name string in, out []string - update policy.Update + update resource.PolicyUpdate wantErr bool }{ { name: "adding annotation with others existing", in: []string{"prometheus.io.scrape", "'false'"}, out: []string{"prometheus.io.scrape", "'false'", "flux.weave.works/automated", "'true'"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.Automated: "true"}, }, }, @@ -32,7 +32,7 @@ func TestUpdatePolicies(t *testing.T) { name: "adding annotation when already has annotation", in: []string{"flux.weave.works/automated", "'true'"}, out: []string{"flux.weave.works/automated", "'true'"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.Automated: "true"}, }, }, @@ -40,7 +40,7 @@ func TestUpdatePolicies(t *testing.T) { name: "adding annotation when already has annotation and others", in: []string{"flux.weave.works/automated", "'true'", "prometheus.io.scrape", "'false'"}, out: []string{"flux.weave.works/automated", "'true'", "prometheus.io.scrape", "'false'"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.Automated: "true"}, }, }, @@ -48,7 +48,7 @@ func TestUpdatePolicies(t *testing.T) { name: "adding first annotation", in: nil, out: []string{"flux.weave.works/automated", "'true'"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.Automated: "true"}, }, }, @@ -56,7 +56,7 @@ func TestUpdatePolicies(t *testing.T) { name: "add and remove different annotations at the same time", in: []string{"flux.weave.works/automated", "'true'", "prometheus.io.scrape", "'false'"}, out: []string{"prometheus.io.scrape", "'false'", "flux.weave.works/locked", "'true'"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.Locked: "true"}, Remove: policy.Set{policy.Automated: "true"}, }, @@ -65,7 +65,7 @@ func TestUpdatePolicies(t *testing.T) { name: "remove overrides add for same key", in: nil, out: nil, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.Locked: "true"}, Remove: policy.Set{policy.Locked: "true"}, }, @@ -74,7 +74,7 @@ func TestUpdatePolicies(t *testing.T) { name: "remove annotation with others existing", in: []string{"flux.weave.works/automated", "true", "prometheus.io.scrape", "false"}, out: []string{"prometheus.io.scrape", "false"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Remove: policy.Set{policy.Automated: "true"}, }, }, @@ -82,7 +82,7 @@ func TestUpdatePolicies(t *testing.T) { name: "remove last annotation", in: []string{"flux.weave.works/automated", "true"}, out: nil, - update: policy.Update{ + update: resource.PolicyUpdate{ Remove: policy.Set{policy.Automated: "true"}, }, }, @@ -90,7 +90,7 @@ func TestUpdatePolicies(t *testing.T) { name: "remove annotation with no annotations", in: nil, out: nil, - update: policy.Update{ + update: resource.PolicyUpdate{ Remove: policy.Set{policy.Automated: "true"}, }, }, @@ -98,7 +98,7 @@ func TestUpdatePolicies(t *testing.T) { name: "remove annotation with only others", in: []string{"prometheus.io.scrape", "false"}, out: []string{"prometheus.io.scrape", "false"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Remove: policy.Set{policy.Automated: "true"}, }, }, @@ -106,7 +106,7 @@ func TestUpdatePolicies(t *testing.T) { name: "multiline", in: []string{"flux.weave.works/locked_msg", "|-\n first\n second"}, out: nil, - update: policy.Update{ + update: resource.PolicyUpdate{ Remove: policy.Set{policy.LockedMsg: "foo"}, }, }, @@ -114,7 +114,7 @@ func TestUpdatePolicies(t *testing.T) { name: "multiline with empty line", in: []string{"flux.weave.works/locked_msg", "|-\n first\n\n third"}, out: nil, - update: policy.Update{ + update: resource.PolicyUpdate{ Remove: policy.Set{policy.LockedMsg: "foo"}, }, }, @@ -122,7 +122,7 @@ func TestUpdatePolicies(t *testing.T) { name: "add tag policy", in: nil, out: []string{"flux.weave.works/tag.nginx", "glob:*"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("nginx"): "glob:*"}, }, }, @@ -130,7 +130,7 @@ func TestUpdatePolicies(t *testing.T) { name: "add non-glob tag policy", in: nil, out: []string{"flux.weave.works/tag.nginx", "foo"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("nginx"): "foo"}, }, }, @@ -138,7 +138,7 @@ func TestUpdatePolicies(t *testing.T) { name: "add semver tag policy", in: nil, out: []string{"flux.weave.works/tag.nginx", "semver:*"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("nginx"): "semver:*"}, }, }, @@ -146,7 +146,7 @@ func TestUpdatePolicies(t *testing.T) { name: "add invalid semver tag policy", in: nil, out: []string{"flux.weave.works/tag.nginx", "semver:*"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("nginx"): "semver:invalid"}, }, wantErr: true, @@ -155,7 +155,7 @@ func TestUpdatePolicies(t *testing.T) { name: "add regexp tag policy", in: nil, out: []string{"flux.weave.works/tag.nginx", "regexp:(.*?)"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("nginx"): "regexp:(.*?)"}, }, }, @@ -163,7 +163,7 @@ func TestUpdatePolicies(t *testing.T) { name: "add invalid regexp tag policy", in: nil, out: []string{"flux.weave.works/tag.nginx", "regexp:(.*?)"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("nginx"): "regexp:*"}, }, wantErr: true, @@ -172,7 +172,7 @@ func TestUpdatePolicies(t *testing.T) { name: "set tag to all containers", in: nil, out: []string{"flux.weave.works/tag.nginx", "semver:*"}, - update: policy.Update{ + update: resource.PolicyUpdate{ Add: policy.Set{policy.TagAll: "semver:*"}, }, }, @@ -180,7 +180,7 @@ func TestUpdatePolicies(t *testing.T) { t.Run(c.name, func(t *testing.T) { caseIn := templToString(t, annotationsTemplate, c.in) caseOut := templToString(t, annotationsTemplate, c.out) - resourceID := flux.MustParseResourceID("default:deployment/nginx") + resourceID := resource.MustParseID("default:deployment/nginx") manifests := NewManifests(ConstNamespacer("default"), log.NewLogfmtLogger(os.Stdout)) out, err := manifests.UpdateWorkloadPolicies([]byte(caseIn), resourceID, c.update) assert.Equal(t, c.wantErr, err != nil, "unexpected error value: %s", err) @@ -192,8 +192,8 @@ func TestUpdatePolicies(t *testing.T) { } func TestUpdatePolicies_invalidTagPattern(t *testing.T) { - resourceID := flux.MustParseResourceID("default:deployment/nginx") - update := policy.Update{ + resourceID := resource.MustParseID("default:deployment/nginx") + update := resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("nginx"): "semver:invalid"}, } _, err := (&manifests{}).UpdateWorkloadPolicies(nil, resourceID, update) diff --git a/cluster/kubernetes/resource/load_test.go b/cluster/kubernetes/resource/load_test.go index ffffbe79f..9d42dcdb3 100644 --- a/cluster/kubernetes/resource/load_test.go +++ b/cluster/kubernetes/resource/load_test.go @@ -8,7 +8,6 @@ import ( "github.com/stretchr/testify/assert" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster/kubernetes/testfiles" "github.com/weaveworks/flux/resource" ) @@ -211,9 +210,9 @@ items: if len(list.Items) != 2 { t.Fatalf("expected two items, got %+v", list.Items) } - for i, id := range []flux.ResourceID{ - flux.MustParseResourceID("ns:deployment/foo"), - flux.MustParseResourceID("ns:service/bar")} { + for i, id := range []resource.ID{ + resource.MustParseID("ns:deployment/foo"), + resource.MustParseID("ns:service/bar")} { if list.Items[i].ResourceID() != id { t.Errorf("At %d, expected %q, got %q", i, id, list.Items[i].ResourceID()) } @@ -246,9 +245,9 @@ items: if len(list.Items) != 2 { t.Fatalf("expected two items, got %+v", list.Items) } - for i, id := range []flux.ResourceID{ - flux.MustParseResourceID("ns:deployment/foo"), - flux.MustParseResourceID("ns:deployment/bar")} { + for i, id := range []resource.ID{ + resource.MustParseID("ns:deployment/foo"), + resource.MustParseID("ns:deployment/bar")} { if list.Items[i].ResourceID() != id { t.Errorf("At %d, expected %q, got %q", i, id, list.Items[i].ResourceID()) } diff --git a/cluster/kubernetes/resource/resource.go b/cluster/kubernetes/resource/resource.go index 4599a9a1b..2a010dec7 100644 --- a/cluster/kubernetes/resource/resource.go +++ b/cluster/kubernetes/resource/resource.go @@ -3,9 +3,8 @@ package resource import ( "strings" - yaml "gopkg.in/yaml.v2" + "gopkg.in/yaml.v2" - "github.com/weaveworks/flux" fluxerr "github.com/weaveworks/flux/errors" "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" @@ -65,12 +64,12 @@ func (o baseObject) GetName() string { return o.Meta.Name } -func (o baseObject) ResourceID() flux.ResourceID { +func (o baseObject) ResourceID() resource.ID { ns := o.Meta.Namespace if ns == "" { ns = ClusterScope } - return flux.MakeResourceID(ns, o.Kind, o.Meta.Name) + return resource.MakeID(ns, o.Kind, o.Meta.Name) } // SetNamespace implements KubeManifest.SetNamespace, so things with diff --git a/cluster/kubernetes/resourcekinds.go b/cluster/kubernetes/resourcekinds.go index a7416c64f..9905b7733 100644 --- a/cluster/kubernetes/resourcekinds.go +++ b/cluster/kubernetes/resourcekinds.go @@ -9,7 +9,6 @@ import ( apiv1 "k8s.io/api/core/v1" meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" kresource "github.com/weaveworks/flux/cluster/kubernetes/resource" "github.com/weaveworks/flux/image" @@ -24,7 +23,7 @@ import ( // FluxHelmRelease. We use this rather than the `OwnerReference` type // built into Kubernetes so that there are no garbage-collection // implications. The value is expected to be a serialised -// `flux.ResourceID`. +// `resource.ID`. const AntecedentAnnotation = "flux.weave.works/antecedent" ///////////////////////////////////////////////////////////////////////////// @@ -56,7 +55,7 @@ type workload struct { podTemplate apiv1.PodTemplateSpec } -func (w workload) toClusterWorkload(resourceID flux.ResourceID) cluster.Workload { +func (w workload) toClusterWorkload(resourceID resource.ID) cluster.Workload { var clusterContainers []resource.Container var excuse string for _, container := range w.podTemplate.Spec.Containers { @@ -78,9 +77,9 @@ func (w workload) toClusterWorkload(resourceID flux.ResourceID) cluster.Workload clusterContainers = append(clusterContainers, resource.Container{Name: container.Name, Image: ref}) } - var antecedent flux.ResourceID + var antecedent resource.ID if ante, ok := w.GetAnnotations()[AntecedentAnnotation]; ok { - id, err := flux.ParseResourceID(ante) + id, err := resource.ParseID(ante) if err == nil { antecedent = id } diff --git a/cluster/kubernetes/sync.go b/cluster/kubernetes/sync.go index 502de85fe..33a70e3f0 100644 --- a/cluster/kubernetes/sync.go +++ b/cluster/kubernetes/sync.go @@ -25,7 +25,6 @@ import ( "k8s.io/client-go/discovery" "k8s.io/client-go/rest" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" kresource "github.com/weaveworks/flux/cluster/kubernetes/resource" "github.com/weaveworks/flux/policy" @@ -169,14 +168,14 @@ type kuberesource struct { namespaced bool } -// ResourceID returns the ResourceID for this resource loaded from the +// ResourceID returns the ID for this resource loaded from the // cluster. -func (r *kuberesource) ResourceID() flux.ResourceID { +func (r *kuberesource) ResourceID() resource.ID { ns, kind, name := r.obj.GetNamespace(), r.obj.GetKind(), r.obj.GetName() if !r.namespaced { ns = kresource.ClusterScope } - return flux.MakeResourceID(ns, kind, name) + return resource.MakeID(ns, kind, name) } // Bytes returns a byte slice description, including enough info to @@ -368,7 +367,7 @@ func makeGCMark(syncSetName, resourceID string) string { // --- internal types for keeping track of syncing type applyObject struct { - ResourceID flux.ResourceID + ResourceID resource.ID Source string Payload []byte } @@ -381,13 +380,13 @@ func makeChangeSet() changeSet { return changeSet{objs: make(map[string][]applyObject)} } -func (c *changeSet) stage(cmd string, id flux.ResourceID, source string, bytes []byte) { +func (c *changeSet) stage(cmd string, id resource.ID, source string, bytes []byte) { c.objs[cmd] = append(c.objs[cmd], applyObject{id, source, bytes}) } // Applier is something that will apply a changeset to the cluster. type Applier interface { - apply(log.Logger, changeSet, map[flux.ResourceID]error) cluster.SyncError + apply(log.Logger, changeSet, map[resource.ID]error) cluster.SyncError } type Kubectl struct { @@ -472,7 +471,7 @@ func (objs applyOrder) Less(i, j int) bool { return ranki < rankj } -func (c *Kubectl) apply(logger log.Logger, cs changeSet, errored map[flux.ResourceID]error) (errs cluster.SyncError) { +func (c *Kubectl) apply(logger log.Logger, cs changeSet, errored map[resource.ID]error) (errs cluster.SyncError) { f := func(objs []applyObject, cmd string, args ...string) { if len(objs) == 0 { return diff --git a/cluster/kubernetes/sync_test.go b/cluster/kubernetes/sync_test.go index e83e297fd..349d019bc 100644 --- a/cluster/kubernetes/sync_test.go +++ b/cluster/kubernetes/sync_test.go @@ -9,26 +9,21 @@ import ( "github.com/ghodss/yaml" "github.com/go-kit/kit/log" + "github.com/stretchr/testify/assert" corev1 "k8s.io/api/core/v1" + crdfake "k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset/fake" "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" - // "k8s.io/apimachinery/pkg/runtime/serializer" - // "k8s.io/apimachinery/pkg/watch" - "k8s.io/client-go/dynamic" - // dynamicfake "k8s.io/client-go/dynamic/fake" - // k8sclient "k8s.io/client-go/kubernetes" - "github.com/stretchr/testify/assert" - crdfake "k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset/fake" "k8s.io/client-go/discovery" + "k8s.io/client-go/dynamic" dynamicfake "k8s.io/client-go/dynamic/fake" k8sclient "k8s.io/client-go/kubernetes" corefake "k8s.io/client-go/kubernetes/fake" k8s_testing "k8s.io/client-go/testing" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" kresource "github.com/weaveworks/flux/cluster/kubernetes/resource" fluxfake "github.com/weaveworks/flux/integrations/client/clientset/versioned/fake" @@ -116,7 +111,7 @@ func groupVersionResource(res *unstructured.Unstructured) schema.GroupVersionRes return schema.GroupVersionResource{Group: gvk.Group, Version: gvk.Version, Resource: strings.ToLower(gvk.Kind) + "s"} } -func (a fakeApplier) apply(_ log.Logger, cs changeSet, errored map[flux.ResourceID]error) cluster.SyncError { +func (a fakeApplier) apply(_ log.Logger, cs changeSet, errored map[resource.ID]error) cluster.SyncError { var errs []cluster.ResourceError operate := func(obj applyObject, cmd string) { @@ -730,9 +725,9 @@ spec: // TestApplyOrder checks that applyOrder works as expected. func TestApplyOrder(t *testing.T) { objs := []applyObject{ - {ResourceID: flux.MakeResourceID("test", "Deployment", "deploy")}, - {ResourceID: flux.MakeResourceID("test", "Secret", "secret")}, - {ResourceID: flux.MakeResourceID("", "Namespace", "namespace")}, + {ResourceID: resource.MakeID("test", "Deployment", "deploy")}, + {ResourceID: resource.MakeID("test", "Secret", "secret")}, + {ResourceID: resource.MakeID("", "Namespace", "namespace")}, } sort.Sort(applyOrder(objs)) for i, name := range []string{"namespace", "secret", "deploy"} { diff --git a/cluster/kubernetes/testfiles/data.go b/cluster/kubernetes/testfiles/data.go index b47ce105c..3504bd3f2 100644 --- a/cluster/kubernetes/testfiles/data.go +++ b/cluster/kubernetes/testfiles/data.go @@ -7,7 +7,7 @@ import ( "strings" "testing" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/resource" ) func TempDir(t *testing.T) (string, func()) { @@ -44,30 +44,30 @@ func WriteTestFiles(dir string) error { // ResourceMap is the map of resource names to relative paths, which // must correspond with `Files` below. -var ResourceMap = map[flux.ResourceID]string{ - flux.MustParseResourceID("default:deployment/helloworld"): "helloworld-deploy.yaml", - flux.MustParseResourceID("default:deployment/locked-service"): "locked-service-deploy.yaml", - flux.MustParseResourceID("default:deployment/test-service"): "test/test-service-deploy.yaml", - flux.MustParseResourceID("default:deployment/multi-deploy"): "multi.yaml", - flux.MustParseResourceID("default:service/multi-service"): "multi.yaml", - flux.MustParseResourceID("default:deployment/list-deploy"): "list.yaml", - flux.MustParseResourceID("default:service/list-service"): "list.yaml", - flux.MustParseResourceID("default:deployment/semver"): "semver-deploy.yaml", - flux.MustParseResourceID("default:daemonset/init"): "init.yaml", +var ResourceMap = map[resource.ID]string{ + resource.MustParseID("default:deployment/helloworld"): "helloworld-deploy.yaml", + resource.MustParseID("default:deployment/locked-service"): "locked-service-deploy.yaml", + resource.MustParseID("default:deployment/test-service"): "test/test-service-deploy.yaml", + resource.MustParseID("default:deployment/multi-deploy"): "multi.yaml", + resource.MustParseID("default:service/multi-service"): "multi.yaml", + resource.MustParseID("default:deployment/list-deploy"): "list.yaml", + resource.MustParseID("default:service/list-service"): "list.yaml", + resource.MustParseID("default:deployment/semver"): "semver-deploy.yaml", + resource.MustParseID("default:daemonset/init"): "init.yaml", } // WorkloadMap ... given a base path, construct the map representing // the services given in the test data. Must be kept in sync with // `Files` below. TODO(michael): derive from ResourceMap, or similar. -func WorkloadMap(dir string) map[flux.ResourceID][]string { - return map[flux.ResourceID][]string{ - flux.MustParseResourceID("default:deployment/helloworld"): []string{filepath.Join(dir, "helloworld-deploy.yaml")}, - flux.MustParseResourceID("default:deployment/locked-service"): []string{filepath.Join(dir, "locked-service-deploy.yaml")}, - flux.MustParseResourceID("default:deployment/test-service"): []string{filepath.Join(dir, "test/test-service-deploy.yaml")}, - flux.MustParseResourceID("default:deployment/multi-deploy"): []string{filepath.Join(dir, "multi.yaml")}, - flux.MustParseResourceID("default:deployment/list-deploy"): []string{filepath.Join(dir, "list.yaml")}, - flux.MustParseResourceID("default:deployment/semver"): []string{filepath.Join(dir, "semver-deploy.yaml")}, - flux.MustParseResourceID("default:daemonset/init"): []string{filepath.Join(dir, "init.yaml")}, +func WorkloadMap(dir string) map[resource.ID][]string { + return map[resource.ID][]string{ + resource.MustParseID("default:deployment/helloworld"): []string{filepath.Join(dir, "helloworld-deploy.yaml")}, + resource.MustParseID("default:deployment/locked-service"): []string{filepath.Join(dir, "locked-service-deploy.yaml")}, + resource.MustParseID("default:deployment/test-service"): []string{filepath.Join(dir, "test/test-service-deploy.yaml")}, + resource.MustParseID("default:deployment/multi-deploy"): []string{filepath.Join(dir, "multi.yaml")}, + resource.MustParseID("default:deployment/list-deploy"): []string{filepath.Join(dir, "list.yaml")}, + resource.MustParseID("default:deployment/semver"): []string{filepath.Join(dir, "semver-deploy.yaml")}, + resource.MustParseID("default:daemonset/init"): []string{filepath.Join(dir, "init.yaml")}, } } diff --git a/cluster/kubernetes/update.go b/cluster/kubernetes/update.go index d2ac4debf..eb9a1fdc8 100644 --- a/cluster/kubernetes/update.go +++ b/cluster/kubernetes/update.go @@ -3,8 +3,8 @@ package kubernetes import ( "strings" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" + "github.com/weaveworks/flux/resource" ) // updateWorkload takes a YAML document stream (one or more YAML @@ -12,7 +12,7 @@ import ( // container name, and the name of the new image that should be used // for the container. It returns a new YAML stream where the image for // the container has been replaced with the imageRef supplied. -func updateWorkload(in []byte, resource flux.ResourceID, container string, newImageID image.Ref) ([]byte, error) { +func updateWorkload(in []byte, resource resource.ID, container string, newImageID image.Ref) ([]byte, error) { namespace, kind, name := resource.Components() if _, ok := resourceKinds[strings.ToLower(kind)]; !ok { return nil, UpdateNotSupportedError(kind) diff --git a/cluster/kubernetes/update_test.go b/cluster/kubernetes/update_test.go index caacf09ea..ea2ed46e8 100644 --- a/cluster/kubernetes/update_test.go +++ b/cluster/kubernetes/update_test.go @@ -3,8 +3,8 @@ package kubernetes import ( "testing" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" + "github.com/weaveworks/flux/resource" ) type update struct { @@ -25,7 +25,7 @@ func testUpdate(t *testing.T, u update) { for _, container := range u.containers { var out []byte var err error - if out, err = updateWorkload([]byte(manifest), flux.MustParseResourceID(u.resourceID), container, id); err != nil { + if out, err = updateWorkload([]byte(manifest), resource.MustParseID(u.resourceID), container, id); err != nil { t.Errorf("Failed: %s", err.Error()) return } diff --git a/cluster/mock/mock.go b/cluster/mock/mock.go index a16633fd0..96aba0406 100644 --- a/cluster/mock/mock.go +++ b/cluster/mock/mock.go @@ -4,11 +4,9 @@ import ( "bytes" "context" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/manifests" - "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/ssh" ) @@ -16,16 +14,16 @@ import ( // Doubles as a cluster.Cluster and cluster.Manifests implementation type Mock struct { AllWorkloadsFunc func(ctx context.Context, maybeNamespace string) ([]cluster.Workload, error) - SomeWorkloadsFunc func(ctx context.Context, ids []flux.ResourceID) ([]cluster.Workload, error) - IsAllowedResourceFunc func(flux.ResourceID) bool + SomeWorkloadsFunc func(ctx context.Context, ids []resource.ID) ([]cluster.Workload, error) + IsAllowedResourceFunc func(resource.ID) bool PingFunc func() error ExportFunc func(ctx context.Context) ([]byte, error) SyncFunc func(cluster.SyncSet) error PublicSSHKeyFunc func(regenerate bool) (ssh.PublicKey, error) - SetWorkloadContainerImageFunc func(def []byte, id flux.ResourceID, container string, newImageID image.Ref) ([]byte, error) + SetWorkloadContainerImageFunc func(def []byte, id resource.ID, container string, newImageID image.Ref) ([]byte, error) LoadManifestsFunc func(base string, paths []string) (map[string]resource.Resource, error) ParseManifestFunc func(def []byte, source string) (map[string]resource.Resource, error) - UpdateWorkloadPoliciesFunc func([]byte, flux.ResourceID, policy.Update) ([]byte, error) + UpdateWorkloadPoliciesFunc func([]byte, resource.ID, resource.PolicyUpdate) ([]byte, error) CreateManifestPatchFunc func(originalManifests, modifiedManifests []byte, originalSource, modifiedSource string) ([]byte, error) ApplyManifestPatchFunc func(originalManifests, patch []byte, originalSource, patchSource string) ([]byte, error) AppendManifestToBufferFunc func([]byte, *bytes.Buffer) error @@ -38,11 +36,11 @@ func (m *Mock) AllWorkloads(ctx context.Context, maybeNamespace string) ([]clust return m.AllWorkloadsFunc(ctx, maybeNamespace) } -func (m *Mock) SomeWorkloads(ctx context.Context, ids []flux.ResourceID) ([]cluster.Workload, error) { +func (m *Mock) SomeWorkloads(ctx context.Context, ids []resource.ID) ([]cluster.Workload, error) { return m.SomeWorkloadsFunc(ctx, ids) } -func (m *Mock) IsAllowedResource(id flux.ResourceID) bool { +func (m *Mock) IsAllowedResource(id resource.ID) bool { return m.IsAllowedResourceFunc(id) } @@ -62,7 +60,7 @@ func (m *Mock) PublicSSHKey(regenerate bool) (ssh.PublicKey, error) { return m.PublicSSHKeyFunc(regenerate) } -func (m *Mock) SetWorkloadContainerImage(def []byte, id flux.ResourceID, container string, newImageID image.Ref) ([]byte, error) { +func (m *Mock) SetWorkloadContainerImage(def []byte, id resource.ID, container string, newImageID image.Ref) ([]byte, error) { return m.SetWorkloadContainerImageFunc(def, id, container, newImageID) } @@ -74,7 +72,7 @@ func (m *Mock) ParseManifest(def []byte, source string) (map[string]resource.Res return m.ParseManifestFunc(def, source) } -func (m *Mock) UpdateWorkloadPolicies(def []byte, id flux.ResourceID, p policy.Update) ([]byte, error) { +func (m *Mock) UpdateWorkloadPolicies(def []byte, id resource.ID, p resource.PolicyUpdate) ([]byte, error) { return m.UpdateWorkloadPoliciesFunc(def, id, p) } diff --git a/cluster/sync.go b/cluster/sync.go index ebe3aab9e..2638302d4 100644 --- a/cluster/sync.go +++ b/cluster/sync.go @@ -3,7 +3,6 @@ package cluster import ( "strings" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/resource" ) @@ -22,7 +21,7 @@ type SyncSet struct { } type ResourceError struct { - ResourceID flux.ResourceID + ResourceID resource.ID Source string Error error } diff --git a/cmd/fluxctl/args.go b/cmd/fluxctl/args.go index 2c864c523..b1fc7916b 100644 --- a/cmd/fluxctl/args.go +++ b/cmd/fluxctl/args.go @@ -3,9 +3,9 @@ package main import ( "bytes" "fmt" - "io/ioutil" + "io/ioutil" "os/exec" - "strings" + "strings" "github.com/spf13/cobra" @@ -35,6 +35,7 @@ func getCommitAuthor() string { } var execCommand = exec.Command + func getUserGitConfigValue(arg string) string { var out bytes.Buffer cmd := execCommand("git", "config", "--get", "--null", arg) diff --git a/cmd/fluxctl/args_test.go b/cmd/fluxctl/args_test.go index 1c634f964..9b6385daf 100644 --- a/cmd/fluxctl/args_test.go +++ b/cmd/fluxctl/args_test.go @@ -1,10 +1,10 @@ package main import ( - "fmt" - "testing" + "fmt" "os" "os/exec" + "testing" ) func helperCommand(command string, s ...string) (cmd *exec.Cmd) { @@ -21,7 +21,6 @@ func TestHelperProcess(t *testing.T) { } defer os.Exit(0) - args := os.Args for len(args) > 0 { if args[0] == "--" { @@ -35,18 +34,18 @@ func TestHelperProcess(t *testing.T) { } _, args = args[0], args[1:] - for _, a := range args { - if a == "user.name" { - fmt.Fprintf(os.Stdout, "Jane Doe") - } else if a == "user.email" { - fmt.Fprintf(os.Stdout, "jd@j.d") - } - } + for _, a := range args { + if a == "user.name" { + fmt.Fprintf(os.Stdout, "Jane Doe") + } else if a == "user.email" { + fmt.Fprintf(os.Stdout, "jd@j.d") + } + } } func checkAuthor(t *testing.T, input string, expected string) { - execCommand = helperCommand - defer func(){ execCommand = exec.Command }() + execCommand = helperCommand + defer func() { execCommand = exec.Command }() author := getUserGitConfigValue(input) if author != expected { t.Fatalf("author %q does not match expected value %q", author, expected) diff --git a/cmd/fluxctl/list_images_cmd.go b/cmd/fluxctl/list_images_cmd.go index 5fbbba7d6..07b5ed9d2 100644 --- a/cmd/fluxctl/list_images_cmd.go +++ b/cmd/fluxctl/list_images_cmd.go @@ -8,10 +8,10 @@ import ( "github.com/spf13/cobra" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/api/v10" "github.com/weaveworks/flux/api/v6" "github.com/weaveworks/flux/registry" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -64,7 +64,7 @@ func (opts *imageListOpts) RunE(cmd *cobra.Command, args []string) error { opts.workload = opts.controller } if len(opts.workload) > 0 { - id, err := flux.ParseResourceIDOptionalNamespace(opts.namespace, opts.workload) + id, err := resource.ParseIDOptionalNamespace(opts.namespace, opts.workload) if err != nil { return err } diff --git a/cmd/fluxctl/policy_cmd.go b/cmd/fluxctl/policy_cmd.go index ba871186b..64d6f5f6f 100644 --- a/cmd/fluxctl/policy_cmd.go +++ b/cmd/fluxctl/policy_cmd.go @@ -6,8 +6,9 @@ import ( "strings" "github.com/spf13/cobra" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/policy" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -97,7 +98,7 @@ func (opts *workloadPolicyOpts) RunE(cmd *cobra.Command, args []string) error { return newUsageError("lock and unlock both specified") } - resourceID, err := flux.ParseResourceIDOptionalNamespace(opts.namespace, opts.workload) + resourceID, err := resource.ParseIDOptionalNamespace(opts.namespace, opts.workload) if err != nil { return err } @@ -108,7 +109,7 @@ func (opts *workloadPolicyOpts) RunE(cmd *cobra.Command, args []string) error { } ctx := context.Background() - updates := policy.Updates{ + updates := resource.PolicyUpdates{ resourceID: changes, } jobID, err := opts.API.UpdateManifests(ctx, update.Spec{ @@ -122,7 +123,7 @@ func (opts *workloadPolicyOpts) RunE(cmd *cobra.Command, args []string) error { return await(ctx, cmd.OutOrStdout(), cmd.OutOrStderr(), opts.API, jobID, false, opts.verbosity) } -func calculatePolicyChanges(opts *workloadPolicyOpts) (policy.Update, error) { +func calculatePolicyChanges(opts *workloadPolicyOpts) (resource.PolicyUpdate, error) { add := policy.Set{} if opts.automate { add = add.Add(policy.Automated) @@ -153,7 +154,7 @@ func calculatePolicyChanges(opts *workloadPolicyOpts) (policy.Update, error) { for _, tagPair := range opts.tags { parts := strings.Split(tagPair, "=") if len(parts) != 2 { - return policy.Update{}, fmt.Errorf("invalid container/tag pair: %q. Expected format is 'container=filter'", tagPair) + return resource.PolicyUpdate{}, fmt.Errorf("invalid container/tag pair: %q. Expected format is 'container=filter'", tagPair) } container, tag := parts[0], parts[1] @@ -164,7 +165,7 @@ func calculatePolicyChanges(opts *workloadPolicyOpts) (policy.Update, error) { } } - return policy.Update{ + return resource.PolicyUpdate{ Add: add, Remove: remove, }, nil diff --git a/cmd/fluxctl/release_cmd.go b/cmd/fluxctl/release_cmd.go index 4be590747..93d8f8916 100644 --- a/cmd/fluxctl/release_cmd.go +++ b/cmd/fluxctl/release_cmd.go @@ -8,11 +8,11 @@ import ( "github.com/spf13/cobra" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/api/v11" "github.com/weaveworks/flux/api/v6" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/job" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -103,7 +103,7 @@ func (opts *workloadReleaseOpts) RunE(cmd *cobra.Command, args []string) error { workloads = []update.ResourceSpec{update.ResourceSpecAll} } else { for _, workload := range opts.workloads { - id, err := flux.ParseResourceIDOptionalNamespace(opts.namespace, workload) + id, err := resource.ParseIDOptionalNamespace(opts.namespace, workload) if err != nil { return err } @@ -130,9 +130,9 @@ func (opts *workloadReleaseOpts) RunE(cmd *cobra.Command, args []string) error { kind = update.ReleaseKindPlan } - var excludes []flux.ResourceID + var excludes []resource.ID for _, exclude := range opts.exclude { - s, err := flux.ParseResourceIDOptionalNamespace(opts.namespace, exclude) + s, err := resource.ParseIDOptionalNamespace(opts.namespace, exclude) if err != nil { return err } diff --git a/cmd/fluxctl/release_cmd_test.go b/cmd/fluxctl/release_cmd_test.go index 472320488..3eb72ff26 100644 --- a/cmd/fluxctl/release_cmd_test.go +++ b/cmd/fluxctl/release_cmd_test.go @@ -5,7 +5,7 @@ import ( "reflect" "testing" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -38,9 +38,9 @@ func TestReleaseCommand_CLIConversion(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{ - flux.MustParseResourceID("default:deployment/test"), - flux.MustParseResourceID("default:deployment/yeah"), + Excludes: []resource.ID{ + resource.MustParseID("default:deployment/test"), + resource.MustParseID("default:deployment/yeah"), }, }}, } { diff --git a/daemon/daemon.go b/daemon/daemon.go index eee5991b4..e7189ea2a 100644 --- a/daemon/daemon.go +++ b/daemon/daemon.go @@ -11,7 +11,6 @@ import ( "github.com/go-kit/kit/log" "github.com/pkg/errors" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/api" "github.com/weaveworks/flux/api/v10" "github.com/weaveworks/flux/api/v11" @@ -199,7 +198,7 @@ func (d *Daemon) ListImagesWithOptions(ctx context.Context, opts v10.ListImagesO if err != nil { return nil, errors.Wrap(err, "treating workload spec as ID") } - workloads, err = d.Cluster.SomeWorkloads(ctx, []flux.ResourceID{id}) + workloads, err = d.Cluster.SomeWorkloads(ctx, []resource.ID{id}) if err != nil { return nil, errors.Wrap(err, "getting some workloads") } @@ -290,7 +289,7 @@ func (d *Daemon) makeLoggingJobFunc(f jobFunc) jobFunc { } logger.Log("revision", result.Revision) if result.Revision != "" { - var workloadIDs []flux.ResourceID + var workloadIDs []resource.ID for id, result := range result.Result { if result.Status == update.ReleaseStatusSuccess { workloadIDs = append(workloadIDs, id) @@ -350,7 +349,7 @@ func (d *Daemon) UpdateManifests(ctx context.Context, spec update.Spec) (job.ID, return id, err } return d.queueJob(d.makeLoggingJobFunc(d.makeJobFromUpdate(d.release(spec, s)))), nil - case policy.Updates: + case resource.PolicyUpdates: return d.queueJob(d.makeLoggingJobFunc(d.makeJobFromUpdate(d.updatePolicies(spec, s)))), nil case update.ManualSync: return d.queueJob(d.sync()), nil @@ -390,10 +389,10 @@ func (d *Daemon) sync() jobFunc { } } -func (d *Daemon) updatePolicies(spec update.Spec, updates policy.Updates) updateFunc { +func (d *Daemon) updatePolicies(spec update.Spec, updates resource.PolicyUpdates) updateFunc { return func(ctx context.Context, jobID job.ID, working *git.Checkout, logger log.Logger) (job.Result, error) { // For each update - var workloadIDs []flux.ResourceID + var workloadIDs []resource.ID result := job.Result{ Spec: &spec, Result: update.Result{}, @@ -704,7 +703,7 @@ func getWorkloadContainers(workload cluster.Workload, imageRepos update.ImageRep return res, nil } -func policyCommitMessage(us policy.Updates, cause update.Cause) string { +func policyCommitMessage(us resource.PolicyUpdates, cause update.Cause) string { // shortcut, since we want roughly the same information events := policyEvents(us, time.Now()) commitMsg := &bytes.Buffer{} @@ -727,14 +726,14 @@ func policyCommitMessage(us policy.Updates, cause update.Cause) string { // policyEvents builds a map of events (by type), for all the events in this set of // updates. There will be one event per type, containing all workload ids // affected by that event. e.g. all automated workload will share an event. -func policyEvents(us policy.Updates, now time.Time) map[string]event.Event { +func policyEvents(us resource.PolicyUpdates, now time.Time) map[string]event.Event { eventsByType := map[string]event.Event{} for workloadID, update := range us { for _, eventType := range policyEventTypes(update) { e, ok := eventsByType[eventType] if !ok { e = event.Event{ - ServiceIDs: []flux.ResourceID{}, + ServiceIDs: []resource.ID{}, Type: eventType, StartedAt: now, EndedAt: now, @@ -749,7 +748,7 @@ func policyEvents(us policy.Updates, now time.Time) map[string]event.Event { } // policyEventTypes is a deduped list of all event types this update contains -func policyEventTypes(u policy.Update) []string { +func policyEventTypes(u resource.PolicyUpdate) []string { types := map[string]struct{}{} for p := range u.Add { switch { diff --git a/daemon/daemon_test.go b/daemon/daemon_test.go index bf27c521d..3333b6938 100644 --- a/daemon/daemon_test.go +++ b/daemon/daemon_test.go @@ -14,7 +14,6 @@ import ( "github.com/go-kit/kit/log" "github.com/stretchr/testify/assert" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/api/v10" "github.com/weaveworks/flux/api/v11" "github.com/weaveworks/flux/api/v6" @@ -156,7 +155,7 @@ func TestDaemon_ListWorkloadsWithOptions(t *testing.T) { t.Run("filter id", func(t *testing.T) { s, err := d.ListServicesWithOptions(ctx, v11.ListServicesOptions{ Namespace: "", - Services: []flux.ResourceID{flux.MustParseResourceID(wl)}}) + Services: []resource.ID{resource.MustParseID(wl)}}) if err != nil { t.Fatalf("Error: %s", err.Error()) } @@ -168,7 +167,7 @@ func TestDaemon_ListWorkloadsWithOptions(t *testing.T) { t.Run("filter id and namespace", func(t *testing.T) { _, err := d.ListServicesWithOptions(ctx, v11.ListServicesOptions{ Namespace: "foo", - Services: []flux.ResourceID{flux.MustParseResourceID(wl)}}) + Services: []resource.ID{resource.MustParseID(wl)}}) if err == nil { t.Fatal("Expected error but got nil") } @@ -177,7 +176,7 @@ func TestDaemon_ListWorkloadsWithOptions(t *testing.T) { t.Run("filter unsupported id kind", func(t *testing.T) { _, err := d.ListServicesWithOptions(ctx, v11.ListServicesOptions{ Namespace: "foo", - Services: []flux.ResourceID{flux.MustParseResourceID("default:unsupportedkind/goodbyeworld")}}) + Services: []resource.ID{resource.MustParseID("default:unsupportedkind/goodbyeworld")}}) if err == nil { t.Fatal("Expected error but got nil") } @@ -195,7 +194,7 @@ func TestDaemon_ListImagesWithOptions(t *testing.T) { specAll := update.ResourceSpec(update.ResourceSpecAll) // Service 1 - svcID, err := flux.ParseResourceID(wl) + svcID, err := resource.ParseID(wl) assert.NoError(t, err) currentImageRef, err := image.ParseRef(currentHelloImage) assert.NoError(t, err) @@ -205,7 +204,7 @@ func TestDaemon_ListImagesWithOptions(t *testing.T) { assert.NoError(t, err) // Service 2 - anotherSvcID, err := flux.ParseResourceID(anotherWl) + anotherSvcID, err := resource.ParseID(anotherWl) assert.NoError(t, err) anotherImageRef, err := image.ParseRef(anotherImage) assert.NoError(t, err) @@ -577,7 +576,7 @@ func TestDaemon_Automated(t *testing.T) { w := newWait(t) workload := cluster.Workload{ - ID: flux.MakeResourceID(ns, "deployment", "helloworld"), + ID: resource.MakeID(ns, "deployment", "helloworld"), Containers: cluster.ContainersOrExcuse{ Containers: []resource.Container{ { @@ -587,7 +586,7 @@ func TestDaemon_Automated(t *testing.T) { }, }, } - k8s.SomeWorkloadsFunc = func(ctx context.Context, ids []flux.ResourceID) ([]cluster.Workload, error) { + k8s.SomeWorkloadsFunc = func(ctx context.Context, ids []resource.ID) ([]cluster.Workload, error) { return []cluster.Workload{workload}, nil } start() @@ -601,7 +600,7 @@ func TestDaemon_Automated_semver(t *testing.T) { defer clean() w := newWait(t) - resid := flux.MustParseResourceID("default:deployment/semver") + resid := resource.MustParseID("default:deployment/semver") workload := cluster.Workload{ ID: resid, Containers: cluster.ContainersOrExcuse{ @@ -613,7 +612,7 @@ func TestDaemon_Automated_semver(t *testing.T) { }, }, } - k8s.SomeWorkloadsFunc = func(ctx context.Context, ids []flux.ResourceID) ([]cluster.Workload, error) { + k8s.SomeWorkloadsFunc = func(ctx context.Context, ids []resource.ID) ([]cluster.Workload, error) { return []cluster.Workload{workload}, nil } start() @@ -638,7 +637,7 @@ func mockDaemon(t *testing.T) (*Daemon, func(), func(), *mock.Mock, *mockEventWr logger := log.NewNopLogger() singleService := cluster.Workload{ - ID: flux.MustParseResourceID(wl), + ID: resource.MustParseID(wl), Containers: cluster.ContainersOrExcuse{ Containers: []resource.Container{ { @@ -651,7 +650,7 @@ func mockDaemon(t *testing.T) (*Daemon, func(), func(), *mock.Mock, *mockEventWr multiService := []cluster.Workload{ singleService, { - ID: flux.MakeResourceID("another", "deployment", "service"), + ID: resource.MakeID("another", "deployment", "service"), Containers: cluster.ContainersOrExcuse{ Containers: []resource.Container{ { @@ -685,10 +684,10 @@ func mockDaemon(t *testing.T) (*Daemon, func(), func(), *mock.Mock, *mockEventWr } return []cluster.Workload{}, nil } - k8s.IsAllowedResourceFunc = func(flux.ResourceID) bool { return true } + k8s.IsAllowedResourceFunc = func(resource.ID) bool { return true } k8s.ExportFunc = func(ctx context.Context) ([]byte, error) { return testBytes, nil } k8s.PingFunc = func() error { return nil } - k8s.SomeWorkloadsFunc = func(ctx context.Context, ids []flux.ResourceID) ([]cluster.Workload, error) { + k8s.SomeWorkloadsFunc = func(ctx context.Context, ids []resource.ID) ([]cluster.Workload, error) { return []cluster.Workload{ singleService, }, nil @@ -887,8 +886,8 @@ func updateImage(ctx context.Context, d *Daemon, t *testing.T) job.ID { func updatePolicy(ctx context.Context, t *testing.T, d *Daemon) job.ID { return updateManifest(ctx, t, d, update.Spec{ Type: update.Policy, - Spec: policy.Updates{ - flux.MustParseResourceID("default:deployment/helloworld"): { + Spec: resource.PolicyUpdates{ + resource.MustParseID("default:deployment/helloworld"): { Add: policy.Set{ policy.Locked: "true", }, diff --git a/daemon/errors.go b/daemon/errors.go index 32e095d0e..45a36f2bf 100644 --- a/daemon/errors.go +++ b/daemon/errors.go @@ -4,13 +4,13 @@ import ( "fmt" "sync" - "github.com/weaveworks/flux" fluxerr "github.com/weaveworks/flux/errors" "github.com/weaveworks/flux/job" + "github.com/weaveworks/flux/resource" ) type SyncErrors struct { - errs map[flux.ResourceID]error + errs map[resource.ID]error mu sync.Mutex } diff --git a/daemon/images.go b/daemon/images.go index f32281b1e..b046703d2 100644 --- a/daemon/images.go +++ b/daemon/images.go @@ -7,7 +7,6 @@ import ( "github.com/go-kit/kit/log" "github.com/pkg/errors" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" @@ -48,9 +47,9 @@ func (d *Daemon) pollForNewImages(logger log.Logger) { } } -type resources map[flux.ResourceID]resource.Resource +type resources map[resource.ID]resource.Resource -func (r resources) IDs() (ids []flux.ResourceID) { +func (r resources) IDs() (ids []resource.ID) { for k, _ := range r { ids = append(ids, k) } @@ -66,7 +65,7 @@ func (d *Daemon) getAllowedAutomatedResources(ctx context.Context) (resources, e return nil, err } - result := map[flux.ResourceID]resource.Resource{} + result := map[resource.ID]resource.Resource{} for _, resource := range resources { policies := resource.Policies() if policies.Has(policy.Automated) && !policies.Has(policy.Locked) && !policies.Has(policy.Ignore) { diff --git a/daemon/images_test.go b/daemon/images_test.go index 9fdcfcb5c..4cc89222a 100644 --- a/daemon/images_test.go +++ b/daemon/images_test.go @@ -1,14 +1,14 @@ package daemon import ( - "github.com/weaveworks/flux/policy" "testing" "time" "github.com/go-kit/kit/log" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/image" + "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/registry" registryMock "github.com/weaveworks/flux/registry/mock" "github.com/weaveworks/flux/resource" @@ -32,11 +32,11 @@ const ( ) type candidate struct { - resourceID flux.ResourceID + resourceID resource.ID policies policy.Set } -func (c candidate) ResourceID() flux.ResourceID { +func (c candidate) ResourceID() resource.ID { return c.resourceID } @@ -54,7 +54,7 @@ func (candidate) Bytes() []byte { func TestCalculateChanges_Automated(t *testing.T) { logger := log.NewNopLogger() - resourceID := flux.MakeResourceID(ns, "deployment", "application") + resourceID := resource.MakeID(ns, "deployment", "application") candidateWorkloads := resources{ resourceID: candidate{ resourceID: resourceID, @@ -102,7 +102,7 @@ func TestCalculateChanges_Automated(t *testing.T) { } func TestCalculateChanges_UntaggedImage(t *testing.T) { logger := log.NewNopLogger() - resourceID := flux.MakeResourceID(ns, "deployment", "application") + resourceID := resource.MakeID(ns, "deployment", "application") candidateWorkloads := resources{ resourceID: candidate{ resourceID: resourceID, @@ -159,7 +159,7 @@ func TestCalculateChanges_UntaggedImage(t *testing.T) { func TestCalculateChanges_ZeroTimestamp(t *testing.T) { logger := log.NewNopLogger() - resourceID := flux.MakeResourceID(ns, "deployment", "application") + resourceID := resource.MakeID(ns, "deployment", "application") candidateWorkloads := resources{ resourceID: candidate{ resourceID: resourceID, diff --git a/daemon/sync.go b/daemon/sync.go index 64a92681d..a306146f5 100644 --- a/daemon/sync.go +++ b/daemon/sync.go @@ -9,7 +9,6 @@ import ( "path/filepath" "time" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/event" "github.com/weaveworks/flux/git" @@ -69,9 +68,9 @@ func (d *Daemon) Sync(ctx context.Context, started time.Time, revision string, s // Determine what resources changed during the sync changedResources, err := getChangedResources(ctx, c, d.GitTimeout, working, resourceStore, resources) - serviceIDs := flux.ResourceIDSet{} + serviceIDs := resource.IDSet{} for _, r := range changedResources { - serviceIDs.Add([]flux.ResourceID{r.ResourceID()}) + serviceIDs.Add([]resource.ID{r.ResourceID()}) } // Retrieve git notes and collect events from them @@ -347,7 +346,7 @@ func collectNoteEvents(ctx context.Context, c changeSet, notes map[string]struct } // logCommitEvent reports all synced commits to the upstream. -func logCommitEvent(el eventLogger, c changeSet, serviceIDs flux.ResourceIDSet, started time.Time, +func logCommitEvent(el eventLogger, c changeSet, serviceIDs resource.IDSet, started time.Time, includesEvents map[string]bool, resourceErrors []event.ResourceError, logger log.Logger) error { if len(c.commits) == 0 { return nil diff --git a/daemon/sync_test.go b/daemon/sync_test.go index 2c0d19065..a069245e7 100644 --- a/daemon/sync_test.go +++ b/daemon/sync_test.go @@ -14,7 +14,6 @@ import ( "github.com/go-kit/kit/log" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/cluster/kubernetes" "github.com/weaveworks/flux/cluster/kubernetes/testfiles" @@ -25,6 +24,7 @@ import ( "github.com/weaveworks/flux/job" "github.com/weaveworks/flux/manifests" registryMock "github.com/weaveworks/flux/registry/mock" + "github.com/weaveworks/flux/resource" ) const ( @@ -93,7 +93,7 @@ func TestPullAndSync_InitialSync(t *testing.T) { syncCalled := 0 var syncDef *cluster.SyncSet - expectedResourceIDs := flux.ResourceIDs{} + expectedResourceIDs := resource.IDs{} for id, _ := range testfiles.ResourceMap { expectedResourceIDs = append(expectedResourceIDs, id) } @@ -132,8 +132,8 @@ func TestPullAndSync_InitialSync(t *testing.T) { t.Errorf("Unexpected event type: %#v", es[0]) } else { gotResourceIDs := es[0].ServiceIDs - flux.ResourceIDs(gotResourceIDs).Sort() - if !reflect.DeepEqual(gotResourceIDs, []flux.ResourceID(expectedResourceIDs)) { + resource.IDs(gotResourceIDs).Sort() + if !reflect.DeepEqual(gotResourceIDs, []resource.ID(expectedResourceIDs)) { t.Errorf("Unexpected event workload ids: %#v, expected: %#v", gotResourceIDs, expectedResourceIDs) } } @@ -174,7 +174,7 @@ func TestDoSync_NoNewCommits(t *testing.T) { syncCalled := 0 var syncDef *cluster.SyncSet - expectedResourceIDs := flux.ResourceIDs{} + expectedResourceIDs := resource.IDs{} for id, _ := range testfiles.ResourceMap { expectedResourceIDs = append(expectedResourceIDs, id) } @@ -288,7 +288,7 @@ func TestDoSync_WithNewCommit(t *testing.T) { syncCalled := 0 var syncDef *cluster.SyncSet - expectedResourceIDs := flux.ResourceIDs{} + expectedResourceIDs := resource.IDs{} for id, _ := range testfiles.ResourceMap { expectedResourceIDs = append(expectedResourceIDs, id) } @@ -326,10 +326,10 @@ func TestDoSync_WithNewCommit(t *testing.T) { t.Errorf("Unexpected event type: %#v", es[0]) } else { gotResourceIDs := es[0].ServiceIDs - flux.ResourceIDs(gotResourceIDs).Sort() + resource.IDs(gotResourceIDs).Sort() // Event should only have changed workload ids - if !reflect.DeepEqual(gotResourceIDs, []flux.ResourceID{flux.MustParseResourceID("default:deployment/helloworld")}) { - t.Errorf("Unexpected event workload ids: %#v, expected: %#v", gotResourceIDs, []flux.ResourceID{flux.MustParseResourceID("default:deployment/helloworld")}) + if !reflect.DeepEqual(gotResourceIDs, []resource.ID{resource.MustParseID("default:deployment/helloworld")}) { + t.Errorf("Unexpected event workload ids: %#v, expected: %#v", gotResourceIDs, []resource.ID{resource.MustParseID("default:deployment/helloworld")}) } } // It moves the tag diff --git a/event/event.go b/event/event.go index dd44d11f3..09f8f3792 100644 --- a/event/event.go +++ b/event/event.go @@ -1,15 +1,14 @@ package event import ( + "encoding/json" "fmt" "sort" "strings" "time" - "encoding/json" - "github.com/pkg/errors" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -42,7 +41,7 @@ type Event struct { // Identifiers of workloads affected by this event. // TODO: rename to WorkloadIDs after adding versioning. - ServiceIDs []flux.ResourceID `json:"serviceIDs"` + ServiceIDs []resource.ID `json:"serviceIDs"` // Type is the type of event, usually "release" for now, but could be other // things later @@ -199,7 +198,7 @@ type Commit struct { } type ResourceError struct { - ID flux.ResourceID + ID resource.ID Path string Error string } diff --git a/flux.go b/flux.go deleted file mode 100644 index e27c5999e..000000000 --- a/flux.go +++ /dev/null @@ -1,240 +0,0 @@ -package flux - -import ( - "encoding/json" - "fmt" - "regexp" - "sort" - "strings" - - "github.com/pkg/errors" -) - -var ( - ErrInvalidServiceID = errors.New("invalid service ID") - - LegacyServiceIDRegexp = regexp.MustCompile("^([a-zA-Z0-9_-]+)/([a-zA-Z0-9_-]+)$") - // The namespace and name components are (apparently - // non-normatively) defined in - // https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/identifiers.md - // In practice, more punctuation is used than allowed there; - // specifically, people use underscores as well as dashes and dots, and in names, colons. - ResourceIDRegexp = regexp.MustCompile("^(|[a-zA-Z0-9_-]+):([a-zA-Z0-9_-]+)/([a-zA-Z0-9_.:-]+)$") - UnqualifiedResourceIDRegexp = regexp.MustCompile("^([a-zA-Z0-9_-]+)/([a-zA-Z0-9_.:-]+)$") -) - -// ResourceID is an opaque type which uniquely identifies a resource in an -// orchestrator. -type ResourceID struct { - resourceIDImpl -} - -type resourceIDImpl interface { - String() string -} - -// Old-style / format -type legacyServiceID struct { - namespace, service string -} - -func (id legacyServiceID) String() string { - return fmt.Sprintf("%s/%s", id.namespace, id.service) -} - -// New :/ format -type resourceID struct { - namespace, kind, name string -} - -func (id resourceID) String() string { - return fmt.Sprintf("%s:%s/%s", id.namespace, id.kind, id.name) -} - -// ParseResourceID constructs a ResourceID from a string representation -// if possible, returning an error value otherwise. -func ParseResourceID(s string) (ResourceID, error) { - if m := ResourceIDRegexp.FindStringSubmatch(s); m != nil { - return ResourceID{resourceID{m[1], strings.ToLower(m[2]), m[3]}}, nil - } - if m := LegacyServiceIDRegexp.FindStringSubmatch(s); m != nil { - return ResourceID{legacyServiceID{m[1], m[2]}}, nil - } - return ResourceID{}, errors.Wrap(ErrInvalidServiceID, "parsing "+s) -} - -// MustParseResourceID constructs a ResourceID from a string representation, -// panicing if the format is invalid. -func MustParseResourceID(s string) ResourceID { - id, err := ParseResourceID(s) - if err != nil { - panic(err) - } - return id -} - -// ParseResourceIDOptionalNamespace constructs a ResourceID from either a fully -// qualified string representation, or an unqualified kind/name representation -// and the supplied namespace. -func ParseResourceIDOptionalNamespace(namespace, s string) (ResourceID, error) { - if m := ResourceIDRegexp.FindStringSubmatch(s); m != nil { - return ResourceID{resourceID{m[1], strings.ToLower(m[2]), m[3]}}, nil - } - if m := UnqualifiedResourceIDRegexp.FindStringSubmatch(s); m != nil { - return ResourceID{resourceID{namespace, strings.ToLower(m[1]), m[2]}}, nil - } - return ResourceID{}, errors.Wrap(ErrInvalidServiceID, "parsing "+s) -} - -// MakeResourceID constructs a ResourceID from constituent components. -func MakeResourceID(namespace, kind, name string) ResourceID { - return ResourceID{resourceID{namespace, strings.ToLower(kind), name}} -} - -// Components returns the constituent components of a ResourceID -func (id ResourceID) Components() (namespace, kind, name string) { - switch impl := id.resourceIDImpl.(type) { - case resourceID: - return impl.namespace, impl.kind, impl.name - case legacyServiceID: - return impl.namespace, "service", impl.service - default: - panic("wrong underlying type") - } -} - -// MarshalJSON encodes a ResourceID as a JSON string. This is -// done to maintain backwards compatibility with previous flux -// versions where the ResourceID is a plain string. -func (id ResourceID) MarshalJSON() ([]byte, error) { - if id.resourceIDImpl == nil { - // Sadly needed as it's possible to construct an empty ResourceID literal - return json.Marshal("") - } - return json.Marshal(id.String()) -} - -// UnmarshalJSON decodes a ResourceID from a JSON string. This is -// done to maintain backwards compatibility with previous flux -// versions where the ResourceID is a plain string. -func (id *ResourceID) UnmarshalJSON(data []byte) (err error) { - var str string - if err := json.Unmarshal(data, &str); err != nil { - return err - } - if string(str) == "" { - // Sadly needed as it's possible to construct an empty ResourceID literal - *id = ResourceID{} - return nil - } - *id, err = ParseResourceID(string(str)) - return err -} - -// MarshalText encodes a ResourceID as a flat string; this is -// required because ResourceIDs are sometimes used as map keys. -func (id ResourceID) MarshalText() (text []byte, err error) { - return []byte(id.String()), nil -} - -// MarshalText decodes a ResourceID from a flat string; this is -// required because ResourceIDs are sometimes used as map keys. -func (id *ResourceID) UnmarshalText(text []byte) error { - result, err := ParseResourceID(string(text)) - if err != nil { - return err - } - *id = result - return nil -} - -type ResourceIDSet map[ResourceID]struct{} - -func (s ResourceIDSet) String() string { - var ids []string - for id := range s { - ids = append(ids, id.String()) - } - return "{" + strings.Join(ids, ", ") + "}" -} - -func (s ResourceIDSet) Add(ids []ResourceID) { - for _, id := range ids { - s[id] = struct{}{} - } -} - -func (s ResourceIDSet) Without(others ResourceIDSet) ResourceIDSet { - if s == nil || len(s) == 0 || others == nil || len(others) == 0 { - return s - } - res := ResourceIDSet{} - for id := range s { - if !others.Contains(id) { - res[id] = struct{}{} - } - } - return res -} - -func (s ResourceIDSet) Contains(id ResourceID) bool { - if s == nil { - return false - } - _, ok := s[id] - return ok -} - -func (s ResourceIDSet) Intersection(others ResourceIDSet) ResourceIDSet { - if s == nil { - return others - } - if others == nil { - return s - } - result := ResourceIDSet{} - for id := range s { - if _, ok := others[id]; ok { - result[id] = struct{}{} - } - } - return result -} - -func (s ResourceIDSet) ToSlice() ResourceIDs { - i := 0 - keys := make(ResourceIDs, len(s)) - for k := range s { - keys[i] = k - i++ - } - return keys -} - -type ResourceIDs []ResourceID - -func (p ResourceIDs) Len() int { return len(p) } -func (p ResourceIDs) Less(i, j int) bool { return p[i].String() < p[j].String() } -func (p ResourceIDs) Swap(i, j int) { p[i], p[j] = p[j], p[i] } -func (p ResourceIDs) Sort() { sort.Sort(p) } - -func (ids ResourceIDs) Without(others ResourceIDSet) (res ResourceIDs) { - for _, id := range ids { - if !others.Contains(id) { - res = append(res, id) - } - } - return res -} - -func (ids ResourceIDs) Contains(id ResourceID) bool { - set := ResourceIDSet{} - set.Add(ids) - return set.Contains(id) -} - -func (ids ResourceIDs) Intersection(others ResourceIDSet) ResourceIDSet { - set := ResourceIDSet{} - set.Add(ids) - return set.Intersection(others) -} diff --git a/git/gittest/repo.go b/git/gittest/repo.go index 5c4dd6ffb..061ff1803 100644 --- a/git/gittest/repo.go +++ b/git/gittest/repo.go @@ -7,9 +7,9 @@ import ( "path/filepath" "testing" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster/kubernetes/testfiles" "github.com/weaveworks/flux/git" + "github.com/weaveworks/flux/resource" ) // Repo creates a new clone-able git repo, pre-populated with some kubernetes @@ -65,7 +65,7 @@ func Repo(t *testing.T) (*git.Repo, func()) { // Workloads is a shortcut to getting the names of the workloads (NB // not all resources, just the workloads) represented in the test // files. -func Workloads() (res []flux.ResourceID) { +func Workloads() (res []resource.ID) { for k, _ := range testfiles.WorkloadMap("") { res = append(res, k) } diff --git a/http/daemon/server.go b/http/daemon/server.go index b27277342..a50933853 100644 --- a/http/daemon/server.go +++ b/http/daemon/server.go @@ -8,16 +8,15 @@ import ( "github.com/gorilla/mux" "github.com/pkg/errors" stdprometheus "github.com/prometheus/client_golang/prometheus" - "github.com/weaveworks/common/middleware" - "github.com/weaveworks/flux" + "github.com/weaveworks/common/middleware" "github.com/weaveworks/flux/api" "github.com/weaveworks/flux/api/v10" "github.com/weaveworks/flux/api/v11" transport "github.com/weaveworks/flux/http" "github.com/weaveworks/flux/job" fluxmetrics "github.com/weaveworks/flux/metrics" - "github.com/weaveworks/flux/policy" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -152,7 +151,7 @@ func (s HTTPServer) ListServicesWithOptions(w http.ResponseWriter, r *http.Reque services := r.URL.Query().Get("services") if services != "" { for _, svc := range strings.Split(services, ",") { - id, err := flux.ParseResourceID(svc) + id, err := resource.ParseID(svc) if err != nil { transport.WriteError(w, r, http.StatusBadRequest, errors.Wrapf(err, "parsing service spec %q", svc)) return @@ -223,9 +222,9 @@ func (s HTTPServer) UpdateImages(w http.ResponseWriter, r *http.Request) { return } - var excludes []flux.ResourceID + var excludes []resource.ID for _, ex := range r.URL.Query()["exclude"] { - s, err := flux.ParseResourceID(ex) + s, err := resource.ParseID(ex) if err != nil { transport.WriteError(w, r, http.StatusBadRequest, errors.Wrapf(err, "parsing excluded service %q", ex)) return @@ -252,7 +251,7 @@ func (s HTTPServer) UpdateImages(w http.ResponseWriter, r *http.Request) { } func (s HTTPServer) UpdatePolicies(w http.ResponseWriter, r *http.Request) { - var updates policy.Updates + var updates resource.PolicyUpdates if err := json.NewDecoder(r.Body).Decode(&updates); err != nil { transport.WriteError(w, r, http.StatusBadRequest, err) return diff --git a/image/image.go b/image/image.go index d580c5ed5..e8e56b10e 100644 --- a/image/image.go +++ b/image/image.go @@ -242,7 +242,7 @@ type Labels struct { BuildDate time.Time `json:"org.label-schema.build-date,omitempty"` // Created holds the Open Container Image spec 'created' label // Ref: https://github.com/opencontainers/image-spec/blob/master/annotations.md#pre-defined-annotation-keys - Created time.Time `json:"org.opencontainers.image.created,omitempty"` + Created time.Time `json:"org.opencontainers.image.created,omitempty"` } // MarshalJSON returns the Labels value in JSON (as bytes). It is diff --git a/image/image_test.go b/image/image_test.go index 818ed124a..09044bd39 100644 --- a/image/image_test.go +++ b/image/image_test.go @@ -238,9 +238,9 @@ func TestImage_OrderByCreationDate(t *testing.T) { imA := mustMakeInfo("my/Image:2", testTime) imB := mustMakeInfo("my/Image:0", time.Time{}).setLabels(Labels{Created: time0}) imC := mustMakeInfo("my/Image:3", time.Time{}).setLabels(Labels{BuildDate: time2}) - imD := mustMakeInfo("my/Image:4", time.Time{}) // test nil + imD := mustMakeInfo("my/Image:4", time.Time{}) // test nil imE := mustMakeInfo("my/Image:1", time.Time{}).setLabels(Labels{Created: testTime}) // test equal - imF := mustMakeInfo("my/Image:5", time.Time{}) // test nil equal + imF := mustMakeInfo("my/Image:5", time.Time{}) // test nil equal imgs := []Info{imA, imB, imC, imD, imE, imF} Sort(imgs, NewerByCreated) checkSorted(t, imgs) diff --git a/integrations/apis/flux.weave.works/v1beta1/types.go b/integrations/apis/flux.weave.works/v1beta1/types.go index d38fbf899..d6e4bdfaf 100644 --- a/integrations/apis/flux.weave.works/v1beta1/types.go +++ b/integrations/apis/flux.weave.works/v1beta1/types.go @@ -8,7 +8,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/helm/pkg/chartutil" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/resource" ) // +genclient @@ -26,8 +26,8 @@ type HelmRelease struct { // ResourceID returns an ID made from the identifying parts of the // resource, as a convenience for Flux, which uses them // everywhere. -func (fhr HelmRelease) ResourceID() flux.ResourceID { - return flux.MakeResourceID(fhr.Namespace, "HelmRelease", fhr.Name) +func (fhr HelmRelease) ResourceID() resource.ID { + return resource.MakeID(fhr.Namespace, "HelmRelease", fhr.Name) } // ValuesFromSource represents a source of values. diff --git a/integrations/helm/release/release.go b/integrations/helm/release/release.go index c32065c13..233f9b22e 100644 --- a/integrations/helm/release/release.go +++ b/integrations/helm/release/release.go @@ -25,11 +25,11 @@ import ( k8shelm "k8s.io/helm/pkg/helm" helmenv "k8s.io/helm/pkg/helm/environment" hapi_release "k8s.io/helm/pkg/proto/hapi/release" + helmutil "k8s.io/helm/pkg/releaseutil" - "github.com/weaveworks/flux" fluxk8s "github.com/weaveworks/flux/cluster/kubernetes" flux_v1beta1 "github.com/weaveworks/flux/integrations/apis/flux.weave.works/v1beta1" - helmutil "k8s.io/helm/pkg/releaseutil" + "github.com/weaveworks/flux/resource" ) type Action string @@ -104,8 +104,8 @@ func (r *Release) GetUpgradableRelease(name string) (*hapi_release.Release, erro case hapi_release.Status_FAILED: return nil, fmt.Errorf("release requires a rollback before it can be upgraded (%s)", status.GetCode().String()) case hapi_release.Status_PENDING_INSTALL, - hapi_release.Status_PENDING_UPGRADE, - hapi_release.Status_PENDING_ROLLBACK: + hapi_release.Status_PENDING_UPGRADE, + hapi_release.Status_PENDING_ROLLBACK: return nil, fmt.Errorf("operation pending for release (%s)", status.GetCode().String()) default: return nil, fmt.Errorf("current state prevents it from being upgraded (%s)", status.GetCode().String()) @@ -288,9 +288,9 @@ func (r *Release) annotateResources(release *hapi_release.Release, fhr flux_v1be } } -// fhrResourceID constructs a flux.ResourceID for a HelmRelease resource. -func fhrResourceID(fhr flux_v1beta1.HelmRelease) flux.ResourceID { - return flux.MakeResourceID(fhr.Namespace, "HelmRelease", fhr.Name) +// fhrResourceID constructs a resource.ID for a HelmRelease resource. +func fhrResourceID(fhr flux_v1beta1.HelmRelease) resource.ID { + return resource.MakeID(fhr.Namespace, "HelmRelease", fhr.Name) } // values tries to resolve all given value file sources and merges diff --git a/manifests/configaware.go b/manifests/configaware.go index a740f3ccf..20429ef27 100644 --- a/manifests/configaware.go +++ b/manifests/configaware.go @@ -11,9 +11,7 @@ import ( "strings" "sync" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" - "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" ) @@ -131,7 +129,7 @@ func findConfigFilePaths(baseDir string, initialPath string) (string, string, er return "", "", configFileNotFoundErr } -func (ca *configAware) SetWorkloadContainerImage(ctx context.Context, resourceID flux.ResourceID, container string, +func (ca *configAware) SetWorkloadContainerImage(ctx context.Context, resourceID resource.ID, container string, newImageID image.Ref) error { resourcesByID, err := ca.getResourcesByID(ctx) if err != nil { @@ -268,8 +266,8 @@ func (ca *configAware) getGeneratedManifests(ctx context.Context, cf *ConfigFile return buf.Bytes(), nil } -func (ca *configAware) UpdateWorkloadPolicies(ctx context.Context, resourceID flux.ResourceID, - update policy.Update) (bool, error) { +func (ca *configAware) UpdateWorkloadPolicies(ctx context.Context, resourceID resource.ID, + update resource.PolicyUpdate) (bool, error) { resourcesByID, err := ca.getResourcesByID(ctx) if err != nil { return false, err @@ -293,7 +291,7 @@ func (ca *configAware) UpdateWorkloadPolicies(ctx context.Context, resourceID fl } func (ca *configAware) updateConfigFileWorkloadPolicies(ctx context.Context, cf *ConfigFile, r resource.Resource, - update policy.Update) (bool, error) { + update resource.PolicyUpdate) (bool, error) { if cf.PatchUpdated != nil { var changed bool err := ca.updatePatchFile(ctx, cf, func(previousManifests []byte) ([]byte, error) { diff --git a/manifests/configaware_test.go b/manifests/configaware_test.go index 6e4ee5edb..0c789b44f 100644 --- a/manifests/configaware_test.go +++ b/manifests/configaware_test.go @@ -10,7 +10,6 @@ import ( "github.com/go-kit/kit/log" "github.com/stretchr/testify/assert" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster/kubernetes" "github.com/weaveworks/flux/cluster/kubernetes/testfiles" "github.com/weaveworks/flux/image" @@ -143,13 +142,13 @@ func TestCommandUpdatedConfigFile(t *testing.T) { resources, err := frs.GetAllResourcesByID(ctx) assert.NoError(t, err) assert.Equal(t, 1, len(resources)) - deploymentID := flux.MustParseResourceID("default:deployment/helloworld") + deploymentID := resource.MustParseID("default:deployment/helloworld") assert.Contains(t, resources, deploymentID.String()) ref, err := image.ParseRef("repo/image:tag") assert.NoError(t, err) err = frs.SetWorkloadContainerImage(ctx, deploymentID, "greeter", ref) assert.NoError(t, err) - _, err = frs.UpdateWorkloadPolicies(ctx, deploymentID, policy.Update{ + _, err = frs.UpdateWorkloadPolicies(ctx, deploymentID, resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("greeter"): "glob:master-*"}, }) assert.NoError(t, err) @@ -189,7 +188,7 @@ func TestPatchUpdatedConfigFile(t *testing.T) { assert.NoError(t, err) assert.Equal(t, 2, len(resources)) var deployment resource.Resource - deploymentID := flux.MustParseResourceID("default:deployment/helloworld") + deploymentID := resource.MustParseID("default:deployment/helloworld") for id, res := range resources { if id == deploymentID.String() { deployment = res @@ -200,7 +199,7 @@ func TestPatchUpdatedConfigFile(t *testing.T) { assert.NoError(t, err) err = frs.SetWorkloadContainerImage(ctx, deploymentID, "greeter", ref) assert.NoError(t, err) - _, err = frs.UpdateWorkloadPolicies(ctx, deploymentID, policy.Update{ + _, err = frs.UpdateWorkloadPolicies(ctx, deploymentID, resource.PolicyUpdate{ Add: policy.Set{policy.TagPrefix("greeter"): "glob:master-*"}, }) expectedPatch := `--- diff --git a/manifests/configfile.go b/manifests/configfile.go index 2fd896454..aa1a65b93 100644 --- a/manifests/configfile.go +++ b/manifests/configfile.go @@ -13,7 +13,7 @@ import ( "github.com/pkg/errors" "gopkg.in/yaml.v2" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/resource" ) const ( @@ -114,7 +114,7 @@ func (cf *ConfigFile) ExecGenerators(ctx context.Context, generators []Generator // ExecContainerImageUpdaters executes all the image updates in the configuration file. // It will stop at the first error, in which case the returned error will be non-nil func (cf *ConfigFile) ExecContainerImageUpdaters(ctx context.Context, - workload flux.ResourceID, container string, image, imageTag string) []ConfigFileCombinedExecResult { + workload resource.ID, container string, image, imageTag string) []ConfigFileCombinedExecResult { env := makeEnvFromResourceID(workload) env = append(env, "FLUX_CONTAINER="+container, @@ -137,7 +137,7 @@ func (cf *ConfigFile) ExecContainerImageUpdaters(ctx context.Context, // policy. It will stop at the first error, in which case the returned // error will be non-nil func (cf *ConfigFile) ExecPolicyUpdaters(ctx context.Context, - workload flux.ResourceID, policyName, policyValue string) []ConfigFileCombinedExecResult { + workload resource.ID, policyName, policyValue string) []ConfigFileCombinedExecResult { env := makeEnvFromResourceID(workload) env = append(env, "FLUX_POLICY="+policyName) if policyValue != "" { @@ -190,7 +190,7 @@ func (cf *ConfigFile) execCommand(ctx context.Context, env []string, stdOut, std return err } -func makeEnvFromResourceID(id flux.ResourceID) []string { +func makeEnvFromResourceID(id resource.ID) []string { ns, kind, name := id.Components() return []string{ "FLUX_WORKLOAD=" + id.String(), diff --git a/manifests/configfile_test.go b/manifests/configfile_test.go index fcef9e121..c7d300a61 100644 --- a/manifests/configfile_test.go +++ b/manifests/configfile_test.go @@ -7,7 +7,7 @@ import ( "github.com/stretchr/testify/assert" "gopkg.in/yaml.v2" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/resource" ) const patchUpdatedConfigFile = `--- @@ -83,7 +83,7 @@ func TestExecContainerImageUpdaters(t *testing.T) { var cf ConfigFile err := yaml.Unmarshal([]byte(echoCmdUpdatedConfigFile), &cf) assert.NoError(t, err) - resourceID := flux.MustParseResourceID("default:deployment/foo") + resourceID := resource.MustParseID("default:deployment/foo") result := cf.ExecContainerImageUpdaters(context.Background(), resourceID, "bar", "repo/image", "latest") assert.Equal(t, 2, len(result), "result: %s", result) assert.Equal(t, @@ -98,7 +98,7 @@ func TestExecAnnotationUpdaters(t *testing.T) { var cf ConfigFile err := yaml.Unmarshal([]byte(echoCmdUpdatedConfigFile), &cf) assert.NoError(t, err) - resourceID := flux.MustParseResourceID("default:deployment/foo") + resourceID := resource.MustParseID("default:deployment/foo") // Test the update/addition of annotations annotationValue := "value" diff --git a/manifests/manifests.go b/manifests/manifests.go index 4c049e8fb..652e644e3 100644 --- a/manifests/manifests.go +++ b/manifests/manifests.go @@ -3,9 +3,7 @@ package manifests import ( "bytes" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" - "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" ) @@ -21,9 +19,9 @@ type Manifests interface { // ParseManifest parses the content of a collection of manifests, into resources ParseManifest(def []byte, source string) (map[string]resource.Resource, error) // Set the image of a container in a manifest's bytes to that given - SetWorkloadContainerImage(def []byte, resourceID flux.ResourceID, container string, newImageID image.Ref) ([]byte, error) + SetWorkloadContainerImage(def []byte, resourceID resource.ID, container string, newImageID image.Ref) ([]byte, error) // UpdateWorkloadPolicies modifies a manifest to apply the policy update specified - UpdateWorkloadPolicies(def []byte, id flux.ResourceID, update policy.Update) ([]byte, error) + UpdateWorkloadPolicies(def []byte, id resource.ID, update resource.PolicyUpdate) ([]byte, error) // CreateManifestPatch obtains a patch between the original and modified manifests CreateManifestPatch(originalManifests, modifiedManifests []byte, originalSource, modifiedSource string) ([]byte, error) // ApplyManifestPatch applies a manifest patch (obtained with CreateManifestDiff) returned the patched manifests diff --git a/manifests/rawfiles.go b/manifests/rawfiles.go index ac202d19e..382bbd610 100644 --- a/manifests/rawfiles.go +++ b/manifests/rawfiles.go @@ -7,9 +7,7 @@ import ( "os" "path/filepath" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" - "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" ) @@ -28,7 +26,7 @@ func NewRawFiles(baseDir string, paths []string, manifests Manifests) *rawFiles } // Set the container image of a resource in the store -func (f *rawFiles) SetWorkloadContainerImage(ctx context.Context, id flux.ResourceID, container string, newImageID image.Ref) error { +func (f *rawFiles) SetWorkloadContainerImage(ctx context.Context, id resource.ID, container string, newImageID image.Ref) error { resourcesByID, err := f.GetAllResourcesByID(ctx) if err != nil { return err @@ -59,7 +57,7 @@ func (f *rawFiles) setManifestWorkloadContainerImage(r resource.Resource, contai // UpdateWorkloadPolicies modifies a resource in the store to apply the policy-update specified. // It returns whether a change in the resource was actually made as a result of the change -func (f *rawFiles) UpdateWorkloadPolicies(ctx context.Context, id flux.ResourceID, update policy.Update) (bool, error) { +func (f *rawFiles) UpdateWorkloadPolicies(ctx context.Context, id resource.ID, update resource.PolicyUpdate) (bool, error) { resourcesByID, err := f.GetAllResourcesByID(ctx) if err != nil { return false, err @@ -71,7 +69,7 @@ func (f *rawFiles) UpdateWorkloadPolicies(ctx context.Context, id flux.ResourceI return f.updateManifestWorkloadPolicies(r, update) } -func (f *rawFiles) updateManifestWorkloadPolicies(r resource.Resource, update policy.Update) (bool, error) { +func (f *rawFiles) updateManifestWorkloadPolicies(r resource.Resource, update resource.PolicyUpdate) (bool, error) { fullFilePath := filepath.Join(f.baseDir, r.Source()) def, err := ioutil.ReadFile(fullFilePath) if err != nil { diff --git a/manifests/store.go b/manifests/store.go index 3c26576af..322fff6fb 100644 --- a/manifests/store.go +++ b/manifests/store.go @@ -4,9 +4,7 @@ import ( "context" "fmt" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" - "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/resource" ) @@ -22,10 +20,10 @@ func ErrResourceNotFound(name string) error { // in a file or not e.g., generated and updated by a .flux.yaml file, explicit Kubernetes .yaml manifests files ... type Store interface { // Set the container image of a resource in the store - SetWorkloadContainerImage(ctx context.Context, resourceID flux.ResourceID, container string, newImageID image.Ref) error + SetWorkloadContainerImage(ctx context.Context, resourceID resource.ID, container string, newImageID image.Ref) error // UpdateWorkloadPolicies modifies a resource in the store to apply the policy-update specified. // It returns whether a change in the resource was actually made as a result of the change - UpdateWorkloadPolicies(ctx context.Context, resourceID flux.ResourceID, update policy.Update) (bool, error) + UpdateWorkloadPolicies(ctx context.Context, resourceID resource.ID, update resource.PolicyUpdate) (bool, error) // Load all the resources in the store. The returned map is indexed by the resource IDs GetAllResourcesByID(ctx context.Context) (map[string]resource.Resource, error) } diff --git a/policy/policy.go b/policy/policy.go index ef846858a..c9ec95f34 100644 --- a/policy/policy.go +++ b/policy/policy.go @@ -3,8 +3,6 @@ package policy import ( "encoding/json" "strings" - - "github.com/weaveworks/flux" ) const ( @@ -47,13 +45,6 @@ func GetTagPattern(policies Set, container string) Pattern { return NewPattern(pattern) } -type Updates map[flux.ResourceID]Update - -type Update struct { - Add Set `json:"add"` - Remove Set `json:"remove"` -} - type Set map[Policy]string // We used to specify a set of policies as []Policy, and in some places diff --git a/registry/client.go b/registry/client.go index 4434a4ef4..b34de4635 100644 --- a/registry/client.go +++ b/registry/client.go @@ -158,9 +158,9 @@ interpret: } var config struct { - Arch string `json:"architecture"` - Created time.Time `json:"created"` - OS string `json:"os"` + Arch string `json:"architecture"` + Created time.Time `json:"created"` + OS string `json:"os"` ContainerConfig struct { Labels image.Labels `json:"labels"` } `json:"container_config"` diff --git a/registry/credentials.go b/registry/credentials.go index b8c49616c..9962b8903 100644 --- a/registry/credentials.go +++ b/registry/credentials.go @@ -145,11 +145,11 @@ func (cs Credentials) credsFor(host string) creds { } } - if hostIsAzureContainerRegistry(host) { - if cred, err := getAzureCloudConfigAADToken(host); err == nil { - return cred - } - } + if hostIsAzureContainerRegistry(host) { + if cred, err := getAzureCloudConfigAADToken(host); err == nil { + return cred + } + } return creds{} } diff --git a/release/context.go b/release/context.go index ac279ae4b..7008c6568 100644 --- a/release/context.go +++ b/release/context.go @@ -6,7 +6,6 @@ import ( "github.com/pkg/errors" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/manifests" "github.com/weaveworks/flux/registry" @@ -68,7 +67,7 @@ func (rc *ReleaseContext) SelectWorkloads(ctx context.Context, results update.Re // Apply prefilters to select the controllers that we'll ask the // cluster about. - var toAskClusterAbout []flux.ResourceID + var toAskClusterAbout []resource.ID for _, s := range allDefined { res := s.Filter(prefilters...) if res.Error == "" { @@ -118,13 +117,13 @@ func (rc *ReleaseContext) SelectWorkloads(ctx context.Context, results update.Re // WorkloadsForUpdate collects all workloads defined in manifests and prepares a list of // workload updates for each of them. It does not consider updatability. -func (rc *ReleaseContext) WorkloadsForUpdate(ctx context.Context) (map[flux.ResourceID]*update.WorkloadUpdate, error) { +func (rc *ReleaseContext) WorkloadsForUpdate(ctx context.Context) (map[resource.ID]*update.WorkloadUpdate, error) { resources, err := rc.GetAllResources(ctx) if err != nil { return nil, err } - var defined = map[flux.ResourceID]*update.WorkloadUpdate{} + var defined = map[resource.ID]*update.WorkloadUpdate{} for _, res := range resources { if wl, ok := res.(resource.Workload); ok { defined[res.ResourceID()] = &update.WorkloadUpdate{ diff --git a/release/releaser_test.go b/release/releaser_test.go index f4d733081..122619173 100644 --- a/release/releaser_test.go +++ b/release/releaser_test.go @@ -13,7 +13,6 @@ import ( "github.com/stretchr/testify/assert" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/cluster/kubernetes" "github.com/weaveworks/flux/cluster/mock" @@ -37,7 +36,7 @@ var ( oldRef, _ = image.ParseRef(oldImage) sidecarImage = "weaveworks/sidecar:master-a000001" sidecarRef, _ = image.ParseRef(sidecarImage) - hwSvcID, _ = flux.ParseResourceID("default:deployment/helloworld") + hwSvcID, _ = resource.ParseID("default:deployment/helloworld") hwSvcSpec, _ = update.ParseResourceSpec(hwSvcID.String()) hwSvc = cluster.Workload{ ID: hwSvcID, @@ -62,7 +61,7 @@ var ( newLockedImg = "quay.io/weaveworks/locked-service:2" newLockedRef, _ = image.ParseRef(newLockedImg) - lockedSvcID, _ = flux.ParseResourceID("default:deployment/locked-service") + lockedSvcID, _ = resource.ParseID("default:deployment/locked-service") lockedSvcSpec, _ = update.ParseResourceSpec(lockedSvcID.String()) lockedSvc = cluster.Workload{ ID: lockedSvcID, @@ -78,7 +77,7 @@ var ( semverHwImg = "quay.io/weaveworks/helloworld:3.0.0" semverHwRef, _ = image.ParseRef(semverHwImg) - semverSvcID = flux.MustParseResourceID("default:deployment/semver") + semverSvcID = resource.MustParseID("default:deployment/semver") semverSvc = cluster.Workload{ ID: semverSvcID, Containers: cluster.ContainersOrExcuse{ @@ -92,7 +91,7 @@ var ( } semverSvcSpec, _ = update.ParseResourceSpec(semverSvc.ID.String()) - testSvcID = flux.MustParseResourceID("default:deployment/test-service") + testSvcID = resource.MustParseID("default:deployment/test-service") testSvc = cluster.Workload{ ID: testSvcID, Containers: cluster.ContainersOrExcuse{ @@ -148,7 +147,7 @@ func mockCluster(running ...cluster.Workload) *mock.Mock { AllWorkloadsFunc: func(ctx context.Context, maybeNamespace string) ([]cluster.Workload, error) { return running, nil }, - SomeWorkloadsFunc: func(ctx context.Context, ids []flux.ResourceID) ([]cluster.Workload, error) { + SomeWorkloadsFunc: func(ctx context.Context, ids []resource.ID) ([]cluster.Workload, error) { var res []cluster.Workload for _, id := range ids { for _, svc := range running { @@ -221,7 +220,7 @@ func (x expected) Result() update.Result { } func Test_InitContainer(t *testing.T) { - initWorkloadID := flux.MustParseResourceID("default:daemonset/init") + initWorkloadID := resource.MustParseID("default:daemonset/init") initSvc := cluster.Workload{ ID: initWorkloadID, Containers: cluster.ContainersOrExcuse{ @@ -287,11 +286,11 @@ func Test_FilterLogic(t *testing.T) { ServiceSpecs: []update.ResourceSpec{hwSvcSpec}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/helloworld"): update.WorkloadResult{ + resource.MustParseID("default:deployment/helloworld"): update.WorkloadResult{ Status: update.ReleaseStatusSuccess, PerContainer: []update.ContainerUpdate{ { @@ -315,11 +314,11 @@ func Test_FilterLogic(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{lockedSvcID}, + Excludes: []resource.ID{lockedSvcID}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/helloworld"): update.WorkloadResult{ + resource.MustParseID("default:deployment/helloworld"): update.WorkloadResult{ Status: update.ReleaseStatusSuccess, PerContainer: []update.ContainerUpdate{ { @@ -334,7 +333,7 @@ func Test_FilterLogic(t *testing.T) { }, }, }, - flux.MustParseResourceID("default:deployment/locked-service"): update.WorkloadResult{ + resource.MustParseID("default:deployment/locked-service"): update.WorkloadResult{ Status: update.ReleaseStatusIgnored, Error: update.Excluded, }, @@ -347,11 +346,11 @@ func Test_FilterLogic(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecFromRef(newHwRef), Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/helloworld"): update.WorkloadResult{ + resource.MustParseID("default:deployment/helloworld"): update.WorkloadResult{ Status: update.ReleaseStatusSuccess, PerContainer: []update.ContainerUpdate{ { @@ -361,7 +360,7 @@ func Test_FilterLogic(t *testing.T) { }, }, }, - flux.MustParseResourceID("default:deployment/locked-service"): update.WorkloadResult{ + resource.MustParseID("default:deployment/locked-service"): update.WorkloadResult{ Status: update.ReleaseStatusIgnored, Error: update.DifferentImage, }, @@ -376,11 +375,11 @@ func Test_FilterLogic(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/helloworld"): update.WorkloadResult{ + resource.MustParseID("default:deployment/helloworld"): update.WorkloadResult{ Status: update.ReleaseStatusSuccess, PerContainer: []update.ContainerUpdate{ { @@ -395,7 +394,7 @@ func Test_FilterLogic(t *testing.T) { }, }, }, - flux.MustParseResourceID("default:deployment/locked-service"): update.WorkloadResult{ + resource.MustParseID("default:deployment/locked-service"): update.WorkloadResult{ Status: update.ReleaseStatusSkipped, Error: update.Locked, }, @@ -408,11 +407,11 @@ func Test_FilterLogic(t *testing.T) { ServiceSpecs: []update.ResourceSpec{hwSvcSpec, update.ResourceSpecAll}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/helloworld"): update.WorkloadResult{ + resource.MustParseID("default:deployment/helloworld"): update.WorkloadResult{ Status: update.ReleaseStatusSuccess, PerContainer: []update.ContainerUpdate{ { @@ -427,7 +426,7 @@ func Test_FilterLogic(t *testing.T) { }, }, }, - flux.MustParseResourceID("default:deployment/locked-service"): update.WorkloadResult{ + resource.MustParseID("default:deployment/locked-service"): update.WorkloadResult{ Status: update.ReleaseStatusSkipped, Error: update.Locked, }, @@ -440,11 +439,11 @@ func Test_FilterLogic(t *testing.T) { ServiceSpecs: []update.ResourceSpec{notInRepoSpec}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID(notInRepoService): skippedNotInRepo, + resource.MustParseID(notInRepoService): skippedNotInRepo, }, Else: ignoredNotIncluded, }, @@ -485,12 +484,12 @@ func Test_Force_lockedWorkload(t *testing.T) { ServiceSpecs: []update.ResourceSpec{lockedSvcSpec}, ImageSpec: update.ImageSpecFromRef(newLockedRef), Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/locked-service"): success, + resource.MustParseID("default:deployment/locked-service"): success, }, Else: ignoredNotIncluded, }, @@ -500,12 +499,12 @@ func Test_Force_lockedWorkload(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecFromRef(newLockedRef), Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/locked-service"): skippedLocked, + resource.MustParseID("default:deployment/locked-service"): skippedLocked, }, Else: skippedNotInCluster, }, @@ -516,12 +515,12 @@ func Test_Force_lockedWorkload(t *testing.T) { ServiceSpecs: []update.ResourceSpec{lockedSvcSpec}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/locked-service"): success, + resource.MustParseID("default:deployment/locked-service"): success, }, Else: ignoredNotIncluded, }, @@ -531,12 +530,12 @@ func Test_Force_lockedWorkload(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/locked-service"): skippedLocked, + resource.MustParseID("default:deployment/locked-service"): skippedLocked, }, Else: skippedNotInCluster, }, @@ -587,12 +586,12 @@ func Test_Force_filteredContainer(t *testing.T) { ServiceSpecs: []update.ResourceSpec{semverSvcSpec}, ImageSpec: update.ImageSpecFromRef(newHwRef), // does not match filter Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/semver"): successNew, + resource.MustParseID("default:deployment/semver"): successNew, }, Else: ignoredNotIncluded, }, @@ -603,12 +602,12 @@ func Test_Force_filteredContainer(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecFromRef(newHwRef), // does not match filter Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/semver"): successNew, + resource.MustParseID("default:deployment/semver"): successNew, }, Else: skippedNotInCluster, }, @@ -619,12 +618,12 @@ func Test_Force_filteredContainer(t *testing.T) { ServiceSpecs: []update.ResourceSpec{semverSvcSpec}, ImageSpec: update.ImageSpecLatest, // will filter images by semver and pick newest version Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/semver"): successSemver, + resource.MustParseID("default:deployment/semver"): successSemver, }, Else: ignoredNotIncluded, }, @@ -635,12 +634,12 @@ func Test_Force_filteredContainer(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, Force: true, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/semver"): successSemver, + resource.MustParseID("default:deployment/semver"): successSemver, }, Else: skippedNotInCluster, }, @@ -685,11 +684,11 @@ func Test_ImageStatus(t *testing.T) { ServiceSpecs: []update.ResourceSpec{testSvcSpec}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/test-service"): update.WorkloadResult{ + resource.MustParseID("default:deployment/test-service"): update.WorkloadResult{ Status: update.ReleaseStatusIgnored, Error: update.DoesNotUseImage, }, @@ -702,11 +701,11 @@ func Test_ImageStatus(t *testing.T) { ServiceSpecs: []update.ResourceSpec{hwSvcSpec}, ImageSpec: update.ImageSpecLatest, Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, }, Expected: expected{ Specific: update.Result{ - flux.MustParseResourceID("default:deployment/helloworld"): update.WorkloadResult{ + resource.MustParseID("default:deployment/helloworld"): update.WorkloadResult{ Status: update.ReleaseStatusSkipped, Error: update.ImageUpToDate, }, @@ -729,7 +728,7 @@ func Test_ImageStatus(t *testing.T) { } func Test_UpdateMultidoc(t *testing.T) { - egID := flux.MustParseResourceID("default:deployment/multi-deploy") + egID := resource.MustParseID("default:deployment/multi-deploy") egSvc := cluster.Workload{ ID: egID, Containers: cluster.ContainersOrExcuse{ @@ -776,7 +775,7 @@ func Test_UpdateMultidoc(t *testing.T) { } func Test_UpdateList(t *testing.T) { - egID := flux.MustParseResourceID("default:deployment/list-deploy") + egID := resource.MustParseID("default:deployment/list-deploy") egSvc := cluster.Workload{ ID: egID, Containers: cluster.ContainersOrExcuse{ @@ -839,7 +838,7 @@ func Test_UpdateContainers(t *testing.T) { } for _, tst := range []struct { Name string - WorkloadID flux.ResourceID + WorkloadID resource.ID Spec []update.ContainerUpdate Force bool @@ -1018,7 +1017,7 @@ func Test_UpdateContainers(t *testing.T) { }, } { specs := update.ReleaseContainersSpec{ - ContainerSpecs: map[flux.ResourceID][]update.ContainerUpdate{tst.WorkloadID: tst.Spec}, + ContainerSpecs: map[resource.ID][]update.ContainerUpdate{tst.WorkloadID: tst.Spec}, Kind: update.ReleaseKindExecute, } @@ -1056,7 +1055,7 @@ type badManifests struct { manifests.Manifests } -func (m *badManifests) SetWorkloadContainerImage(def []byte, id flux.ResourceID, container string, image image.Ref) ([]byte, error) { +func (m *badManifests) SetWorkloadContainerImage(def []byte, id resource.ID, container string, image image.Ref) ([]byte, error) { return def, nil } @@ -1066,7 +1065,7 @@ func Test_BadRelease(t *testing.T) { ServiceSpecs: []update.ResourceSpec{update.ResourceSpecAll}, ImageSpec: update.ImageSpecFromRef(newHwRef), Kind: update.ReleaseKindExecute, - Excludes: []flux.ResourceID{}, + Excludes: []resource.ID{}, } checkout1, cleanup1 := setup(t) defer cleanup1() diff --git a/remote/mock.go b/remote/mock.go index 8f4e5a9bd..1d8891727 100644 --- a/remote/mock.go +++ b/remote/mock.go @@ -8,7 +8,6 @@ import ( "testing" "time" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/api" "github.com/weaveworks/flux/api/v10" "github.com/weaveworks/flux/api/v11" @@ -17,6 +16,7 @@ import ( "github.com/weaveworks/flux/guid" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/job" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -113,9 +113,9 @@ var _ api.UpstreamServer = &MockServer{} func ServerTestBattery(t *testing.T, wrap func(mock api.UpstreamServer) api.UpstreamServer) { // set up namespace := "the-space-of-names" - serviceID := flux.MustParseResourceID(namespace + "/service") - serviceList := []flux.ResourceID{serviceID} - services := flux.ResourceIDSet{} + serviceID := resource.MustParseID(namespace + "/service") + serviceList := []resource.ID{serviceID} + services := resource.IDSet{} services.Add(serviceList) now := time.Now().UTC() @@ -123,7 +123,7 @@ func ServerTestBattery(t *testing.T, wrap func(mock api.UpstreamServer) api.Upst imageID, _ := image.ParseRef("quay.io/example.com/frob:v0.4.5") serviceAnswer := []v6.ControllerStatus{ v6.ControllerStatus{ - ID: flux.MustParseResourceID("foobar/hello"), + ID: resource.MustParseID("foobar/hello"), Status: "ok", Containers: []v6.Container{ v6.Container{ @@ -140,7 +140,7 @@ func ServerTestBattery(t *testing.T, wrap func(mock api.UpstreamServer) api.Upst imagesAnswer := []v6.ImageStatus{ v6.ImageStatus{ - ID: flux.MustParseResourceID("barfoo/yello"), + ID: resource.MustParseID("barfoo/yello"), Containers: []v6.Container{ { Name: "flubnicator", diff --git a/remote/rpc/clientV11.go b/remote/rpc/clientV11.go index 5a38a7312..2f4d063c8 100644 --- a/remote/rpc/clientV11.go +++ b/remote/rpc/clientV11.go @@ -8,7 +8,7 @@ import ( "github.com/weaveworks/flux/api/v11" "github.com/weaveworks/flux/api/v6" "github.com/weaveworks/flux/remote" - ) +) // RPCClientV11 is the rpc-backed implementation of a server, for // talking to remote daemons. This version introduces methods which accept an diff --git a/remote/rpc/compat.go b/remote/rpc/compat.go index 5f8b9561d..35282bb27 100644 --- a/remote/rpc/compat.go +++ b/remote/rpc/compat.go @@ -5,13 +5,13 @@ import ( "errors" "fmt" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/api/v10" "github.com/weaveworks/flux/api/v11" "github.com/weaveworks/flux/api/v6" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/remote" + "github.com/weaveworks/flux/resource" "github.com/weaveworks/flux/update" ) @@ -29,7 +29,7 @@ func requireServiceSpecKinds(ss update.ResourceSpec, kinds []string) error { return nil } -func requireServiceIDKinds(id flux.ResourceID, kinds []string) error { +func requireServiceIDKinds(id resource.ID, kinds []string) error { _, kind, _ := id.Components() if !contains(kinds, kind) { return fmt.Errorf("Unsupported resource kind: %s", kind) @@ -40,7 +40,7 @@ func requireServiceIDKinds(id flux.ResourceID, kinds []string) error { func requireSpecKinds(s update.Spec, kinds []string) error { switch s := s.Spec.(type) { - case policy.Updates: + case resource.PolicyUpdates: for id, _ := range s { _, kind, _ := id.Components() if !contains(kinds, kind) { @@ -120,7 +120,7 @@ func listServicesWithOptions(ctx context.Context, p listServicesWithoutOptionsCl } // Polyfill the service IDs filter - want := map[flux.ResourceID]struct{}{} + want := map[resource.ID]struct{}{} for _, svc := range opts.Services { want[svc] = struct{}{} } @@ -160,7 +160,7 @@ func listImagesWithOptions(ctx context.Context, client listImagesWithoutOptionsC return statuses, err } - policyMap := map[flux.ResourceID]map[string]string{} + policyMap := map[resource.ID]map[string]string{} for _, service := range services { policyMap[service.ID] = service.Policies } diff --git a/remote/rpc/server.go b/remote/rpc/server.go index 256c8c7ba..2d9f850bf 100644 --- a/remote/rpc/server.go +++ b/remote/rpc/server.go @@ -21,7 +21,7 @@ import ( // Server takes an api.Server and makes it available over RPC. type Server struct { - server *rpc.Server + server *rpc.Server } // NewServer instantiates a new RPC server, handling requests on the diff --git a/resource/id.go b/resource/id.go new file mode 100644 index 000000000..280557f2e --- /dev/null +++ b/resource/id.go @@ -0,0 +1,240 @@ +package resource + +import ( + "encoding/json" + "fmt" + "regexp" + "sort" + "strings" + + "github.com/pkg/errors" +) + +var ( + ErrInvalidServiceID = errors.New("invalid service ID") + + LegacyServiceIDRegexp = regexp.MustCompile("^([a-zA-Z0-9_-]+)/([a-zA-Z0-9_-]+)$") + // The namespace and name components are (apparently + // non-normatively) defined in + // https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/identifiers.md + // In practice, more punctuation is used than allowed there; + // specifically, people use underscores as well as dashes and dots, and in names, colons. + IDRegexp = regexp.MustCompile("^(|[a-zA-Z0-9_-]+):([a-zA-Z0-9_-]+)/([a-zA-Z0-9_.:-]+)$") + UnqualifiedIDRegexp = regexp.MustCompile("^([a-zA-Z0-9_-]+)/([a-zA-Z0-9_.:-]+)$") +) + +// ID is an opaque type which uniquely identifies a resource in an +// orchestrator. +type ID struct { + resourceIDImpl +} + +type resourceIDImpl interface { + String() string +} + +// Old-style / format +type legacyServiceID struct { + namespace, service string +} + +func (id legacyServiceID) String() string { + return fmt.Sprintf("%s/%s", id.namespace, id.service) +} + +// New :/ format +type resourceID struct { + namespace, kind, name string +} + +func (id resourceID) String() string { + return fmt.Sprintf("%s:%s/%s", id.namespace, id.kind, id.name) +} + +// ParseID constructs a ID from a string representation +// if possible, returning an error value otherwise. +func ParseID(s string) (ID, error) { + if m := IDRegexp.FindStringSubmatch(s); m != nil { + return ID{resourceID{m[1], strings.ToLower(m[2]), m[3]}}, nil + } + if m := LegacyServiceIDRegexp.FindStringSubmatch(s); m != nil { + return ID{legacyServiceID{m[1], m[2]}}, nil + } + return ID{}, errors.Wrap(ErrInvalidServiceID, "parsing "+s) +} + +// MustParseID constructs a ID from a string representation, +// panicing if the format is invalid. +func MustParseID(s string) ID { + id, err := ParseID(s) + if err != nil { + panic(err) + } + return id +} + +// ParseIDOptionalNamespace constructs a ID from either a fully +// qualified string representation, or an unqualified kind/name representation +// and the supplied namespace. +func ParseIDOptionalNamespace(namespace, s string) (ID, error) { + if m := IDRegexp.FindStringSubmatch(s); m != nil { + return ID{resourceID{m[1], strings.ToLower(m[2]), m[3]}}, nil + } + if m := UnqualifiedIDRegexp.FindStringSubmatch(s); m != nil { + return ID{resourceID{namespace, strings.ToLower(m[1]), m[2]}}, nil + } + return ID{}, errors.Wrap(ErrInvalidServiceID, "parsing "+s) +} + +// MakeID constructs a ID from constituent components. +func MakeID(namespace, kind, name string) ID { + return ID{resourceID{namespace, strings.ToLower(kind), name}} +} + +// Components returns the constituent components of a ID +func (id ID) Components() (namespace, kind, name string) { + switch impl := id.resourceIDImpl.(type) { + case resourceID: + return impl.namespace, impl.kind, impl.name + case legacyServiceID: + return impl.namespace, "service", impl.service + default: + panic("wrong underlying type") + } +} + +// MarshalJSON encodes a ID as a JSON string. This is +// done to maintain backwards compatibility with previous flux +// versions where the ID is a plain string. +func (id ID) MarshalJSON() ([]byte, error) { + if id.resourceIDImpl == nil { + // Sadly needed as it's possible to construct an empty ID literal + return json.Marshal("") + } + return json.Marshal(id.String()) +} + +// UnmarshalJSON decodes a ID from a JSON string. This is +// done to maintain backwards compatibility with previous flux +// versions where the ID is a plain string. +func (id *ID) UnmarshalJSON(data []byte) (err error) { + var str string + if err := json.Unmarshal(data, &str); err != nil { + return err + } + if string(str) == "" { + // Sadly needed as it's possible to construct an empty ID literal + *id = ID{} + return nil + } + *id, err = ParseID(string(str)) + return err +} + +// MarshalText encodes a ID as a flat string; this is +// required because ResourceIDs are sometimes used as map keys. +func (id ID) MarshalText() (text []byte, err error) { + return []byte(id.String()), nil +} + +// MarshalText decodes a ID from a flat string; this is +// required because ResourceIDs are sometimes used as map keys. +func (id *ID) UnmarshalText(text []byte) error { + result, err := ParseID(string(text)) + if err != nil { + return err + } + *id = result + return nil +} + +type IDSet map[ID]struct{} + +func (s IDSet) String() string { + var ids []string + for id := range s { + ids = append(ids, id.String()) + } + return "{" + strings.Join(ids, ", ") + "}" +} + +func (s IDSet) Add(ids []ID) { + for _, id := range ids { + s[id] = struct{}{} + } +} + +func (s IDSet) Without(others IDSet) IDSet { + if s == nil || len(s) == 0 || others == nil || len(others) == 0 { + return s + } + res := IDSet{} + for id := range s { + if !others.Contains(id) { + res[id] = struct{}{} + } + } + return res +} + +func (s IDSet) Contains(id ID) bool { + if s == nil { + return false + } + _, ok := s[id] + return ok +} + +func (s IDSet) Intersection(others IDSet) IDSet { + if s == nil { + return others + } + if others == nil { + return s + } + result := IDSet{} + for id := range s { + if _, ok := others[id]; ok { + result[id] = struct{}{} + } + } + return result +} + +func (s IDSet) ToSlice() IDs { + i := 0 + keys := make(IDs, len(s)) + for k := range s { + keys[i] = k + i++ + } + return keys +} + +type IDs []ID + +func (p IDs) Len() int { return len(p) } +func (p IDs) Less(i, j int) bool { return p[i].String() < p[j].String() } +func (p IDs) Swap(i, j int) { p[i], p[j] = p[j], p[i] } +func (p IDs) Sort() { sort.Sort(p) } + +func (ids IDs) Without(others IDSet) (res IDs) { + for _, id := range ids { + if !others.Contains(id) { + res = append(res, id) + } + } + return res +} + +func (ids IDs) Contains(id ID) bool { + set := IDSet{} + set.Add(ids) + return set.Contains(id) +} + +func (ids IDs) Intersection(others IDSet) IDSet { + set := IDSet{} + set.Add(ids) + return set.Intersection(others) +} diff --git a/resourceid_test.go b/resource/id_test.go similarity index 87% rename from resourceid_test.go rename to resource/id_test.go index 786792cc3..32befd6fb 100644 --- a/resourceid_test.go +++ b/resource/id_test.go @@ -1,4 +1,4 @@ -package flux +package resource import ( "testing" @@ -24,14 +24,14 @@ func TestResourceIDParsing(t *testing.T) { for _, tc := range valid { t.Run(tc.name, func(t *testing.T) { - if _, err := ParseResourceID(tc.id); err != nil { + if _, err := ParseID(tc.id); err != nil { t.Error(err) } }) } for _, tc := range invalid { t.Run(tc.name, func(t *testing.T) { - if _, err := ParseResourceID(tc.id); err == nil { + if _, err := ParseID(tc.id); err == nil { t.Errorf("expected %q to be considered invalid", tc.id) } }) diff --git a/resource/policy.go b/resource/policy.go index b3910c337..fb3dbea52 100644 --- a/resource/policy.go +++ b/resource/policy.go @@ -17,7 +17,7 @@ import ( // variables. When represented in manifests, policies are expected to // have a non-empty value when present, even if it's `"true"`; so an // empty value can safely denote deletion. -func ChangesForPolicyUpdate(workload Workload, update policy.Update) (map[string]string, error) { +func ChangesForPolicyUpdate(workload Workload, update PolicyUpdate) (map[string]string, error) { add, del := update.Add, update.Remove // We may be sent the pseudo-policy `policy.TagAll`, which means // apply this filter to all containers. To do so, we need to know @@ -45,3 +45,11 @@ func ChangesForPolicyUpdate(workload Workload, update policy.Update) (map[string } return result, nil } + +type PolicyUpdates map[ID]PolicyUpdate + +type PolicyUpdate struct { + Add policy.Set `json:"add"` + Remove policy.Set `json:"remove"` +} + diff --git a/resource/resource.go b/resource/resource.go index a09a21bbf..8f0bf55db 100644 --- a/resource/resource.go +++ b/resource/resource.go @@ -1,17 +1,16 @@ package resource import ( - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/policy" ) // For the minute we just care about type Resource interface { - ResourceID() flux.ResourceID // name, to correlate with what's in the cluster - Policies() policy.Set // policy for this resource; e.g., whether it is locked, automated, ignored - Source() string // where did this come from (informational) - Bytes() []byte // the definition, for sending to cluster.Sync + ResourceID() ID // name, to correlate with what's in the cluster + Policies() policy.Set // policy for this resource; e.g., whether it is locked, automated, ignored + Source() string // where did this come from (informational) + Bytes() []byte // the definition, for sending to cluster.Sync } type Container struct { diff --git a/sync/mock.go b/sync/mock.go index cd9d5b466..814ef9604 100644 --- a/sync/mock.go +++ b/sync/mock.go @@ -1,8 +1,8 @@ package sync import ( - "github.com/weaveworks/flux" "github.com/weaveworks/flux/policy" + "github.com/weaveworks/flux/resource" ) type rsc struct { @@ -26,8 +26,8 @@ func (rs rsc) Bytes() []byte { return []byte{} } -func (rs rsc) ResourceID() flux.ResourceID { - return flux.MakeResourceID(rs.Meta.Namespace, rs.Kind, rs.Meta.Name) +func (rs rsc) ResourceID() resource.ID { + return resource.MakeID(rs.Meta.Namespace, rs.Kind, rs.Meta.Name) } func (rs rsc) Policy() policy.Set { diff --git a/update/automated.go b/update/automated.go index 0c32575ac..e2328c21d 100644 --- a/update/automated.go +++ b/update/automated.go @@ -6,7 +6,7 @@ import ( "fmt" "github.com/go-kit/kit/log" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/resource" ) @@ -16,12 +16,12 @@ type Automated struct { } type Change struct { - WorkloadID flux.ResourceID + WorkloadID resource.ID Container resource.Container ImageID image.Ref } -func (a *Automated) Add(service flux.ResourceID, container resource.Container, image image.Ref) { +func (a *Automated) Add(service resource.ID, container resource.Container, image image.Ref) { a.Changes = append(a.Changes, Change{service, container, image}) } @@ -139,18 +139,18 @@ func (a *Automated) calculateImageUpdates(rc ReleaseContext, candidates []*Workl } // workloadMap transposes the changes so they can be looked up by ID -func (a *Automated) workloadMap() map[flux.ResourceID][]Change { - set := map[flux.ResourceID][]Change{} +func (a *Automated) workloadMap() map[resource.ID][]Change { + set := map[resource.ID][]Change{} for _, change := range a.Changes { set[change.WorkloadID] = append(set[change.WorkloadID], change) } return set } -func (a *Automated) workloadIDs() []flux.ResourceID { - slice := []flux.ResourceID{} +func (a *Automated) workloadIDs() []resource.ID { + slice := []resource.ID{} for workload, _ := range a.workloadMap() { - slice = append(slice, flux.MustParseResourceID(workload.String())) + slice = append(slice, resource.MustParseID(workload.String())) } return slice } diff --git a/update/filter.go b/update/filter.go index 0482f90fe..0c6a2ba16 100644 --- a/update/filter.go +++ b/update/filter.go @@ -1,9 +1,9 @@ package update import ( - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/policy" + "github.com/weaveworks/flux/resource" ) const ( @@ -47,7 +47,7 @@ func (f *SpecificImageFilter) Filter(u WorkloadUpdate) WorkloadResult { } type ExcludeFilter struct { - IDs []flux.ResourceID + IDs []resource.ID } func (f *ExcludeFilter) Filter(u WorkloadUpdate) WorkloadResult { @@ -63,7 +63,7 @@ func (f *ExcludeFilter) Filter(u WorkloadUpdate) WorkloadResult { } type IncludeFilter struct { - IDs []flux.ResourceID + IDs []resource.ID } func (f *IncludeFilter) Filter(u WorkloadUpdate) WorkloadResult { diff --git a/update/menu.go b/update/menu.go index 2605b1530..129e3dbf7 100644 --- a/update/menu.go +++ b/update/menu.go @@ -7,7 +7,7 @@ import ( "io" "text/tabwriter" - "github.com/weaveworks/flux" + "github.com/weaveworks/flux/resource" ) const ( @@ -69,7 +69,7 @@ func (c *writer) flush() error { } type menuItem struct { - id flux.ResourceID + id resource.ID status WorkloadUpdateStatus error string update ContainerUpdate @@ -101,7 +101,7 @@ func NewMenu(out io.Writer, results Result, verbosity int) *Menu { func (m *Menu) fromResults(results Result, verbosity int) { for _, workloadID := range results.WorkloadIDs() { - resourceID := flux.MustParseResourceID(workloadID) + resourceID := resource.MustParseID(workloadID) result := results[resourceID] switch result.Status { case ReleaseStatusIgnored: @@ -147,8 +147,8 @@ func (m *Menu) AddItem(mi menuItem) { } // Run starts the interactive menu mode. -func (m *Menu) Run() (map[flux.ResourceID][]ContainerUpdate, error) { - specs := make(map[flux.ResourceID][]ContainerUpdate) +func (m *Menu) Run() (map[resource.ID][]ContainerUpdate, error) { + specs := make(map[resource.ID][]ContainerUpdate) if m.selectable == 0 { return specs, errors.New("No changes found.") } @@ -194,7 +194,7 @@ func (m *Menu) Run() (map[flux.ResourceID][]ContainerUpdate, error) { func (m *Menu) Print() { m.wr.writeln(tableHeading) - var previd flux.ResourceID + var previd resource.ID for _, item := range m.items { inline := previd == item.id m.wr.writeln(m.renderItem(item, inline)) @@ -207,7 +207,7 @@ func (m *Menu) printInteractive() { m.wr.clear() m.wr.writeln(" " + tableHeading) i := 0 - var previd flux.ResourceID + var previd resource.ID for _, item := range m.items { inline := previd == item.id m.wr.writeln(m.renderInteractiveItem(item, inline, i)) diff --git a/update/print_test.go b/update/print_test.go index ec76dd015..d56eb16f0 100644 --- a/update/print_test.go +++ b/update/print_test.go @@ -4,8 +4,8 @@ import ( "bytes" "testing" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" + "github.com/weaveworks/flux/resource" ) func mustParseRef(s string) image.Ref { @@ -26,7 +26,7 @@ func TestPrintResults(t *testing.T) { { name: "basic, just results", result: Result{ - flux.MustParseResourceID("default/helloworld"): WorkloadResult{ + resource.MustParseID("default/helloworld"): WorkloadResult{ Status: ReleaseStatusSuccess, Error: "", PerContainer: []ContainerUpdate{ @@ -47,7 +47,7 @@ default/helloworld success helloworld: quay.io/weaveworks/helloworld:master-a0 { name: "With an error, *and* results", result: Result{ - flux.MustParseResourceID("default/helloworld"): WorkloadResult{ + resource.MustParseID("default/helloworld"): WorkloadResult{ Status: ReleaseStatusSuccess, Error: "test error", PerContainer: []ContainerUpdate{ @@ -69,10 +69,10 @@ default/helloworld success test error { name: "Service results should be sorted", result: Result{ - flux.MustParseResourceID("default/d"): WorkloadResult{Status: ReleaseStatusSuccess}, - flux.MustParseResourceID("default/c"): WorkloadResult{Status: ReleaseStatusSuccess}, - flux.MustParseResourceID("default/b"): WorkloadResult{Status: ReleaseStatusSuccess}, - flux.MustParseResourceID("default/a"): WorkloadResult{Status: ReleaseStatusSuccess}, + resource.MustParseID("default/d"): WorkloadResult{Status: ReleaseStatusSuccess}, + resource.MustParseID("default/c"): WorkloadResult{Status: ReleaseStatusSuccess}, + resource.MustParseID("default/b"): WorkloadResult{Status: ReleaseStatusSuccess}, + resource.MustParseID("default/a"): WorkloadResult{Status: ReleaseStatusSuccess}, }, expected: ` WORKLOAD STATUS UPDATES diff --git a/update/release_containers.go b/update/release_containers.go index 7231ab5db..e225f9205 100644 --- a/update/release_containers.go +++ b/update/release_containers.go @@ -9,7 +9,6 @@ import ( "github.com/go-kit/kit/log" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/resource" ) @@ -19,7 +18,7 @@ var zeroImageRef = image.Ref{} // ReleaseContainersSpec defines the spec for a `containers` manifest update. type ReleaseContainersSpec struct { Kind ReleaseKind - ContainerSpecs map[flux.ResourceID][]ContainerUpdate + ContainerSpecs map[resource.ID][]ContainerUpdate SkipMismatches bool Force bool } @@ -59,7 +58,7 @@ func (s ReleaseContainersSpec) resultsError(results Result) error { } func (s ReleaseContainersSpec) filters() ([]WorkloadFilter, []WorkloadFilter) { - var rids []flux.ResourceID + var rids []resource.ID for rid := range s.ContainerSpecs { rids = append(rids, rid) } diff --git a/update/release_image.go b/update/release_image.go index 5a0add724..795a61769 100644 --- a/update/release_image.go +++ b/update/release_image.go @@ -8,10 +8,10 @@ import ( "github.com/pkg/errors" "github.com/go-kit/kit/log" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" "github.com/weaveworks/flux/policy" "github.com/weaveworks/flux/registry" + "github.com/weaveworks/flux/resource" ) const ( @@ -56,7 +56,7 @@ type ReleaseImageSpec struct { ServiceSpecs []ResourceSpec // TODO: rename to WorkloadSpecs after adding versioning ImageSpec ImageSpec Kind ReleaseKind - Excludes []flux.ResourceID + Excludes []resource.ID Force bool } @@ -119,14 +119,14 @@ func (s ReleaseImageSpec) selectWorkloads(ctx context.Context, rc ReleaseContext func (s ReleaseImageSpec) filters(rc ReleaseContext) ([]WorkloadFilter, []WorkloadFilter, error) { var prefilters, postfilters []WorkloadFilter - ids := []flux.ResourceID{} + ids := []resource.ID{} for _, ss := range s.ServiceSpecs { if ss == ResourceSpecAll { // "" Overrides any other filters - ids = []flux.ResourceID{} + ids = []resource.ID{} break } - id, err := flux.ParseResourceID(string(ss)) + id, err := resource.ParseID(string(ss)) if err != nil { return nil, nil, err } @@ -297,25 +297,25 @@ func (s ReleaseImageSpec) calculateImageUpdates(rc ReleaseContext, candidates [] return updates, nil } -type ResourceSpec string // ResourceID or "" +type ResourceSpec string // ID or "" func ParseResourceSpec(s string) (ResourceSpec, error) { if s == string(ResourceSpecAll) { return ResourceSpecAll, nil } - id, err := flux.ParseResourceID(s) + id, err := resource.ParseID(s) if err != nil { return "", errors.Wrap(err, "invalid workload spec") } return ResourceSpec(id.String()), nil } -func MakeResourceSpec(id flux.ResourceID) ResourceSpec { +func MakeResourceSpec(id resource.ID) ResourceSpec { return ResourceSpec(id.String()) } -func (s ResourceSpec) AsID() (flux.ResourceID, error) { - return flux.ParseResourceID(string(s)) +func (s ResourceSpec) AsID() (resource.ID, error) { + return resource.ParseID(string(s)) } func (s ResourceSpec) String() string { diff --git a/update/result.go b/update/result.go index 9f4c7c16e..ff057f048 100644 --- a/update/result.go +++ b/update/result.go @@ -5,8 +5,8 @@ import ( "sort" "strings" - "github.com/weaveworks/flux" "github.com/weaveworks/flux/image" + "github.com/weaveworks/flux/resource" ) type WorkloadUpdateStatus string @@ -19,7 +19,7 @@ const ( ReleaseStatusUnknown WorkloadUpdateStatus = "unknown" ) -type Result map[flux.ResourceID]WorkloadResult +type Result map[resource.ID]WorkloadResult func (r Result) WorkloadIDs() []string { var result []string @@ -30,8 +30,8 @@ func (r Result) WorkloadIDs() []string { return result } -func (r Result) AffectedResources() flux.ResourceIDs { - ids := flux.ResourceIDs{} +func (r Result) AffectedResources() resource.IDs { + ids := resource.IDs{} for id, result := range r { if result.Status == ReleaseStatusSuccess { ids = append(ids, id) diff --git a/update/spec.go b/update/spec.go index a84ef5f20..e285e72e0 100644 --- a/update/spec.go +++ b/update/spec.go @@ -4,7 +4,7 @@ import ( "encoding/json" "errors" - "github.com/weaveworks/flux/policy" + "github.com/weaveworks/flux/resource" ) const ( @@ -43,7 +43,7 @@ func (spec *Spec) UnmarshalJSON(in []byte) error { spec.Cause = wire.Cause switch wire.Type { case Policy: - var update policy.Updates + var update resource.PolicyUpdates if err := json.Unmarshal(wire.SpecBytes, &update); err != nil { return err } diff --git a/update/workload.go b/update/workload.go index 7eeacbc49..bb14fc044 100644 --- a/update/workload.go +++ b/update/workload.go @@ -1,13 +1,12 @@ package update import ( - "github.com/weaveworks/flux" "github.com/weaveworks/flux/cluster" "github.com/weaveworks/flux/resource" ) type WorkloadUpdate struct { - ResourceID flux.ResourceID + ResourceID resource.ID Workload cluster.Workload Resource resource.Workload Updates []ContainerUpdate From f16a99583cbcdac31e8022cacd6d22d2e69d7238 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Wed, 10 Jul 2019 12:46:35 +0200 Subject: [PATCH 013/158] Chart: support Helm operator `--workers` flag --- chart/flux/README.md | 1 + chart/flux/templates/helm-operator-deployment.yaml | 3 +++ chart/flux/values.yaml | 2 ++ 3 files changed, 6 insertions(+) diff --git a/chart/flux/README.md b/chart/flux/README.md index 94d3fa78d..86e680381 100755 --- a/chart/flux/README.md +++ b/chart/flux/README.md @@ -263,6 +263,7 @@ The following tables lists the configurable parameters of the Flux chart and the | `helmOperator.git.timeout` | `git.timeout` | Duration after which git operations time out | `helmOperator.git.secretName` | `None` | The name of the kubernetes secret with the SSH private key, supercedes `git.secretName` | `helmOperator.chartsSyncInterval` | `3m` | Interval at which to check for changed charts +| `helmOperator.workers` | `None` | (Experimental) amount of workers processing releases | `helmOperator.extraEnvs` | `[]` | Extra environment variables for the Helm operator pod | `helmOperator.logReleaseDiffs` | `false` | Helm operator should log the diff when a chart release diverges (possibly insecure) | `helmOperator.allowNamespace` | `None` | If set, this limits the scope to a single namespace. If not specified, all namespaces will be watched diff --git a/chart/flux/templates/helm-operator-deployment.yaml b/chart/flux/templates/helm-operator-deployment.yaml index 228856205..cf949360b 100644 --- a/chart/flux/templates/helm-operator-deployment.yaml +++ b/chart/flux/templates/helm-operator-deployment.yaml @@ -126,6 +126,9 @@ spec: - --charts-sync-interval={{ .Values.helmOperator.chartsSyncInterval }} - --update-chart-deps={{ .Values.helmOperator.updateChartDeps }} - --log-release-diffs={{ .Values.helmOperator.logReleaseDiffs }} + {{- if .Values.helmOperator.workers }} + - --workers={{ .Values.helmOperator.workers }} + {{- end }} {{- if not .Values.clusterRole.create }} - --allow-namespace={{ .Release.Namespace }} {{- else if .Values.helmOperator.allowNamespace }} diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index 285c669ab..9fc39452d 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -33,6 +33,8 @@ helmOperator: logReleaseDiffs: false # Interval at which to check for changed charts chartsSyncInterval: "3m" + # (Experimental) amount of workers processing releases + workers: # Tiller settings tillerNamespace: kube-system tls: From 835993334b404b58527512f5cc9c7c70a9630758 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Wed, 10 Jul 2019 12:25:08 +0100 Subject: [PATCH 014/158] Update changelog for v1.13.2 --- CHANGELOG.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 551f5bdc0..167b0da62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,53 @@ This is the changelog for the Flux daemon; the changelog for the Helm operator is in [./CHANGELOG-helmop.md](./CHANGELOG-helmop.md). +## 1.13.2 (2019-07-10) + +This is a patch release, including a fix for [problems with using image +labels as timestamps][weaveworks/flux#2176]. + +### Fixes + +- Because image labels are inherited from base images, fluxd cannot + indiscriminately use labels to determine the image created date. You + must now explicitly allow that behaviour with the argument + `--registry-use-labels` [weaveworks/flux#2176][] +- Image timestamps can be missing (or zero) if ordering them by semver + version rather than timestamp [weaveworks/flux#2175][] +- Environment variables needed by the Google Cloud SDK helper are now + propagated to git [weaveworks/flux#2222][] + +### Maintenance and documentation + +- Image builds are pushed to both weaveworks/ and fluxcd/ orgs on + DockerHub, in preparation for the project moving organisations + [weaveworks/flux#2213][] +- Calculate Go dependencies more efficiently during the build + [weaveworks/flux#2207][] +- Refactor to remove a spurious top-level package + [weaveworks/flux#2201][] +- Update the version of Kubernetes-in-Docker used in end-to-end test, + to v0.4.0 [weaveworks/flux#2202][] +- Bump the Ubuntu version used in CI [weaveworks/flux#2195][] + +### Thanks + +Thanks go to the following for contributions: @2opremio, @4c74356b41, +@ArchiFleKs, @adrian, @alanjcastonguay, @alexanderbuhler, +@alexhumphreys, @bobbytables, @derrickburns, @dholbach, @dlespiau, +@gaffneyd4, @hiddeco, @hkalsi, @hlascelles, @jaksonwkr, @jblunck, +@jwenz723, @linuxbsdfreak, @luxas, @mpashka, @nlamot, @semyonslepov, +@squaremo, @stefanprodan, @tegamckinney, @ysaakpr. + +[weaveworks/flux#2175]: https://github.com/weaveworks/flux/pull/2175 +[weaveworks/flux#2176]: https://github.com/weaveworks/flux/pull/2171 +[weaveworks/flux#2195]: https://github.com/weaveworks/flux/pull/2195 +[weaveworks/flux#2201]: https://github.com/weaveworks/flux/pull/2201 +[weaveworks/flux#2202]: https://github.com/weaveworks/flux/pull/2202 +[weaveworks/flux#2207]: https://github.com/weaveworks/flux/pull/2207 +[weaveworks/flux#2213]: https://github.com/weaveworks/flux/pull/2213 +[weaveworks/flux#2222]: https://github.com/weaveworks/flux/pull/2222 + ## 1.13.1 (2019-06-27) This is a patch release. From 8ff9385e0e30cca98496c2bf611f4772bf8b5701 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Wed, 10 Jul 2019 12:26:36 +0100 Subject: [PATCH 015/158] Bump fluxd version in example and chart --- CHANGELOG.md | 2 +- chart/flux/values.yaml | 2 +- deploy/flux-deployment.yaml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 167b0da62..bcac59854 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -40,7 +40,7 @@ Thanks go to the following for contributions: @2opremio, @4c74356b41, @squaremo, @stefanprodan, @tegamckinney, @ysaakpr. [weaveworks/flux#2175]: https://github.com/weaveworks/flux/pull/2175 -[weaveworks/flux#2176]: https://github.com/weaveworks/flux/pull/2171 +[weaveworks/flux#2176]: https://github.com/weaveworks/flux/pull/2176 [weaveworks/flux#2195]: https://github.com/weaveworks/flux/pull/2195 [weaveworks/flux#2201]: https://github.com/weaveworks/flux/pull/2201 [weaveworks/flux#2202]: https://github.com/weaveworks/flux/pull/2202 diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index 63fbd6668..a87313cd8 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -9,7 +9,7 @@ logFormat: fmt image: repository: docker.io/weaveworks/flux - tag: 1.13.1 + tag: 1.13.2 pullPolicy: IfNotPresent pullSecret: diff --git a/deploy/flux-deployment.yaml b/deploy/flux-deployment.yaml index 86d235576..adbd6b2fe 100644 --- a/deploy/flux-deployment.yaml +++ b/deploy/flux-deployment.yaml @@ -63,7 +63,7 @@ spec: # There are no ":latest" images for flux. Find the most recent # release or image version at https://hub.docker.com/r/weaveworks/flux/tags # and replace the tag here. - image: docker.io/weaveworks/flux:1.13.1 + image: docker.io/weaveworks/flux:1.13.2 imagePullPolicy: IfNotPresent resources: requests: From bfc77d92b6e0fe1f5c9a55ea2917fe83c6481ff8 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Wed, 10 Jul 2019 15:00:41 +0100 Subject: [PATCH 016/158] Change badges to fluxcd --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f70447194..12dff374b 100644 --- a/README.md +++ b/README.md @@ -33,8 +33,8 @@ new infrastructure. ![Deployment Pipeline](site/images/flux-cd-diagram.png) -[![CircleCI](https://circleci.com/gh/weaveworks/flux.svg?style=svg)](https://circleci.com/gh/weaveworks/flux) -[![GoDoc](https://godoc.org/github.com/weaveworks/flux?status.svg)](https://godoc.org/github.com/weaveworks/flux) +[![CircleCI](https://circleci.com/gh/fluxcd/flux.svg?style=svg)](https://circleci.com/gh/fluxcd/flux) +[![GoDoc](https://godoc.org/github.com/fluxcd/flux?status.svg)](https://godoc.org/github.com/fluxcd/flux) ## What Flux does From cf2cde9d8fac7417753268a11b6b30dea2468836 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 5 Jul 2019 11:48:34 +0200 Subject: [PATCH 017/158] Change org in image definitions to 'fluxcd' --- chart/flux/README.md | 4 ++-- chart/flux/values.yaml | 4 ++-- deploy-helm/helm-operator-deployment.yaml | 2 +- deploy-helm/weave-cloud-helm-operator-deployment.yaml | 2 +- deploy/flux-deployment.yaml | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/chart/flux/README.md b/chart/flux/README.md index 86e680381..98ccbc87f 100755 --- a/chart/flux/README.md +++ b/chart/flux/README.md @@ -183,7 +183,7 @@ The following tables lists the configurable parameters of the Flux chart and the | Parameter | Default | Description | ----------------------------------------------- | ---------------------------------------------------- | --- -| `image.repository` | `docker.io/weaveworks/flux` | Image repository +| `image.repository` | `docker.io/fluxcd/flux` | Image repository | `image.tag` | `` | Image tag | `replicaCount` | `1` | Number of Flux pods to deploy, more than one is not desirable. | `image.pullPolicy` | `IfNotPresent` | Image pull policy @@ -253,7 +253,7 @@ The following tables lists the configurable parameters of the Flux chart and the | `memcached.securityContext` | [See values.yaml](/chart/flux/values.yaml#L192-L195) | Container security context for memcached | `helmOperator.create` | `false` | If `true`, install the Helm operator | `helmOperator.createCRD` | `true` | Create the `v1beta1` and `v1alpha2` Flux CRDs. Dependent on `helmOperator.create=true` -| `helmOperator.repository` | `docker.io/weaveworks/helm-operator` | Helm operator image repository +| `helmOperator.repository` | `docker.io/fluxcd/helm-operator` | Helm operator image repository | `helmOperator.tag` | `` | Helm operator image tag | `helmOperator.replicaCount` | `1` | Number of helm operator pods to deploy, more than one is not desirable. | `helmOperator.pullPolicy` | `IfNotPresent` | Helm operator image pull policy diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index c9086aea2..c4d72566d 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -8,7 +8,7 @@ replicaCount: 1 logFormat: fmt image: - repository: docker.io/weaveworks/flux + repository: docker.io/fluxcd/flux tag: 1.13.2 pullPolicy: IfNotPresent pullSecret: @@ -21,7 +21,7 @@ helmOperator: replicaCount: 1 create: false createCRD: false - repository: docker.io/weaveworks/helm-operator + repository: docker.io/fluxcd/helm-operator tag: 0.10.0 pullPolicy: IfNotPresent pullSecret: diff --git a/deploy-helm/helm-operator-deployment.yaml b/deploy-helm/helm-operator-deployment.yaml index 0db1ed8af..3f3afa1f9 100644 --- a/deploy-helm/helm-operator-deployment.yaml +++ b/deploy-helm/helm-operator-deployment.yaml @@ -62,7 +62,7 @@ spec: # There are no ":latest" images for helm-operator. Find the most recent # release or image version at https://hub.docker.com/r/weaveworks/helm-operator/tags # and replace the tag here. - image: docker.io/weaveworks/helm-operator:0.10.0 + image: docker.io/fluxcd/helm-operator:0.10.0 imagePullPolicy: IfNotPresent ports: - name: http diff --git a/deploy-helm/weave-cloud-helm-operator-deployment.yaml b/deploy-helm/weave-cloud-helm-operator-deployment.yaml index 8080094e3..920f6c6a6 100644 --- a/deploy-helm/weave-cloud-helm-operator-deployment.yaml +++ b/deploy-helm/weave-cloud-helm-operator-deployment.yaml @@ -27,7 +27,7 @@ spec: secretName: flux-git-deploy containers: - name: flux-helm-operator - image: docker.io/weaveworks/helm-operator:0.10.0 + image: docker.io/fluxcd/helm-operator:0.10.0 imagePullPolicy: IfNotPresent args: - --git-timeout=20s diff --git a/deploy/flux-deployment.yaml b/deploy/flux-deployment.yaml index 5f493da1c..6af51af25 100644 --- a/deploy/flux-deployment.yaml +++ b/deploy/flux-deployment.yaml @@ -63,7 +63,7 @@ spec: # There are no ":latest" images for flux. Find the most recent # release or image version at https://hub.docker.com/r/weaveworks/flux/tags # and replace the tag here. - image: docker.io/weaveworks/flux:1.13.2 + image: docker.io/fluxcd/flux:1.13.2 imagePullPolicy: IfNotPresent resources: requests: From 992582c0d6997560cc1af961612106178dbf0d68 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 5 Jul 2019 11:54:41 +0200 Subject: [PATCH 018/158] Change org in image definition docs to 'fluxcd' --- site/get-started-developing.md | 18 +++++++++--------- site/troubleshooting.md | 2 +- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/site/get-started-developing.md b/site/get-started-developing.md index 93d7a7648..2e3f9606e 100644 --- a/site/get-started-developing.md +++ b/site/get-started-developing.md @@ -7,7 +7,7 @@ This guide shows a workflow for making a small (actually, tiny) change to Flux, > From a very high level, there are at least 3 ways you can develop on Flux once you have your environment set up: > 1. The "minimalist" approach (only requires and `kubectl`): > 1. `make` -> 1. copy the specific image tag (e.g. `docker.io/weaveworks/flux:master-a86167e4`) for what you just built and paste it into `/deploy/flux-deployment.yaml` as the image you're targeting to deploy +> 1. copy the specific image tag (e.g. `docker.io/fluxcd/flux:master-a86167e4`) for what you just built and paste it into `/deploy/flux-deployment.yaml` as the image you're targeting to deploy > 1. deploy the resources in `/develop/*.yaml` manually with `kubectl apply` > 1. make a change to the code > 1. see your code changes have been deployed @@ -98,19 +98,19 @@ Now that we know everything is working with `flux-getting-started`, we're going In the same terminal you ran `eval $(minikube docker-env)`, run `GO111MODULE=on go mod download` followed by `make` from the root directory of the Flux repo. You'll see docker's usual output as it builds the image layers. Once it's done, you should see something like this in the middle of the output: ``` Successfully built 606610e0f4ef - Successfully tagged docker.io/weaveworks/flux:latest - Successfully tagged docker.io/weaveworks/flux:master-a86167e4 + Successfully tagged docker.io/fluxcd/flux:latest + Successfully tagged docker.io/fluxcd/flux:master-a86167e4 ``` This confirms that a new docker image was tagged for your image. -1. Open up [`deploy/flux-deployment.yaml`](deploy/flux-deployment.yaml) and update the image at `spec.template.spec.containers[0].image` to be simply `docker.io/weaveworks/flux`. While we're here, also change the `git-url` to point towards your fork. It will look something like this in the yaml: +1. Open up [`deploy/flux-deployment.yaml`](deploy/flux-deployment.yaml) and update the image at `spec.template.spec.containers[0].image` to be simply `docker.io/fluxcd/flux`. While we're here, also change the `git-url` to point towards your fork. It will look something like this in the yaml: ```yaml spec: template: spec: containers: - name: flux - image: docker.io/weaveworks/flux + image: docker.io/fluxcd/flux imagePullPolicy: IfNotPresent args: - --git-url=git@github.com:/flux-getting-started @@ -157,8 +157,8 @@ Now that we know everything is working with `flux-getting-started`, we're going You should see an output that looks something like this: ``` - ts=2019-02-28T18:58:45.091531939Z caller=warming.go:268 component=warmer info="refreshing image" image=docker.io/weaveworks/flux tag_count=60 to_update=60 of_which_refresh=0 of_which_missing=60 - ts=2019-02-28T18:58:46.233723421Z caller=warming.go:364 component=warmer updated=docker.io/weaveworks/flux successful=60 attempted=60 + ts=2019-02-28T18:58:45.091531939Z caller=warming.go:268 component=warmer info="refreshing image" image=docker.io/fluxcd/flux tag_count=60 to_update=60 of_which_refresh=0 of_which_missing=60 + ts=2019-02-28T18:58:46.233723421Z caller=warming.go:364 component=warmer updated=docker.io/fluxcd/flux successful=60 attempted=60 ts=2019-02-28T18:58:46.234086642Z caller=images.go:17 component=sync-loop msg="polling images" ts=2019-02-28T18:58:46.234125646Z caller=images.go:27 component=sync-loop msg="no automated services" ts=2019-02-28T18:58:46.749598558Z caller=warming.go:268 component=warmer info="refreshing image" image=memcached tag_count=66 to_update=66 of_which_refresh=0 of_which_missing=66 @@ -186,9 +186,9 @@ Now that we know everything is working with `flux-getting-started`, we're going flux-6f7fd5bbc-6j9d5 1/1 Running 0 10s ``` - This pod was deployed even though we didn't run any `kubectl` commands or interact with Kubernetes directly because of the `freshpod` Minikube addon that we enabled earlier. Freshpod saw that a new Docker image was tagged for `docker.io/weaveworks/flux:latest` and it went ahead and redeployed that pod for us. + This pod was deployed even though we didn't run any `kubectl` commands or interact with Kubernetes directly because of the `freshpod` Minikube addon that we enabled earlier. Freshpod saw that a new Docker image was tagged for `docker.io/fluxcd/flux:latest` and it went ahead and redeployed that pod for us. - Consider that simply applying the `flux-deployment.yaml` file again wouldn't do anything since the actual image we're targeting (which is actually `docker.io/weaveworks/flux` with no `:latest` tag, but it's the same difference) hasn't changed. The Kubernetes api server will get that JSON request from kubectl and go: "right... so nothing has changed in the file so I have nothing to do... IGNORE!". + Consider that simply applying the `flux-deployment.yaml` file again wouldn't do anything since the actual image we're targeting (which is actually `docker.io/fluxcd/flux` with no `:latest` tag, but it's the same difference) hasn't changed. The Kubernetes api server will get that JSON request from kubectl and go: "right... so nothing has changed in the file so I have nothing to do... IGNORE!". There is another way to do this, of course. Remember that before when we ran `make` that we did _also_ get an image tagged with the `:-` syntax (in our specific example above it was `:master-a86167e4`). We could, in theory, grab that tag every time we `make`, and then paste it into `spec.template.spec.containers[0].image` of our deployment. That's tedious and error prone. Instead, `freshpod` cuts this step out for us and accomplishes the same end goal. diff --git a/site/troubleshooting.md b/site/troubleshooting.md index 159001685..55703833c 100644 --- a/site/troubleshooting.md +++ b/site/troubleshooting.md @@ -97,7 +97,7 @@ reasons this can happen: ```yaml spec: containers: - image: docker.io/weaveworks/flux + image: docker.io/fluxcd/flux ... volumeMounts: - name: acr-credentials From b74cb7b9558c07c7985eee1af6e7942a8e486d03 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 5 Jul 2019 11:56:59 +0200 Subject: [PATCH 019/158] Change org in image definition tests to 'fluxcd' --- api/v9/change_test.go | 2 +- image/image.go | 4 ++-- registry/cache/registry.go | 2 +- registry/client.go | 2 +- registry/imageentry_test.go | 4 ++-- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/api/v9/change_test.go b/api/v9/change_test.go index cdc2c4432..136ff3db3 100644 --- a/api/v9/change_test.go +++ b/api/v9/change_test.go @@ -9,7 +9,7 @@ import ( ) func TestChangeEncoding(t *testing.T) { - ref, _ := image.ParseRef("docker.io/weaveworks/flux") + ref, _ := image.ParseRef("docker.io/fluxcd/flux") name := ref.Name for _, update := range []Change{ diff --git a/image/image.go b/image/image.go index e8e56b10e..5f9a49afb 100644 --- a/image/image.go +++ b/image/image.go @@ -33,7 +33,7 @@ var ( // Examples (stringified): // * alpine // * library/alpine -// * docker.io/weaveworks/flux +// * docker.io/fluxcd/flux // * localhost:5000/arbitrary/path/to/repo type Name struct { Domain, Image string @@ -107,7 +107,7 @@ func (i Name) ToRef(tag string) Ref { // Examples (stringified): // * alpine:3.5 // * library/alpine:3.5 -// * docker.io/weaveworks/flux:1.1.0 +// * docker.io/fluxcd/flux:1.1.0 // * localhost:5000/arbitrary/path/to/repo:revision-sha1 type Ref struct { Name diff --git a/registry/cache/registry.go b/registry/cache/registry.go index d238dc5e9..89df2edc7 100644 --- a/registry/cache/registry.go +++ b/registry/cache/registry.go @@ -74,7 +74,7 @@ func (l TimestampLabelWhitelist) apply(r *ImageRepository) { } // GetImageRepositoryMetadata returns the metadata from an image -// repository (e.g,. at "docker.io/weaveworks/flux") +// repository (e.g,. at "docker.io/fluxcd/flux") func (c *Cache) GetImageRepositoryMetadata(id image.Name) (image.RepositoryMetadata, error) { repoKey := NewRepositoryKey(id.CanonicalName()) bytes, _, err := c.Reader.GetKey(repoKey) diff --git a/registry/client.go b/registry/client.go index b34de4635..2afe6dc02 100644 --- a/registry/client.go +++ b/registry/client.go @@ -56,7 +56,7 @@ func (entry *ImageEntry) UnmarshalJSON(bytes []byte) error { } // Client is a remote registry client for a particular image -// repository (e.g., for docker.io/weaveworks/flux). It is an interface +// repository (e.g., for docker.io/fluxcd/flux). It is an interface // so we can wrap it in instrumentation, write fake implementations, // and so on. type Client interface { diff --git a/registry/imageentry_test.go b/registry/imageentry_test.go index 6263f1234..0789745f8 100644 --- a/registry/imageentry_test.go +++ b/registry/imageentry_test.go @@ -22,7 +22,7 @@ func TestImageEntryRoundtrip(t *testing.T) { assert.Equal(t, entry, entry2) } - ref, err := image.ParseRef("docker.io/weaveworks/flux:1.0.0") + ref, err := image.ParseRef("docker.io/fluxcd/flux:1.0.0") assert.NoError(t, err) info := image.Info{ @@ -44,7 +44,7 @@ func TestImageEntryRoundtrip(t *testing.T) { // Check that existing entries, which are image.Info, will parse into // the ImageEntry struct. func TestImageInfoParsesAsEntry(t *testing.T) { - ref, err := image.ParseRef("docker.io/weaveworks/flux:1.0.0") + ref, err := image.ParseRef("docker.io/fluxcd/flux:1.0.0") assert.NoError(t, err) info := image.Info{ ID: ref, From 7d48f03a2d16016b59a0e93b8b83935cebd4d962 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Wed, 10 Jul 2019 18:00:44 +0200 Subject: [PATCH 020/158] Update labels in Dockerfiles to match `fluxcd` org --- docker/Dockerfile.flux | 18 +++++++++--------- docker/Dockerfile.helm-operator | 18 +++++++++--------- 2 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docker/Dockerfile.flux b/docker/Dockerfile.flux index dd629fd40..bef07cfe5 100644 --- a/docker/Dockerfile.flux +++ b/docker/Dockerfile.flux @@ -18,18 +18,18 @@ COPY ./kubectl /usr/local/bin/ COPY ./kustomize /usr/local/bin # These are pretty static -LABEL maintainer="Weaveworks " \ +LABEL maintainer="Flux CD " \ org.opencontainers.image.title="flux" \ - org.opencontainers.image.description="The Flux daemon, for synchronising your cluster with a git repo, and deploying new images" \ - org.opencontainers.image.url="https://github.com/weaveworks/flux" \ - org.opencontainers.image.source="git@github.com:weaveworks/flux" \ - org.opencontainers.image.vendor="Weaveworks" \ + org.opencontainers.image.description="The GitOps operator for Kubernetes" \ + org.opencontainers.image.url="https://github.com/fluxcd/flux" \ + org.opencontainers.image.source="git@github.com:fluxcd/flux" \ + org.opencontainers.image.vendor="Flux CD" \ org.label-schema.schema-version="1.0" \ org.label-schema.name="flux" \ - org.label-schema.description="The Flux daemon, for synchronising your cluster with a git repo, and deploying new images" \ - org.label-schema.url="https://github.com/weaveworks/flux" \ - org.label-schema.vcs-url="git@github.com:weaveworks/flux" \ - org.label-schema.vendor="Weaveworks" + org.label-schema.description="The GitOps operator for Kubernetes" \ + org.label-schema.url="https://github.com/fluxcd/flux" \ + org.label-schema.vcs-url="git@github.com:fluxcd/flux" \ + org.label-schema.vendor="Flux CD" ENTRYPOINT [ "/sbin/tini", "--", "fluxd" ] diff --git a/docker/Dockerfile.helm-operator b/docker/Dockerfile.helm-operator index 9b70a9e62..dcaa7ef7e 100644 --- a/docker/Dockerfile.helm-operator +++ b/docker/Dockerfile.helm-operator @@ -18,18 +18,18 @@ COPY ./kubectl /usr/local/bin/ COPY ./helm /usr/local/bin/ # These are pretty static -LABEL maintainer="Weaveworks " \ +LABEL maintainer="Flux CD " \ org.opencontainers.image.title="flux-helm-operator" \ - org.opencontainers.image.description="The Flux Helm operator, for releasing Helm charts according to git" \ - org.opencontainers.image.url="https://github.com/weaveworks/flux" \ - org.opencontainers.image.source="git@github.com:weaveworks/flux" \ - org.opencontainers.image.vendor="Weaveworks" \ + org.opencontainers.image.description="The Helm operator for Kubernetes" \ + org.opencontainers.image.url="https://github.com/fluxcd/flux" \ + org.opencontainers.image.source="git@github.com:fluxcd/flux" \ + org.opencontainers.image.vendor="Flux CD" \ org.label-schema.schema-version="1.0" \ org.label-schema.name="flux-helm-operator" \ - org.label-schema.description="The Flux Helm operator, for releasing Helm charts according to git" \ - org.label-schema.url="https://github.com/weaveworks/flux" \ - org.label-schema.vcs-url="git@github.com:weaveworks/flux" \ - org.label-schema.vendor="Weaveworks" + org.label-schema.description="The Helm operator for Kubernetes" \ + org.label-schema.url="https://github.com/fluxcd/flux" \ + org.label-schema.vcs-url="git@github.com:fluxcd/flux" \ + org.label-schema.vendor="Flux CD" ENTRYPOINT [ "/sbin/tini", "--", "helm-operator" ] From 76a2fa4814da1ab300fbbd7b4dfa2fbf3ddbcecd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Felix=20Mei=C3=9Fner?= <22190992+fnmeissner@users.noreply.github.com> Date: Wed, 10 Jul 2019 17:05:34 +0200 Subject: [PATCH 021/158] helm repository is now fluxcd instead weaveworks The helm repository has moved to fluxcd and the documentation should be updated accordingly. See #2241 --- site/helm-get-started.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site/helm-get-started.md b/site/helm-get-started.md index dc5dac3d3..a31d647ab 100644 --- a/site/helm-get-started.md +++ b/site/helm-get-started.md @@ -62,16 +62,16 @@ helm init --skip-refresh --upgrade --service-account tiller --history-max 10 ## Install Flux -Add the Flux repository of Weaveworks: +Add the Flux repository: ```sh -helm repo add weaveworks https://weaveworks.github.io/flux +helm repo add fluxcd https://fluxcd.github.io/flux ``` Apply the Helm Release CRD: ```sh -kubectl apply -f https://raw.githubusercontent.com/weaveworks/flux/master/deploy-helm/flux-helm-release-crd.yaml +kubectl apply -f https://raw.githubusercontent.com/fluxcd/flux/master/deploy-helm/flux-helm-release-crd.yaml ``` In this next step you install Flux using `helm`. Simply @@ -92,7 +92,7 @@ In this next step you install Flux using `helm`. Simply --set helmOperator.createCRD=false \ --set git.url=git@github.com:YOURUSER/flux-get-started \ --namespace flux \ - weaveworks/flux + fluxcd/flux ``` - Using a private git server: From afff06d0a03b29ddc9ced118ed1e3a2eedabd3ed Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Wed, 10 Jul 2019 18:37:00 +0200 Subject: [PATCH 022/158] Release chart v0.11.0 --- chart/flux/CHANGELOG.md | 27 ++++++++++++++++++++++++++- chart/flux/Chart.yaml | 10 +++++----- 2 files changed, 31 insertions(+), 6 deletions(-) diff --git a/chart/flux/CHANGELOG.md b/chart/flux/CHANGELOG.md index 1528f3f62..c27c7f34a 100644 --- a/chart/flux/CHANGELOG.md +++ b/chart/flux/CHANGELOG.md @@ -1,4 +1,29 @@ -## 0.11.0 (2019-06-27) +## 0.11.0 (2019-07-10) + +### Improvements + + - Updated Flux to `1.13.2` and the Helm operator to `0.10.0` + [fluxcd/flux#2235](https://github.com/fluxcd/flux/pull/2235) + [fluxcd/flux#2237](https://github.com/fluxcd/flux/pull/2237) + - Changed from DockerHub organization `weaveworks` -> `fluxcd` + [fluxcd/flux#2224](https://github.com/fluxcd/flux/pull/2224) + - Updated `HelmRelease` CRD to support rollbacks + [fluxcd/flux#2006](https://github.com/fluxcd/flux/pull/2006) + - Allow namespace scoping for both Flux and the Helm operator + [fluxcd/flux#2206](https://github.com/fluxcd/flux/pull/2206) + [fluxcd/flux#2209](https://github.com/fluxcd/flux/pull/2209) + - Removed long deprecated `FluxHelmRelease` CRD and disabled CRD + creation as the default to follow our own best practices + [fluxcd/flux#2190](https://github.com/fluxcd/flux/pull/2190) + - Enable `PodSecurityPolicy` + [fluxcd/flux#2223](https://github.com/fluxcd/flux/pull/2223) + [fluxcd/flux#2225](https://github.com/fluxcd/flux/pull/2225) + - Support new Flux `--registry-use-labels` flag (`registry.useTimestampLabels`) + [fluxcd/flux#2176](https://github.com/fluxcd/flux/pull/2176) + - Support new Helm operator `--workers` flag (`helmOperator.workers`) + [fluxcd/flux#2236](https://github.com/fluxcd/flux/pull/2236) + +## 0.10.2 (2019-06-27) ### Improvements diff --git a/chart/flux/Chart.yaml b/chart/flux/Chart.yaml index befb491af..124406bd8 100644 --- a/chart/flux/Chart.yaml +++ b/chart/flux/Chart.yaml @@ -1,16 +1,16 @@ apiVersion: v1 -appVersion: "1.13.1" -version: 0.10.2 +appVersion: "1.13.2" +version: 0.11.0 kubeVersion: ">=1.9.0-0" name: flux description: Flux is a tool that automatically ensures that the state of a cluster matches what is specified in version control -home: https://weave.works +home: https://fluxcd.io sources: -- https://github.com/weaveworks/flux +- https://github.com/fluxcd/flux maintainers: - name: stefanprodan email: stefan@weave.works engine: gotpl -icon: https://raw.githubusercontent.com/weaveworks/flux/master/site/images/weave-flux.png +icon: https://raw.githubusercontent.com/fluxcd/flux/master/site/images/weave-flux.png keywords: - gitops From efccecbd35bf8788c3c1ed22bf4c677bf97649cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Felix=20Mei=C3=9Fner?= Date: Thu, 11 Jul 2019 14:08:23 +0200 Subject: [PATCH 023/158] parse credentials without scheme When the host within the docker config has format IP:PORT url.Parse(host) fails. This commit will first try reparsing with prepended scheme before returning any error. --- registry/credentials.go | 23 ++++++++++++----------- registry/credentials_test.go | 4 ++++ 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/registry/credentials.go b/registry/credentials.go index 9962b8903..ed9e53419 100644 --- a/registry/credentials.go +++ b/registry/credentials.go @@ -74,28 +74,29 @@ func ParseCredentials(from string, b []byte) (Credentials, error) { return Credentials{}, err } + if host == "http://" || host == "https://" { + return Credentials{}, errors.New("Empty registry auth url") + } + // Some users were passing in credentials in the form of // http://docker.io and http://docker.io/v1/, etc. // So strip everything down to the host. // Also, the registry might be local and on a different port. // So we need to check for that because url.Parse won't parse the ip:port format very well. u, err := url.Parse(host) - if err != nil { - return Credentials{}, err - } - if u.Host == "" && u.Path == "" && !strings.Contains(host, ":") || host == "http://" || host == "https://" { - return Credentials{}, errors.New("Empty registry auth url") - } - if u.Host == "" { // If there's no https:// prefix, it won't parse the host. + + // if anything went wrong try to prepend https:// + if err != nil || u.Host == "" { u, err = url.Parse(fmt.Sprintf("https://%s/", host)) if err != nil { return Credentials{}, err } - // If the host is still empty, then there's probably a rogue / - if u.Host == "" { - return Credentials{}, errors.New("Invalid registry auth url. Must be a valid http address (e.g. https://gcr.io/v1/)") - } } + + if u.Host == "" { // If host is still empty the url must be broken. + return Credentials{}, errors.New("Invalid registry auth url. Must be a valid http address (e.g. https://gcr.io/v1/)") + } + host = u.Host creds.registry = host diff --git a/registry/credentials_test.go b/registry/credentials_test.go index b93c8f282..44c013ebc 100644 --- a/registry/credentials_test.go +++ b/registry/credentials_test.go @@ -39,6 +39,10 @@ func TestRemoteFactory_ParseHost(t *testing.T) { host: "localhost:5000/v2/", imagePrefix: "localhost:5000", }, + { + host: "192.168.99.100:5000", + imagePrefix: "192.168.99.100:5000", + }, { host: "https://192.168.99.100:5000/v2", imagePrefix: "192.168.99.100:5000", From 2363a65a8bc3bd58e6b3ed831856df319a1f8754 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 12 Jul 2019 14:28:03 +0200 Subject: [PATCH 024/158] Fixed rollback.timeout type definition in CRD Trying to a apply a `HelmRelease` with a set `rollback.timeout` would result in a validation error: `spec.rollback.timeout in body must be of type int64: "integer"`. This was due to the fact that the validation scheme definition contained a mistake, the `type` should be `integer` and the format is `int64`. --- chart/flux/templates/helm-operator-crd.yaml | 3 ++- deploy-helm/flux-helm-release-crd.yaml | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/chart/flux/templates/helm-operator-crd.yaml b/chart/flux/templates/helm-operator-crd.yaml index 626cc0ecd..dfadfbb2a 100644 --- a/chart/flux/templates/helm-operator-crd.yaml +++ b/chart/flux/templates/helm-operator-crd.yaml @@ -55,7 +55,8 @@ spec: disableHooks: type: boolean timeout: - type: int64 + type: integer + format: int64 wait: type: boolean valueFileSecrets: diff --git a/deploy-helm/flux-helm-release-crd.yaml b/deploy-helm/flux-helm-release-crd.yaml index 11dae71dd..307efdf0e 100644 --- a/deploy-helm/flux-helm-release-crd.yaml +++ b/deploy-helm/flux-helm-release-crd.yaml @@ -47,7 +47,8 @@ spec: disableHooks: type: boolean timeout: - type: int64 + type: integer + format: int64 wait: type: boolean valueFileSecrets: From 371807a8a26b18781c8516ae03f4654030e8e4fe Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Fri, 12 Jul 2019 14:50:12 +0200 Subject: [PATCH 025/158] Use correct Helm repository domain in docs As it has changed after our move to the `fluxcd` organization, and the old URL no longer works. --- chart/flux/README.md | 8 ++++---- site/annotations-tutorial.md | 6 +++--- site/helm-operator.md | 4 ++-- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/chart/flux/README.md b/chart/flux/README.md index 98ccbc87f..d2994253b 100755 --- a/chart/flux/README.md +++ b/chart/flux/README.md @@ -47,7 +47,7 @@ You will have a fully working Flux installation deploying workloads to your clus Add the weaveworks repo: ```sh -helm repo add weaveworks https://weaveworks.github.io/flux +helm repo add fluxcd https://fluxcd.github.io/flux ``` #### To install the chart with the release name `flux` @@ -58,7 +58,7 @@ Replace `weaveworks/flux-get-started` with your own git repository and run helm $ helm install --name flux \ --set git.url=git@github.com:weaveworks/flux-get-started \ --namespace flux \ -weaveworks/flux +fluxcd/flux ``` #### To connect Flux to a Weave Cloud instance: @@ -68,7 +68,7 @@ helm install --name flux \ --set git.url=git@github.com:weaveworks/flux-get-started \ --set token=YOUR_WEAVE_CLOUD_SERVICE_TOKEN \ --namespace flux \ -weaveworks/flux +fluxcd/flux ``` #### To install Flux with the Helm operator: @@ -87,7 +87,7 @@ $ helm install --name flux \ --set helmOperator.create=true \ --set helmOperator.createCRD=false \ --namespace flux \ -weaveworks/flux +fluxcd/flux ``` #### To install Flux with a private git host: diff --git a/site/annotations-tutorial.md b/site/annotations-tutorial.md index d0ab3f73b..bc7459d08 100644 --- a/site/annotations-tutorial.md +++ b/site/annotations-tutorial.md @@ -74,13 +74,13 @@ Now you can take care of the actual installation. First add the Flux repository of Weaveworks: ```sh -helm repo add weaveworks https://weaveworks.github.io/flux +helm repo add fluxcd https://fluxcd.github.io/flux ``` Apply the Helm Release CRD: ```sh -kubectl apply -f https://raw.githubusercontent.com/weaveworks/flux/master/deploy-helm/flux-helm-release-crd.yaml +kubectl apply -f https://raw.githubusercontent.com/fluxcd/flux/master/deploy-helm/flux-helm-release-crd.yaml ``` Install Flux and its Helm Operator by specifying your fork URL. Just @@ -93,7 +93,7 @@ helm upgrade -i Flux \ --set helmOperator.createCRD=false \ --set git.url=git@github.com:YOURUSER/flux-get-started \ --namespace default \ -weaveworks/flux +fluxcd/flux ``` > **Note:** In this tutorial we keep things simple, so we deploy Flux into diff --git a/site/helm-operator.md b/site/helm-operator.md index d3d3f59c2..c8d75dc48 100644 --- a/site/helm-operator.md +++ b/site/helm-operator.md @@ -184,7 +184,7 @@ kubectl create secret tls helm-client --cert=tls/flux-helm-operator.pem --key=./ Deploy Flux with Helm; ```bash -helm repo add weaveworks https://weaveworks.github.io/flux +helm repo add fluxcd https://fluxcd.github.io/flux helm upgrade --install \ --set helmOperator.create=true \ @@ -194,7 +194,7 @@ helm upgrade --install \ --set helmOperator.tls.secretName=helm-client \ --set helmOperator.tls.caContent="$(cat ./tls/ca.pem)" \ flux \ - weaveworks/flux + fluxcd/flux ``` > **Note:** > - include --tls flags for `helm` as in the `helm ls` example, if talking to a tiller with TLS From def50b6286aff08ce2816317706e12dae2e41ad2 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Mon, 15 Jul 2019 10:26:26 +0100 Subject: [PATCH 026/158] Run credentials tests in parallel A small change to use testing.T.Run for the credentials parsing test; also removes a time.Sleep that according to git blame, I left in there long ago. --- registry/credentials_test.go | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/registry/credentials_test.go b/registry/credentials_test.go index 44c013ebc..d8473f607 100644 --- a/registry/credentials_test.go +++ b/registry/credentials_test.go @@ -4,7 +4,6 @@ import ( "encoding/base64" "fmt" "testing" - "time" "github.com/stretchr/testify/assert" ) @@ -88,19 +87,20 @@ func TestRemoteFactory_ParseHost(t *testing.T) { error: true, }, } { - stringCreds := fmt.Sprintf(tmpl, v.host, okCreds) - creds, err := ParseCredentials("test", []byte(stringCreds)) - time.Sleep(100 * time.Millisecond) - if (err != nil) != v.error { - t.Fatalf("For test %q, expected error = %v but got %v", v.host, v.error, err != nil) - } - if v.error { - continue - } - actualUser := creds.credsFor(v.imagePrefix).username - assert.Equal(t, user, actualUser, "For test %q, expected %q but got %v", v.host, user, actualUser) - actualPass := creds.credsFor(v.imagePrefix).password - assert.Equal(t, pass, actualPass, "For test %q, expected %q but got %v", v.host, user, actualPass) + t.Run(v.host, func(t *testing.T) { + stringCreds := fmt.Sprintf(tmpl, v.host, okCreds) + creds, err := ParseCredentials("test", []byte(stringCreds)) + if (err != nil) != v.error { + t.Fatalf("For test %q, expected error = %v but got %v", v.host, v.error, err != nil) + } + if v.error { + return + } + actualUser := creds.credsFor(v.imagePrefix).username + assert.Equal(t, user, actualUser, "For test %q, expected %q but got %v", v.host, user, actualUser) + actualPass := creds.credsFor(v.imagePrefix).password + assert.Equal(t, pass, actualPass, "For test %q, expected %q but got %v", v.host, user, actualPass) + }) } } From 68f932fee69306a92e622b0609151a967a335fe1 Mon Sep 17 00:00:00 2001 From: Jiri Pinkava Date: Tue, 9 Jul 2019 20:35:32 +0200 Subject: [PATCH 027/158] Re-implement registry.ecr.require As it got removed by accident (probably during a rebase) in #1918. --- chart/flux/templates/deployment.yaml | 3 +++ chart/flux/values.yaml | 1 + 2 files changed, 4 insertions(+) diff --git a/chart/flux/templates/deployment.yaml b/chart/flux/templates/deployment.yaml index 5b44f4778..59f82efee 100644 --- a/chart/flux/templates/deployment.yaml +++ b/chart/flux/templates/deployment.yaml @@ -190,6 +190,9 @@ spec: - --registry-rps={{ .Values.registry.rps }} - --registry-burst={{ .Values.registry.burst }} - --registry-trace={{ .Values.registry.trace }} + {{- if .Values.registry.ecr.require }} + - --registry-require=ecr + {{- end }} {{- if .Values.registry.insecureHosts }} - --registry-insecure-host={{ .Values.registry.insecureHosts }} {{- end }} diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index c4d72566d..9813663f3 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -193,6 +193,7 @@ registry: region: includeId: excludeId: + require: false # Azure ACR settings acr: enabled: false From 5644b6ea402e73f25eeb18a31a3dcdafdeb104b6 Mon Sep 17 00:00:00 2001 From: Jiri Pinkava Date: Mon, 15 Jul 2019 16:32:52 +0200 Subject: [PATCH 028/158] Log correct `--registry-require` flag (remove D from end) --- cmd/fluxd/main.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cmd/fluxd/main.go b/cmd/fluxd/main.go index 81a29befc..25e07c2b0 100644 --- a/cmd/fluxd/main.go +++ b/cmd/fluxd/main.go @@ -288,7 +288,7 @@ func main() { possiblyRequired := stringset(RequireValues) for _, r := range *registryRequire { if !possiblyRequired.has(r) { - logger.Log("err", fmt.Sprintf("--registry-required value %q is not in possible values {%s}", r, strings.Join(RequireValues, ","))) + logger.Log("err", fmt.Sprintf("--registry-require value %q is not in possible values {%s}", r, strings.Join(RequireValues, ","))) os.Exit(1) } } @@ -454,7 +454,7 @@ func main() { awsPreflight, credsWithAWSAuth := registry.ImageCredsWithAWSAuth(imageCreds, log.With(logger, "component", "aws"), awsConf) if mandatoryRegistry.has(RequireECR) { if err := awsPreflight(); err != nil { - logger.Log("error", "AWS API required (due to --registry-required=ecr), but not available", "err", err) + logger.Log("error", "AWS API required (due to --registry-require=ecr), but not available", "err", err) os.Exit(1) } } From 4377e8583ce733ef18eba5dceb06c5393bd8de24 Mon Sep 17 00:00:00 2001 From: Hidde Beydals Date: Mon, 15 Jul 2019 13:48:35 +0200 Subject: [PATCH 029/158] Make `fluxctl` respect the kube config context NS By looking at the configured namespace in the active context of the resolved config, if none is found we return the given `defaultNamespace`. --- cmd/fluxctl/args.go | 19 +++++++++++++++++++ cmd/fluxctl/automate_cmd.go | 2 +- cmd/fluxctl/deautomate_cmd.go | 2 +- cmd/fluxctl/list_images_cmd.go | 2 +- cmd/fluxctl/list_workloads_cmd.go | 2 +- cmd/fluxctl/lock_cmd.go | 2 +- cmd/fluxctl/policy_cmd.go | 2 +- cmd/fluxctl/release_cmd.go | 2 +- cmd/fluxctl/unlock_cmd.go | 2 +- 9 files changed, 27 insertions(+), 8 deletions(-) diff --git a/cmd/fluxctl/args.go b/cmd/fluxctl/args.go index b1fc7916b..0ba0b1374 100644 --- a/cmd/fluxctl/args.go +++ b/cmd/fluxctl/args.go @@ -7,6 +7,8 @@ import ( "os/exec" "strings" + "k8s.io/client-go/tools/clientcmd" + "github.com/spf13/cobra" "github.com/weaveworks/flux/update" @@ -49,3 +51,20 @@ func getUserGitConfigValue(arg string) string { res := out.String() return strings.Trim(res, "\x00") } + +var getKubeConfigContextNamespace = func(defaultNamespace string) string { + config, err := clientcmd.NewNonInteractiveDeferredLoadingClientConfig( + clientcmd.NewDefaultClientConfigLoadingRules(), + &clientcmd.ConfigOverrides{}, + ).RawConfig() + if err != nil { + return defaultNamespace + } + + cc := config.CurrentContext + if c, ok := config.Contexts[cc]; ok && c.Namespace != "" { + return c.Namespace + } + + return defaultNamespace +} diff --git a/cmd/fluxctl/automate_cmd.go b/cmd/fluxctl/automate_cmd.go index c53b28592..03a8843da 100644 --- a/cmd/fluxctl/automate_cmd.go +++ b/cmd/fluxctl/automate_cmd.go @@ -32,7 +32,7 @@ func (opts *workloadAutomateOpts) Command() *cobra.Command { } AddOutputFlags(cmd, &opts.outputOpts) AddCauseFlags(cmd, &opts.cause) - cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", "default", "Workload namespace") + cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace("default"), "Workload namespace") cmd.Flags().StringVarP(&opts.workload, "workload", "w", "", "Workload to automate") // Deprecated diff --git a/cmd/fluxctl/deautomate_cmd.go b/cmd/fluxctl/deautomate_cmd.go index c006ffdbd..8c5a3a242 100644 --- a/cmd/fluxctl/deautomate_cmd.go +++ b/cmd/fluxctl/deautomate_cmd.go @@ -32,7 +32,7 @@ func (opts *workloadDeautomateOpts) Command() *cobra.Command { } AddOutputFlags(cmd, &opts.outputOpts) AddCauseFlags(cmd, &opts.cause) - cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", "default", "Workload namespace") + cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace("default"), "Workload namespace") cmd.Flags().StringVarP(&opts.workload, "workload", "w", "", "Workload to deautomate") // Deprecated diff --git a/cmd/fluxctl/list_images_cmd.go b/cmd/fluxctl/list_images_cmd.go index 07b5ed9d2..148f3d2a2 100644 --- a/cmd/fluxctl/list_images_cmd.go +++ b/cmd/fluxctl/list_images_cmd.go @@ -36,7 +36,7 @@ func (opts *imageListOpts) Command() *cobra.Command { Example: makeExample("fluxctl list-images --namespace default --workload=deployment/foo"), RunE: opts.RunE, } - cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", "", "Namespace") + cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace(""), "Namespace") cmd.Flags().StringVarP(&opts.workload, "workload", "w", "", "Show images for this workload") cmd.Flags().IntVarP(&opts.limit, "limit", "l", 10, "Number of images to show (0 for all)") diff --git a/cmd/fluxctl/list_workloads_cmd.go b/cmd/fluxctl/list_workloads_cmd.go index 715439531..3516ceaf0 100644 --- a/cmd/fluxctl/list_workloads_cmd.go +++ b/cmd/fluxctl/list_workloads_cmd.go @@ -30,7 +30,7 @@ func (opts *workloadListOpts) Command() *cobra.Command { Example: makeExample("fluxctl list-workloads"), RunE: opts.RunE, } - cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", "default", "Confine query to namespace") + cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace("default"), "Confine query to namespace") cmd.Flags().BoolVarP(&opts.allNamespaces, "all-namespaces", "a", false, "Query across all namespaces") return cmd } diff --git a/cmd/fluxctl/lock_cmd.go b/cmd/fluxctl/lock_cmd.go index d968e4026..e7015269c 100644 --- a/cmd/fluxctl/lock_cmd.go +++ b/cmd/fluxctl/lock_cmd.go @@ -32,7 +32,7 @@ func (opts *workloadLockOpts) Command() *cobra.Command { } AddOutputFlags(cmd, &opts.outputOpts) AddCauseFlags(cmd, &opts.cause) - cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", "default", "Controller namespace") + cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace("default"), "Controller namespace") cmd.Flags().StringVarP(&opts.workload, "workload", "w", "", "Workload to lock") // Deprecated diff --git a/cmd/fluxctl/policy_cmd.go b/cmd/fluxctl/policy_cmd.go index 64d6f5f6f..7d15a5292 100644 --- a/cmd/fluxctl/policy_cmd.go +++ b/cmd/fluxctl/policy_cmd.go @@ -60,7 +60,7 @@ containers which aren't explicitly named. AddOutputFlags(cmd, &opts.outputOpts) AddCauseFlags(cmd, &opts.cause) flags := cmd.Flags() - flags.StringVarP(&opts.namespace, "namespace", "n", "default", "Workload namespace") + flags.StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace("default"), "Workload namespace") flags.StringVarP(&opts.workload, "workload", "w", "", "Workload to modify") flags.StringVar(&opts.tagAll, "tag-all", "", "Tag filter pattern to apply to all containers") flags.StringSliceVar(&opts.tags, "tag", nil, "Tag filter container/pattern pairs") diff --git a/cmd/fluxctl/release_cmd.go b/cmd/fluxctl/release_cmd.go index 93d8f8916..80e0576a1 100644 --- a/cmd/fluxctl/release_cmd.go +++ b/cmd/fluxctl/release_cmd.go @@ -53,7 +53,7 @@ func (opts *workloadReleaseOpts) Command() *cobra.Command { AddOutputFlags(cmd, &opts.outputOpts) AddCauseFlags(cmd, &opts.cause) - cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", "default", "Workload namespace") + cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace("default"), "Workload namespace") // Note: we cannot define a shorthand for --workload since it clashes with the shorthand of --watch cmd.Flags().StringSliceVarP(&opts.workloads, "workload", "", []string{}, "List of workloads to release :/") cmd.Flags().BoolVar(&opts.allWorkloads, "all", false, "Release all workloads") diff --git a/cmd/fluxctl/unlock_cmd.go b/cmd/fluxctl/unlock_cmd.go index add44c2d7..1932b919c 100644 --- a/cmd/fluxctl/unlock_cmd.go +++ b/cmd/fluxctl/unlock_cmd.go @@ -32,7 +32,7 @@ func (opts *workloadUnlockOpts) Command() *cobra.Command { } AddOutputFlags(cmd, &opts.outputOpts) AddCauseFlags(cmd, &opts.cause) - cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", "default", "Controller namespace") + cmd.Flags().StringVarP(&opts.namespace, "namespace", "n", getKubeConfigContextNamespace("default"), "Controller namespace") cmd.Flags().StringVarP(&opts.workload, "workload", "w", "", "Controller to unlock") // Deprecated From f49cde3274314a992f1047b8ca6d9bbdfb9f48b6 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Mon, 15 Jul 2019 16:30:06 +0100 Subject: [PATCH 030/158] Update contact info in README This updates _most_ contact info and other links to point to the new repo or new Slack channel. --- README.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 12dff374b..fea50d583 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Flux -> **Notice** Flux is moving house! Please see https://github.com/weaveworks/flux/wiki/MoveToFluxCD +> **Notice** Flux has moved house! Please see https://github.com/weaveworks/flux/wiki/MoveToFluxCD We believe in GitOps: @@ -99,7 +99,7 @@ Get started by browsing through the documentation below: - Background about Flux - [Introduction to Flux](/site/introduction.md) - - [FAQ](/site/faq.md) and [frequently encountered issues](https://github.com/weaveworks/flux/labels/FAQ) + - [FAQ](/site/faq.md) and [frequently encountered issues](https://github.com/fluxcd/flux/labels/FAQ) - [How it works](/site/how-it-works.md) - [Considerations regarding installing Flux](/site/installing.md) - [Flux <-> Helm integration](/site/helm-integration.md) @@ -111,7 +111,7 @@ Get started by browsing through the documentation below: - [Using fluxctl to control Flux](/site/fluxctl.md) - [Helm Operator](/site/helm-operator.md) - [Troubleshooting](/site/troubleshooting.md) - - [Frequently encountered issues](https://github.com/weaveworks/flux/labels/FAQ) + - [Frequently encountered issues](https://github.com/fluxcd/flux/labels/FAQ) - [Upgrading to Flux v1](/site/upgrading-to-1.0.md) ### Integrations @@ -145,14 +145,12 @@ be interested in the following: If you have any questions about Flux and continuous delivery: -- Read [the Flux docs](https://github.com/weaveworks/flux/tree/master/site). -- Invite yourself to the Weave community - slack and ask a question on the [#flux](https://weave-community.slack.com/messages/flux/) +- Read [the Flux docs](https://github.com/fluxcd/flux/tree/master/site). +- Invite yourself to the CNCF community + slack and ask a question on the [#flux](https://cloud-native.slack.com/messages/flux/) channel. - To be part of the conversation about Flux's development, join the [flux-dev mailing list](https://groups.google.com/forum/#!forum/flux-dev). -- Join the [Weave User Group](https://www.meetup.com/pro/Weave/) and get - invited to online talks, hands-on training and meetups in your area. -- [File an issue.](https://github.com/weaveworks/flux/issues/new) +- [File an issue.](https://github.com/fluxcd/flux/issues/new) Your feedback is always welcome! From ed6b8ee3e2192298c8d848e2d287d766ba47c6e5 Mon Sep 17 00:00:00 2001 From: Daniel Holbach Date: Tue, 16 Jul 2019 09:39:57 +0200 Subject: [PATCH 031/158] Update Slack address --- CONTRIBUTING.md | 4 ++-- site/upgrading-to-1.0.md | 6 ------ 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f9b02fb39..9137af8b7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -35,8 +35,8 @@ contribution. No action from you is required, but it's a good idea to see the ## Communications The project uses Slack: To join the conversation, simply join the -[Weave community](https://slack.weave.works/) Slack workspace and use the -[#flux](https://weave-community.slack.com/messages/flux/) channel. +[CNCF](https://slack.cncf.io/) Slack workspace and use the +[#flux](https://cloud-native.slack.com/messages/flux/) channel. The Flux developers use a mailing list to discuss development as well. Simply subscribe to [flux-dev on Google diff --git a/site/upgrading-to-1.0.md b/site/upgrading-to-1.0.md index bd23949e0..f01f2ba97 100644 --- a/site/upgrading-to-1.0.md +++ b/site/upgrading-to-1.0.md @@ -193,9 +193,3 @@ create those as resources. If that’s the case, you will need to remove the manifests from git before running Flux v1. - -### Something else went wrong - -You can reach Weaveworks developers in our community Slack -- -https://weaveworks.github.io/community-slack/ -- where we will be able -to help. From 781b60555d63e2962dd2c900297b2006951dea6c Mon Sep 17 00:00:00 2001 From: Daniel Holbach Date: Tue, 16 Jul 2019 10:50:08 +0200 Subject: [PATCH 032/158] update meeting doc URL --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9137af8b7..ab45719fa 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -42,7 +42,7 @@ The Flux developers use a mailing list to discuss development as well. Simply subscribe to [flux-dev on Google Groups](https://groups.google.com/forum/#!forum/flux-dev) to join the conversation (this will also add an invitation to your Google calendar -for our [Flux meeting](https://github.com/weaveworks/flux/wiki/Meeting)). +for our [Flux meeting](https://docs.google.com/document/d/1l_M0om0qUEN_NNiGgpqJ2tvsF2iioHkaARDeh6b70B0/edit#)). ## Getting Started From 1e8daf5c7f7b28d3378798b31241f15730a6cb91 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Tue, 16 Jul 2019 12:46:50 +0100 Subject: [PATCH 033/158] Change CoC contact --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index fea50d583..641368249 100644 --- a/README.md +++ b/README.md @@ -129,10 +129,12 @@ a few popular ones you might want to check out: We welcome all kinds of contributions to Flux, be it code, issues you found, documentation, external tools, help and support or anything else really. +The Flux project adheres to the [CNCF Code of +Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md). + Instances of abusive, harassing, or otherwise unacceptable behavior -may be reported by contacting a *Flux* project maintainer, or Alexis -Richardson ``. Please refer to our [code of -conduct](CODE_OF_CONDUCT.md) as well. +may be reported by contacting a *Flux* project maintainer, or the CNCF +mediator, Mishi Choudhary . To familiarise yourself with the project and how things work, you might be interested in the following: From 0d89de5a80b5d25d630926e01eeeebe2e6a35ec6 Mon Sep 17 00:00:00 2001 From: Michael Bridgen Date: Wed, 17 Jul 2019 14:25:17 +0100 Subject: [PATCH 034/158] Let people supply Tiller hostname/IP to chart It's possible to tell the Helm operator to use a hostname or IP address and port, but this isn't exposed in the chart. This commit adds chart parameters for the Tiller host and IP. In theory this would let you point at a Tiller listening on localhost -- but since the chart doesn't also include Tiller in the pod, it's not quite enough. --- chart/flux/templates/helm-operator-deployment.yaml | 5 +++++ chart/flux/values.yaml | 8 ++++++++ 2 files changed, 13 insertions(+) diff --git a/chart/flux/templates/helm-operator-deployment.yaml b/chart/flux/templates/helm-operator-deployment.yaml index cf949360b..37282e8b1 100644 --- a/chart/flux/templates/helm-operator-deployment.yaml +++ b/chart/flux/templates/helm-operator-deployment.yaml @@ -134,7 +134,12 @@ spec: {{- else if .Values.helmOperator.allowNamespace }} - --allow-namespace={{ .Values.helmOperator.allowNamespace }} {{- end }} + {{- if .Values.helmOperator.tillerHost }} + - --tiller-ip={{ .Values.helmOperator.tillerHost }} + - --tiller-port={{ .Values.helmOperator.tillerPort }} + {{- else }} - --tiller-namespace={{ .Values.helmOperator.tillerNamespace }} + {{- end }} {{- if .Values.helmOperator.tls.enable }} - --tiller-tls-enable={{ .Values.helmOperator.tls.enable }} - --tiller-tls-key-path=/etc/fluxd/helm/{{ .Values.helmOperator.tls.keyFile }} diff --git a/chart/flux/values.yaml b/chart/flux/values.yaml index 9813663f3..3e1e5744c 100644 --- a/chart/flux/values.yaml +++ b/chart/flux/values.yaml @@ -35,7 +35,15 @@ helmOperator: chartsSyncInterval: "3m" # (Experimental) amount of workers processing releases workers: + # Tiller settings + + # If a hostname or IP is given here, that will be combined with the + # tillerPort and used for connecting to Tiller. Otherwise, the + # cluster-ip of the `tiller-deploy` service in .tillerNamespace is + # looked up. + tillerHost: + tillerPort: 44134 tillerNamespace: kube-system tls: secretName: "helm-client-certs" From b84b34548327f895ecd6f0abb6752f072b07d013 Mon Sep 17 00:00:00 2001 From: Jose Bovet Derpich Date: Wed, 17 Jul 2019 14:51:05 -0400 Subject: [PATCH 035/158] Added Walmart to production users --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 641368249..3c2c3662f 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,7 @@ Its major features are: - [Troii](https://troii.com/) - [Under Armour](https://www.underarmour.com) - [VSHN](https://vshn.ch) +- [Walmart Chile](https://www.walmartchile.cl) - [Weave Cloud](https://cloud.weave.works) If you too are using Flux in production; please submit a PR to add your organization to the list! From f6d7eeac2dc2f226e04eb81932ca50937f3095e2 Mon Sep 17 00:00:00 2001 From: Calle Pettersson Date: Thu, 18 Jul 2019 12:15:17 +0200 Subject: [PATCH 036/158] Fix file names --- site/fluxyaml-config-files.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/site/fluxyaml-config-files.md b/site/fluxyaml-config-files.md index c836730d4..0587f87d2 100644 --- a/site/fluxyaml-config-files.md +++ b/site/fluxyaml-config-files.md @@ -144,8 +144,8 @@ in your own Flux-based image or, if the tools are popular enough, Flux maintaine (please create an issue). In the future (once [Ephemeral containers](https://github.com/kubernetes/kubernetes/pull/59416) are available), you will be able to specify an container image for each command. -The working directory (also known as CWD) of the `command`s executed from a `.fluxctl.yaml` file will be set to the -target path (`--git-path` entry) used when finding that `.fluxctl.yaml` file. +The working directory (also known as CWD) of the `command`s executed from a `.flux.yaml` file will be set to the +target path (`--git-path` entry) used when finding that `.flux.yaml` file. For example, when using flux with `--git-path=staging` on a git repository with this structure: From 9b9484b65e385bf0ccb06536671f5a42c5c993ce Mon Sep 17 00:00:00 2001 From: Calle Pettersson Date: Thu, 18 Jul 2019 12:18:43 +0200 Subject: [PATCH 037/158] weaveworks -> fluxcd --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 09823d67f..42c913644 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,7 +1,7 @@