Global Blue/Green deployments in multiple regions and accounts for ECS Fargate using AWS Codepipeline and Cloudformation
- Customer is looking for a deployment from a centralised account to multiple accounts and across regions for a Global application. The software development /code development happens only in the centralised account and workload / applications are spread across multiple accounts and across regions. The encryption and decryption must happen only with a single CMK Multi region key (covering Disaster Recovery scenario).
- As per security guidelines from customer, same CMK should be used across regions for encryption and decryption. CMK is region specific and it has to be maintained or created in 3 different regions for pipeline artifacts. Multi region CMK helps to retain the same key id across regions.
- Gitflow workflow branching model should be implemented with two branches (develop and main) for merging the codes from develop to main branch with a PR.
- This pattern covers all the above requirements and provides a sample IaC setup using AWS cloudformation stacks) to demonstrate this.
- Blue/Green deployment of microservices is implemented using AWS CodeDeploy in AWS Codepipeline.
- The regions considered here in the steps are as follows:
- Tools / CodeCommit Account - ap-south-1
- Workload / Test account 1 - ap-south-1
- Workload / Test account 2 - eu-central-1
- Workload / Test account 3 - us-east-1
- The lamdba function deployed from this stack will create a PR from develop to main branch. The PR merge to main branch will trigger the Codepipeline to orchestrate the CI/CD flow and deploy the stacks across accounts.
- Four AWS accounts to implement this pattern.
- Tools Account - Codepipeline is managed and Codecommit repository is maintained here.
- Three workload /test accounts to deploy the microservice workload.
- S3 bucket created specific to respective deployment region in each workload account. (Referred in below as S3BUCKETNAMETESTACCOUNT1, S3BUCKETNAMETESTACCOUNT2 and S3BUCKETNAMETESTACCOUNT3)
- Example - To create bucket in specific region with unique available bucket name (replace
xxxxwith some random numbers in below command):- In Test Account 1
aws s3 mb s3://ecs-codepipeline-
xxxx-ap-south-1 --region ap-south-1 - In Test Account 2
aws s3 mb s3://ecs-codepipeline-
xxxx-eu-central-1 --region eu-central-1 - In Test Account 3
aws s3 mb s3://ecs-codepipeline-
xxxx-us-east-1 --region us-east-1
- In Test Account 1
aws s3 mb s3://ecs-codepipeline-
- Example - To create bucket in specific region with unique available bucket name (replace
- Git - command line utility to work with codecommit repository.
- docker - To build and test docker images locally.
- cfn-lint and cfn-nag - To review the cloudformation stacks for any errors and security check.
The above discussed architecture is implemented in below steps using cloudformation templates.
- Create the infrastructure components in the architecture like VPC, Application Load balancer, ECS, ECR the cloudformation stacks in
Infradirectory. - Package and deploy the cloudformation stacks in all three work load accounts in the respective region.
- Ensure the values in multiple
exportsfrom the respective cloudformation stacks are not modified as they are referred in codepipeline, codedeploy stacks. - Push a sample image to ECR and scale ECS Service.
- Verify if the initial service deployment is accessible using Load balancer.
- Create codecommit repository in ToolsAccount. The templates are in
codedirectory. - Create S3 bucket for managing artifacts generated by codepipeline in all 3 regions.
- Setup Multi region CMK KMS Key as primary key and replica keys which will be used by Codepipeline.
- Setup CodeBuild Project in ToolsAccount to build and push images to ECR in all 3 regions.
- Setup CodeDeploy in Workload accounts in respective regions. Ensure the codedeploy is associated with ALB listener ARN created from Infra setup.
- Create codepipeline in Tools Account. Provide access for codepipeline, codebuild roles in KMS key policy and S3 bucket policy.
- Create a new index.html or modify the contents in index.html.
- Modify the input artifacts
taskdef.json,appspec.yamlwith relevant parameters like accountid - Push Dockerfile, buildspec.yaml to codecommit repository.
- Invoke pipeline and verify if in respective stages:
- Codecommit - Source code includes the latest commit.
- Codebuild - Builds and pushes image to ECR.
- Codedeploy - Deploys the latest image to ECS using latest taskdef.json and appspec.yaml.
- Access the load balancer using FQDN and verify if the latest changes are visible.
- Create a new feature branch from main branch.
- Push changes to feature branch and create a PR to main branch.
- Verify if lambda function approves the merge and invoke pipeline.
- Development: The software/code development occurs in a centralized account.
- Deployment Across Multiple Accounts and Regions: The deployment involves spreading workload/applications across multiple accounts and regions.
- Encryption and Decryption: The use of a single Customer Master Key (CMK) for encryption and decryption across regions, specifically for a Disaster Recovery scenario.
- CMK Management: The CMK is region-specific and needs to be maintained or created in three different regions for pipeline artifacts. A multi-region CMK is preferred for consistency in key identification across regions.
- Gitflow Workflow: Implementation of a Git workflow branching model with two branches (develop and main) for merging code changes from the develop branch to the main branch using pull requests (PR).
- Scale down ECS Service to 0. Clean up any images from ECR repository.
- Delete all objects from S3 artifact buckets.
- Delete the cloudformation stacks deployed in Tools account and respective workload accounts.
See CONTRIBUTING for more information.
See SECURITY for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.
