New upgrade process in mgmt-lambda #163

sshelomentsev · 2023-12-12T10:29:44Z

Updated dependencies for AWS to V3
Fixed tests according to V3 SDK and Node 20
Authentication for Lambda function calls
Endpoints for /status and /update
Added Lambda function deployment

github-actions · 2023-12-13T01:01:19Z

Coverage report

St.^❔	Category	Percentage	Covered / Total
🟢	Statements	94.74% (-4.28% 🔻)	486/513
🟢	Branches	88.97% (-7.23% 🔻)	121/136
🟢	Functions	92.59% (-7.41% 🔻)	100/108
🟢	Lines	94.82% (-4.1% 🔻)	458/483

Show new covered files 🐣

St.^❔	File	Statements	Branches	Functions	Lines
🟢	`...` / headers.ts	100%	100%	100%	100%
🟢	`...` / cookie.ts	100%	100%	100%	100%
🟢	`...` / cache-control.ts	100%	100%	100%	100%
🟢	`...` / selectors.ts	100%	100%	100%	100%
🟢	`...` / types.ts	100%	100%	100%	100%
🟢	`...` / request.ts	100%	100%	100%	100%
🟢	`...` / traffic.ts	100%	100%	100%	100%
🟢	`...` / routing.ts	100%	100%	100%	100%
🟢	`...` / customer-variables.ts	100%	100%	100%	100%
🟢	`...` / defaults.ts	100%	100%	100%	100%
🟢	proxy/logger.ts	100%	71.43%	100%	100%
🟢	`...` / header-customer-variables.ts	100%	100%	100%	100%
🟢	`...` / secrets-manager-variables.ts	88%	100%	100%	87.5%
🟢	`...` / retrieve-secret.ts	100%	100%	100%	100%
🟢	`...` / buffer.ts	100%	100%	100%	100%
🟢	`...` / validate-secret.ts	100%	100%	100%	100%
🟢	`...` / handleAgentDowloading.ts	96%	66.67%	100%	95.83%
🟡	`...` / handleResult.ts	76.19%	100%	76.92%	77.5%
🟢	`...` / handleStatus.ts	100%	100%	100%	100%
🟢	`...` / maybe-obfuscate-variable.ts	100%	100%	100%	100%
🟡	proxy/test/aws.ts	77.78%	100%	50%	71.43%
🟢	`...` / updateHandler.ts	93.65%	82.35%	100%	93.55%
🟢	`...` / DefaultSettings.ts	100%	100%	100%	100%
🟢	mgmt-lambda/app.ts	97.67%	89.47%	100%	97.67%
🟢	mgmt-lambda/auth.ts	100%	100%	100%	100%
🟢	`...` / errorHandlers.ts	100%	50%	100%	100%
🟢	`...` / statusHandler.ts	100%	100%	100%	100%
🟢	`...` / in-memory-customer-variables.ts	100%	100%	100%	100%
🔴	`...` / is-blob.ts	0%	0%	0%	0%
🔴	`...` / normalize-secret.ts	0%	100%	0%	0%

Test suite run success

102 tests passing in 18 suites.

Report generated by 🧪jest coverage report action from d816e8f

Show full coverage report

St	File	% Stmts	% Branch	% Funcs	% Lines	Uncovered Line #s
🟢	All files	94.73	88.97	92.59	94.82
🟢	mgmt-lambda	98.27	90.9	100	98.27
🟢	DefaultSettings.ts	100	100	100	100
🟢	app.ts	97.67	89.47	100	97.67	38
🟢	auth.ts	100	100	100	100
🟢	mgmt-lambda/handlers	94.8	78.94	100	94.73
🟢	errorHandlers.ts	100	50	100	100	20
🟢	statusHandler.ts	100	100	100	100
🟢	updateHandler.ts	93.65	82.35	100	93.54	43,60,90,127
🟢	proxy	100	71.42	100	100
🟢	logger.ts	100	71.42	100	100	10,17
🟢	proxy/handlers	87.35	85.71	89.65	87.95
🟢	handleAgentDowloading.ts	96	66.66	100	95.83	26
🟡	handleResult.ts	76.19	100	76.92	77.5	88-103
🟢	handleStatus.ts	100	100	100	100
🟡	proxy/test	77.77	100	50	71.42
🟡	aws.ts	77.77	100	50	71.42	4-5
🟢	proxy/test/utils/customer-variables	100	100	100	100
🟢	in-memory-customer-variables.ts	100	100	100	100
🟢	proxy/utils	98.49	88.57	96.29	98.36
🟢	buffer.ts	100	100	100	100
🟢	cache-control.ts	100	100	100	100
🟢	cookie.ts	100	100	100	100
🟢	headers.ts	100	100	100	100
🔴	is-blob.ts	0	0	0	0	6-7
🟢	request.ts	100	100	100	100
🟢	routing.ts	100	100	100	100
🟢	traffic.ts	100	100	100	100
🟢	proxy/utils/customer-variables	100	100	100	100
🟢	customer-variables.ts	100	100	100	100
🟢	defaults.ts	100	100	100	100
🟢	header-customer-variables.ts	100	100	100	100
🟢	maybe-obfuscate-variable.ts	100	100	100	100
🟢	selectors.ts	100	100	100	100
🟢	types.ts	100	100	100	100
🟢	proxy/utils/customer-variables/secrets-manager	89.7	100	84.61	90.9
🔴	normalize-secret.ts	0	100	0	0	1-4
🟢	retrieve-secret.ts	100	100	100	100
🟢	secrets-manager-variables.ts	88	100	100	87.5	33,55-60
🟢	validate-secret.ts	100	100	100	100

jest.config.js

necipallef · 2023-12-19T08:16:32Z

mgmt-lambda/app.ts

-    },
+  if (path.startsWith('/update') && method === 'POST') {
+    return handleUpdate(lambdaClient, cloudFrontClient, deploymentSettings)
+  } else if (path.startsWith('/status') && method === 'GET') {


we can remove else here to make to more readable

could you do it?

necipallef · 2023-12-19T08:16:37Z

mgmt-lambda/app.ts

+    return handleUpdate(lambdaClient, cloudFrontClient, deploymentSettings)
+  } else if (path.startsWith('/status') && method === 'GET') {
+    return handleStatus(lambdaClient, deploymentSettings)
+  } else {


we can remove else here to make to more readable

could you do it?

necipallef · 2023-12-19T08:17:35Z

mgmt-lambda/app.ts

+function loadDeploymentSettings(): DeploymentSettings {
+  const cfDistributionId = process.env.CFDistributionId
+  if (!cfDistributionId) {
+    throw new Error('No CloudFront distribution Id')


I'd suggest to have more clear error message, for instance load failed: env variable CFDistributionId not found.

necipallef · 2023-12-19T08:18:17Z

mgmt-lambda/app.ts

-    return Promise.resolve(undefined)
+  const lambdaFunctionName = process.env.LambdaFunctionArn
+  if (!lambdaFunctionName) {
+    throw new Error('No lambda function name')


I'd suggest to have more clear error message, for instance load failed: env variable LambdaFunctionArn not found.

This seems to be duplicated. Did you mean something like process.env.LambdaFunctionName?

This part is reworked and test added
Also, I will have validation for values given in env vars as a part of POST /update test

necipallef · 2023-12-19T08:19:36Z

mgmt-lambda/app.ts

-    defaultsMode: 'standard',
+  const lambdaFunctionArn = process.env.LambdaFunctionArn
+  if (!lambdaFunctionArn) {
+    throw new Error('No lambda function ARN')


I'd suggest to have more clear error message, for instance load failed: env variable LambdaFunctionArn not found.

necipallef · 2023-12-19T08:20:42Z

mgmt-lambda/auth.ts

+export async function getAuthSettings(secretManagerClient: SecretsManagerClient): Promise<AuthSettings> {
+  const secretName = process.env.SettingsSecretName
+  if (!secretName) {
+    throw new Error('Unable to retrieve secret. Error: unable to get secret name')


I'd suggest to have more clear error message, for instance getAuthSettings failed: env variable SettingsSecretName not found.

necipallef · 2023-12-19T08:21:12Z

mgmt-lambda/auth.ts

+
+    if (response.SecretBinary) {
+      return JSON.parse(Buffer.from(response.SecretBinary).toString('utf8'))
+    } else if (response.SecretString) {


Let's remove else here to make code more readable

necipallef · 2023-12-19T08:21:18Z

mgmt-lambda/auth.ts

+      return JSON.parse(Buffer.from(response.SecretBinary).toString('utf8'))
+    } else if (response.SecretString) {
+      return JSON.parse(response.SecretString)
+    } else {


Let's remove else here to make code more readable

necipallef · 2023-12-19T08:24:17Z

mgmt-lambda/app.ts

+  try {
+    const authSettings = await getAuthSettings(secretManagerClient)
+    const authorization = event.headers['authorization']
+    if (authorization !== authSettings.token) {


Will authorization header be directly the token, no scheme? This header needs a scheme by standard, such as Basic or something, no?

In general, I supposed that we have a single option for authorization header so I didn't add any scheme.
Basic is supposed to be base64 encoded username+password.
We can introduce some specific scheme name, let's say token or mgmt-token.

So what will we do about this in this PR?

github-actions · 2023-12-19T22:53:04Z

This PR will create a minor release 🚀

1.5.0 (2023-12-19)

Features

add endpoints structure and error handlers (f5b3709)
add event and ctx types (6f72caf)
add lambda function update (2b2a2c4)
introduce deployment settings (850ca61)
pass AWS clients into handlers (d84dd9e)
reworked getting env, updated tests (2f7ceb6)
update lambda existence check (d816e8f)
update secrets manager to V3, retrieve secret in mgmt-lambda (64e1e39)
use AWS SDK v3 Client mock for testing (2eb947c)

Bug Fixes

remove aws-sdk v2 usage (9f76031)
remove CodePipeline client (f98c8d5)
remove CodePipeline part from mgmt code (01a3739)
set correct type for public URL events (baba28d)
update handleResult tests with comparing hrefs (0c53e8f)
update logging (3b6c713)

ilfa force-pushed the rc branch from 8e5d880 to 8f0aeff Compare December 13, 2023 11:50

sshelomentsev force-pushed the feature/lambda-endpoints branch from 8442eb5 to e220693 Compare December 14, 2023 12:57

sshelomentsev marked this pull request as ready for review December 14, 2023 13:09

sshelomentsev requested a review from TheUnderScorer as a code owner December 14, 2023 13:09

sshelomentsev requested a review from necipallef December 14, 2023 13:09

sshelomentsev force-pushed the feature/lambda-endpoints branch from e220693 to f991634 Compare December 14, 2023 20:26

ilfa force-pushed the rc branch 2 times, most recently from c35ccbd to 4139b8c Compare December 15, 2023 10:08

necipallef requested changes Dec 19, 2023

View reviewed changes

Sergey Shelomentsev added 20 commits December 19, 2023 18:31

fix: remove CodePipeline part from mgmt code

01a3739

fix: remove CodePipeline client

f98c8d5

feat: add event and ctx types

6f72caf

fix: set correct type for public URL events

baba28d

feat: update secrets manager to V3, retrieve secret in mgmt-lambda

64e1e39

feat: introduce deployment settings

850ca61

feat: add endpoints structure and error handlers

f5b3709

feat: add lambda function update

2b2a2c4

fix: remove aws-sdk v2 usage

9f76031

feat: use AWS SDK v3 Client mock for testing

2eb947c

fix: update logging

3b6c713

fix: update handleResult tests with comparing hrefs

0c53e8f

chore: remove unused test function

569cada

feat: pass AWS clients into handlers

d84dd9e

test: add tests for mgmt-lambda handlers

f1d2e97

test: add basic test for update handler

e8c90dd

test: add basic tests for lambda call

4d201e5

test: update app test

870e4bf

test: fix mgmt-lambda app test

700c2c6

ci: update coverage source

716a131

Sergey Shelomentsev added 5 commits December 19, 2023 18:31

chore: lint fix

3355aeb

test: add more test for mgmt-lambda handler

65b1798

chore: address review comments, fix tests

5b72c7b

feat: reworked getting env, updated tests

2f7ceb6

chore: else block

741dab9

sshelomentsev force-pushed the feature/lambda-endpoints branch from 24dcbe2 to 741dab9 Compare December 19, 2023 16:33

sshelomentsev changed the base branch from rc to feature/rollout-as-code December 19, 2023 16:33

feat: update lambda existence check

d816e8f

necipallef approved these changes Dec 20, 2023

View reviewed changes

sshelomentsev merged commit b33d50c into feature/rollout-as-code Dec 20, 2023
6 checks passed

sshelomentsev deleted the feature/lambda-endpoints branch December 20, 2023 09:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New upgrade process in mgmt-lambda #163

New upgrade process in mgmt-lambda #163

sshelomentsev commented Dec 12, 2023 •

edited

Loading

github-actions bot commented Dec 13, 2023 •

edited

Loading

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023

necipallef Dec 20, 2023

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023

necipallef Dec 20, 2023

necipallef Dec 19, 2023

necipallef Dec 19, 2023

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023

necipallef Dec 19, 2023

sshelomentsev Dec 19, 2023 •

edited

Loading

necipallef Dec 20, 2023

github-actions bot commented Dec 19, 2023

New upgrade process in mgmt-lambda #163

New upgrade process in mgmt-lambda #163

Conversation

sshelomentsev commented Dec 12, 2023 • edited Loading

github-actions bot commented Dec 13, 2023 • edited Loading

Coverage report

Test suite run success

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sshelomentsev Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Dec 19, 2023

This PR will create a minor release 🚀

1.5.0 (2023-12-19)

Features

Bug Fixes

sshelomentsev commented Dec 12, 2023 •

edited

Loading

github-actions bot commented Dec 13, 2023 •

edited

Loading

sshelomentsev Dec 19, 2023 •

edited

Loading