Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New upgrade process in mgmt-lambda #163

Merged
merged 26 commits into from
Dec 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ module.exports = {
testEnvironment: 'node',
testRegex: '/test/.+test.tsx?$',
passWithNoTests: true,
collectCoverageFrom: ['./proxy/**/**.ts', '!**/handlers/**.ts', '!**/model/**.ts', '!./proxy/app.ts', '!**/index.ts'],
collectCoverageFrom: [
'./proxy/**/**.ts',
'!**/model/**.ts',
'!./proxy/app.ts',
'!**/index.ts',
'./mgmt-lambda/**/**.ts',
],
coverageReporters: ['lcov', 'json-summary', ['text', { file: 'coverage.txt', path: './' }]],
}
6 changes: 6 additions & 0 deletions mgmt-lambda/DefaultSettings.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
export const defaults = {
AWS_REGION: 'us-east-1',
LAMBDA_DISTRIBUTION_BUCKET: 'fingerprint-pro-cloudfront-integration-lambda-function',
LAMBDA_DISTRIBUTION_BUCKET_KEY: 'release/lambda_latest.zip',
LAMBDA_HANDLER_NAME: 'fingerprintjs-pro-cloudfront-lambda-function.handler',
}
213 changes: 55 additions & 158 deletions mgmt-lambda/app.ts
Original file line number Diff line number Diff line change
@@ -1,175 +1,72 @@
import {
CloudFrontClient,
CreateInvalidationCommand,
CreateInvalidationCommandInput,
GetDistributionConfigCommand,
GetDistributionConfigCommandOutput,
UpdateDistributionCommand,
UpdateDistributionCommandInput,
} from '@aws-sdk/client-cloudfront'
import {
CodePipelineClient,
CodePipelineClientConfig,
FailureType,
PutJobFailureResultCommand,
PutJobFailureResultCommandInput,
PutJobSuccessResultCommand,
PutJobSuccessResultCommandInput,
} from '@aws-sdk/client-codepipeline'
import {
LambdaClient,
ListVersionsByFunctionCommand,
ListVersionsByFunctionCommandInput,
ListVersionsByFunctionCommandOutput,
} from '@aws-sdk/client-lambda'
import { APIGatewayProxyEventV2WithRequestContext, APIGatewayEventRequestContextV2 } from 'aws-lambda'
import { SecretsManagerClient } from '@aws-sdk/client-secrets-manager'
import { getAuthSettings } from './auth'
import type { DeploymentSettings } from './model/DeploymentSettings'
import { handleNoAthentication, handleWrongConfiguration, handleNotFound } from './handlers/errorHandlers'
import { defaults } from './DefaultSettings'
import { handleStatus } from './handlers/statusHandler'
import { handleUpdate } from './handlers/updateHandler'
import { LambdaClient } from '@aws-sdk/client-lambda'
import { CloudFrontClient } from '@aws-sdk/client-cloudfront'

export async function handler(event: APIGatewayProxyEventV2WithRequestContext<APIGatewayEventRequestContextV2>) {
const secretManagerClient = new SecretsManagerClient({ region: defaults.AWS_REGION })

const REGION = 'us-east-1'

export async function handler(event: any, ctx: any) {
console.info(JSON.stringify(event))

const job = event['CodePipeline.job']
if (!job) {
console.error('No job found')
return
}

const userInput = JSON.parse(job.data.actionConfiguration.configuration.UserParameters)
const lambdaFunctionName = userInput.LAMBDA_NAME
const cloudFrontDistrId = userInput.CF_DISTR_ID

console.info(`Going to upgrade Fingerprint Pro function association at CloudFront distbution.`)
console.info(`Lambda function: ${lambdaFunctionName}. CloudFront ID: ${cloudFrontDistrId}`)

const latestFunctionArn = await getLambdaLatestVersionArn(lambdaFunctionName)
if (!latestFunctionArn) {
return publishJobFailure(ctx, job, 'No lambda versions')
}

if (latestFunctionArn.length === 1) {
console.info('No updates yet')
return publishJobSuccess(ctx, job)
}

const cloudFrontClient = new CloudFrontClient({ region: REGION })

const configParams = {
Id: cloudFrontDistrId,
}
const getConfigCommand = new GetDistributionConfigCommand(configParams)
const cfConfig: GetDistributionConfigCommandOutput = await cloudFrontClient.send(getConfigCommand)

if (!cfConfig.ETag || !cfConfig.DistributionConfig) {
return publishJobFailure(ctx, job, 'CloudFront distribution not found')
}

const cacheBehaviors = cfConfig.DistributionConfig.CacheBehaviors
const fpCbs = cacheBehaviors?.Items?.filter((it) => it.TargetOriginId === 'fpcdn.io')
if (!fpCbs || fpCbs?.length === 0) {
return publishJobFailure(ctx, job, 'Cache behavior not found')
}
const cacheBehavior = fpCbs[0]
const lambdas = cacheBehavior.LambdaFunctionAssociations?.Items?.filter(
(it) => it && it.EventType === 'origin-request' && it.LambdaFunctionARN?.includes(lambdaFunctionName),
)
if (!lambdas || lambdas?.length === 0) {
return publishJobFailure(ctx, job, 'Lambda function association not found')
}
const lambda = lambdas[0]
lambda.LambdaFunctionARN = latestFunctionArn

const updateParams: UpdateDistributionCommandInput = {
DistributionConfig: cfConfig.DistributionConfig,
Id: cloudFrontDistrId,
IfMatch: cfConfig.ETag,
try {
const authSettings = await getAuthSettings(secretManagerClient)
const authorization = event.headers['authorization']
if (authorization !== authSettings.token) {
return handleNoAthentication()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will authorization header be directly the token, no scheme? This header needs a scheme by standard, such as Basic or something, no?

Copy link
Contributor Author

@sshelomentsev sshelomentsev Dec 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, I supposed that we have a single option for authorization header so I didn't add any scheme.
Basic is supposed to be base64 encoded username+password.
We can introduce some specific scheme name, let's say token or mgmt-token.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what will we do about this in this PR?

}
} catch (error) {
return handleWrongConfiguration(error)
}

const updateConfigCommand = new UpdateDistributionCommand(updateParams)
const updateCFResult = await cloudFrontClient.send(updateConfigCommand)
console.info(`CloudFront update has finished, ${JSON.stringify(updateCFResult)}`)

console.info('Going to invalidate routes for upgraded cache behavior')
if (!cacheBehavior.PathPattern) {
return publishJobFailure(ctx, job, 'Path pattern is not defined')
let deploymentSettings: DeploymentSettings
try {
deploymentSettings = loadDeploymentSettings()
} catch (error) {
return handleWrongConfiguration(error)
}

let pathPattern = cacheBehavior.PathPattern
if (!pathPattern.startsWith('/')) {
pathPattern = '/' + pathPattern
}
const path = event.rawPath
const method = event.requestContext.http.method
const lambdaClient = new LambdaClient({ region: defaults.AWS_REGION })
const cloudFrontClient = new CloudFrontClient({ region: defaults.AWS_REGION })

const invalidationParams: CreateInvalidationCommandInput = {
DistributionId: cloudFrontDistrId,
InvalidationBatch: {
Paths: {
Quantity: 1,
Items: [pathPattern],
},
CallerReference: 'fingerprint-pro-management-lambda-function',
},
if (path.startsWith('/update') && method === 'POST') {
return handleUpdate(lambdaClient, cloudFrontClient, deploymentSettings)
} else if (path.startsWith('/status') && method === 'GET') {
return handleStatus(lambdaClient, deploymentSettings)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can remove else here to make to more readable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you do it?

} else {
return handleNotFound()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can remove else here to make to more readable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you do it?

}
const invalidationCommand = new CreateInvalidationCommand(invalidationParams)
const invalidationResult = await cloudFrontClient.send(invalidationCommand)
console.info(`Invalidation has finished, ${JSON.stringify(invalidationResult)}`)

await publishJobSuccess(ctx, job)
}

async function getLambdaLatestVersionArn(functionName: string): Promise<string | undefined> {
const client = new LambdaClient({ region: REGION })
const params: ListVersionsByFunctionCommandInput = {
FunctionName: functionName,
function loadDeploymentSettings(): DeploymentSettings {
const missedVariables = []
const cfDistributionId = process.env.CFDistributionId || ''
if (cfDistributionId === '') {
missedVariables.push('CFDistributionId')
}
const command = new ListVersionsByFunctionCommand(params)
const result: ListVersionsByFunctionCommandOutput = await client.send(command)
if (!result.Versions || result.Versions?.length === 0) {
return Promise.resolve(undefined)
const lambdaFunctionName = process.env.LambdaFunctionName || ''
if (lambdaFunctionName === '') {
missedVariables.push('LambdaFunctionName')
}

const latest = result.Versions.filter((it) => it.Version && Number.isFinite(Number.parseInt(it.Version))).sort(
(a, b) => Number.parseInt(b.Version!!) - Number.parseInt(a.Version!!),
)[0]
return Promise.resolve(latest.FunctionArn)
}

function getCodePipelineClient(): CodePipelineClient {
const config: CodePipelineClientConfig = {
region: REGION,
defaultsMode: 'standard',
const lambdaFunctionArn = process.env.LambdaFunctionArn || ''
if (lambdaFunctionArn === '') {
missedVariables.push('LambdaFunctionArn')
}

return new CodePipelineClient(config)
}

async function publishJobSuccess(ctx: any, job: any) {
const params: PutJobSuccessResultCommandInput = {
jobId: job.id,
}
try {
const command = new PutJobSuccessResultCommand(params)
const result = await getCodePipelineClient().send(command)
console.info(`Job successfully finished with ${JSON.stringify(result)}`)
ctx.succeed()
} catch (err) {
ctx.fail(err)
if (missedVariables.length > 0) {
const vars = missedVariables.join(', ')
throw new Error(`environment variables not found: ${vars}`)
}
}

async function publishJobFailure(ctx: any, job: any, message: string) {
console.info(`Publishing failure status with message=${message}`)
const params: PutJobFailureResultCommandInput = {
jobId: job.id,
failureDetails: {
message: message,
type: FailureType.ConfigurationError,
},
}
try {
const command = new PutJobFailureResultCommand(params)
const result = await getCodePipelineClient().send(command)
console.info(`Job failed with ${JSON.stringify(result)}`)
ctx.fail(message)
} catch (err) {
ctx.fail(err)
const settings: DeploymentSettings = {
CFDistributionId: cfDistributionId,
LambdaFunctionArn: lambdaFunctionArn,
LambdaFunctionName: lambdaFunctionName,
}
return settings
}
27 changes: 27 additions & 0 deletions mgmt-lambda/auth.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import type { AuthSettings } from './model/AuthSettings'
import { SecretsManagerClient, GetSecretValueCommand, GetSecretValueResponse } from '@aws-sdk/client-secrets-manager'

export async function getAuthSettings(secretManagerClient: SecretsManagerClient): Promise<AuthSettings> {
const secretName = process.env.SettingsSecretName
if (!secretName) {
throw new Error('Unable to retrieve secret. Error: environment variable SettingsSecretName not found')
}

try {
const command = new GetSecretValueCommand({
SecretId: secretName,
})

const response: GetSecretValueResponse = await secretManagerClient.send(command)

if (response.SecretBinary) {
return JSON.parse(Buffer.from(response.SecretBinary).toString('utf8'))
}
if (response.SecretString) {
return JSON.parse(response.SecretString)
}
throw new Error('secret is empty')
} catch (error: any) {
throw new Error(`Unable to retrieve secret. ${error}`)
}
}
42 changes: 42 additions & 0 deletions mgmt-lambda/handlers/errorHandlers.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import { APIGatewayProxyResult } from 'aws-lambda'

export async function handleNoAthentication(): Promise<APIGatewayProxyResult> {
const body = {
status: 'Token is not specified or not valid',
}
return {
statusCode: 401,
body: JSON.stringify(body),
headers: {
'content-type': 'application/json',
},
}
}

export async function handleWrongConfiguration(error: any): Promise<APIGatewayProxyResult> {
const body = {
status:
'Wrong function configuration. Check environment variables for Lambda@Edge function and CloudFront Distribution id',
error: error.message || error,

Check warning on line 20 in mgmt-lambda/handlers/errorHandlers.ts

View workflow job for this annotation

GitHub Actions / Coverage annotations (🧪 jest-coverage-report-action)

🌿 Branch is not covered

Warning! Not covered branch
}
return {
statusCode: 500,
body: JSON.stringify(body),
headers: {
'content-type': 'application/json',
},
}
}

export async function handleNotFound(): Promise<APIGatewayProxyResult> {
const body = {
status: 'Path not found',
}
return {
statusCode: 404,
body: JSON.stringify(body),
headers: {
'content-type': 'application/json',
},
}
}
19 changes: 19 additions & 0 deletions mgmt-lambda/handlers/statusHandler.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import { APIGatewayProxyResult } from 'aws-lambda'
import type { DeploymentSettings } from '../model/DeploymentSettings'
import { LambdaClient, GetFunctionCommand } from '@aws-sdk/client-lambda'

export async function handleStatus(
lambdaClient: LambdaClient,
settings: DeploymentSettings,
): Promise<APIGatewayProxyResult> {
const command = new GetFunctionCommand({ FunctionName: settings.LambdaFunctionName })
const functionResult = await lambdaClient.send(command)

return {
statusCode: 200,
body: JSON.stringify(functionResult),
headers: {
'content-type': 'application/json',
},
}
}
Loading
Loading