CRUD_Bucket_S3_AWS

CRUD model for object handling implemented with Systems Manager Parameter Store, S3 Bucket, Api-Gateway, Serverless-Framework, Lambda, NodeJs, aws-sdk-v3, among others others. AWS services are deployed locally. The project code and documentation (except technical docs) have been developed in English.

Functionality test playlist

Table of Contents 📜

View

Section 1) Description, configuration and technologies

1.0) Description 🔝

View

1.0.0) General Description

This app is divided into several functionalities/components. The first component or connection layer (/bucket) is the interaction with aws-sdk and the bucket. It is modularized in such a way that we have .js files for creating s3 clients, bucket reading, bucket writing, etc. Then for the application layer (/helpers) we have header validations, request body, date formats, authentication, etc. Next, the controller/view layer (/controllers) is defined by the CRUD operations possible in each of the defined lambdas.

1.0.1) Architecture and Operation Description

The image of the aws architecture used describes the general flow of the app. Any request to the bucket starts from a client (Postman, server, etc.).
Step 0 : This request is received by the api-gateway and will only be validated if the correct x-api-key is found within the headers of said request.
Steps 1A, 1B, etc. : All these steps correspond to an endpoint with its specific resource. For example, for uploadObject (1A) it is http://localhost:4000/dev/upload-object .... check these endpoints in endpoints section. Each lambda performs x-api-key and Bearer token verification, among others.
Steps 2 : The lambdas perform the corresponding ssm validations with the System Manager Parameter Store.. they validate tokens, values for the s3 bucket, etc.
Steps 3 : The lambdas perform the necessary requests and operations against the s3 bucket (reading, updating, deleting and inserting objects).
Clarifications : This operation is emulated within the same network and in a local environment with the corresponding serverless plugins.

1.1) Project Execution 🔝

View

Once a work environment has been created through an ide, we clone the project

git clone https://github.com/andresWeitzel/CRUD_Bucket_S3_AWS

We position ourselves on the project

cd 'projectName'

We install the latest LTS version of Nodejs(v18)
We install Serverless Framework globally if we have not done so already

npm install -g serverless

We check the installed version of Serverless

sls -v

We install all the necessary packages

npm i

The ssm variables used in the project are kept to simplify the configuration process of the project. It is recommended to add the corresponding file (serverless_ssm.yml) to the .gitignore.
The following script configured in the package.json of the project is in charge of
Starting serverless-offline (serverless-offline)

"scripts": {
"serverless-offline": "sls offline start",
"start": "npm run serverless-offline"
},

We run the app from the terminal.

npm start

If a message appears indicating that port 4000 is already in use, we can terminate all dependent processes and re-run the app

npx kill-port 4000
npm start

1.2) Setting up the project from scratch 🔝

View

We create a work environment through an IDE, after creating a folder we position ourselves on it

cd 'projectName'

We install the latest LTS version of Nodejs(v18)
We install Serverless Framework globally if we have not done so yet

npm install -g serverless

Check the installed Serverless version

sls -v

Initialize a serverless template

serverless create --template aws-nodejs

Initialize an npm project

npm init -y

Install local S3

npm install serverless-s3-local --save-dev

Install the s3 Client

npm install @aws-sdk/client-s3

Install serverless offline

npm i serverless-offline --save-dev

Install serverless ssm

npm i serverless-offline-ssm --save-dev

The ssm variables used in the project are kept to simplify the configuration process of the project. It is recommended to add the corresponding file (serverless_ssm.yml) to the .gitignore.
The following script configured in the package.json of the project is in charge of
Starting serverless-offline (serverless-offline)

"scripts": {
"serverless-offline": "sls offline start",
"start": "npm run serverless-offline"
},

We run the app from the terminal.

npm start

If any message appears indicating that the port 4000 is already in use, we can kill all dependent processes and re-run the app

npx kill-port 4000
npm start

1.3) Technologies 🔝

View

| Technologies | Version | Purpose |
| ------------- | ------------- | ------------- | | SDK | 4.3.2 | Automatic Module Injection for Lambdas | | Serverless Framework Core v3 | 3.23.0 | AWS Core Services | | Systems Manager Parameter Store (SSM) | 3.0 | Management of Environment Variables | | Amazon Api Gateway | 2.0 | API Manager, Authentication, Control and Processing | | Amazon S3 | 3.0 | Object Container | | NodeJS | 14.18.1 | JS Library | | VSC | 1.72.2 | IDE | | Postman | 10.11 | Http Client | | CMD | 10 | Command Prompt | | Git | 2.29.1 | Version Control |

| Plugin | Description |
| ------------- | ------------- | | Serverless Plugin | Libraries for Modular Definition | | serverless-offline | This serverless plugin emulates AWS λ and API Gateway on-premises | | serverless-offline-ssm | finds environment variables that match SSM parameters at build time and replaces them from a file | | serverless-s3-local | serverless plugin to run S3 clones locally

| Extension |
| ------------- | | Prettier - Code formatter | | YAML - Autoformatter .yml (alt+shift+f) |

Section 2) Endpoints and Examples.

2.0) Endpoints and Resources 🔝

View

2.1.0) Variables in Postman

| Variable | Initial value | Current value |
| ------------- | ------------- | ------------- | | base_url | http://localhost:4000 | http://localhost:4000 | | x-api-key | f98d8cd98h73s204e3456998ecl9427j | f98d8cd98h73s204e3456998ecl9427j | | bearer_token | Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva G4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c | Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva G4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c |

2.1.1) Upload an object to the s3 bucket

Request | code snippet

``postman curl --location 'http://localhost:4000/dev/upload-object'
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j'
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva G4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c'
--header 'Content-Type: application/json'
--data '{ "type":"image", "format":"png", "description":"5000 × 3061 png", "url":"https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846n eirVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a%2f %2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5 000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0" }'


#### Response

``postman
{
 "message": {
 "type": "image",
 "format": "png",
 "description": "5000 × 3061 png",
 "url": "https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846nei rVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a% 2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw= 5000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0",
 "uuid": 104851112
 }
}

2.1.2) Get an object from the bucket based on its uuuid

Request | code snippet

``postman curl --location 'http://localhost:4000/dev/get-object/103053674'
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j'
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI' --header 'Content-Type: application/json'
--data ''


#### Response

``postman
{
 "message": {
 "type": "image",
 "format": "jpg",
 "description": "1000 × 1261 png",
 "url": "https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846nei rVSebC0vAHaEi&mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.png&cdnurl=https%3a% 2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw= 5000&q=jpg+nodejs&simid=608055434302923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0",
 "uuid": 103053674
 }
}

2.1.3) Update an object in the s3 bucket

Request | code snippet

``postman curl --location --request PUT 'http://localhost:4000/dev/edit-object/104851112'
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j'
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva G4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c'
--header 'Content-Type: application/json'
--data '{ "type":"image", "format":"jpg", "description":"1200 × 1201 png", "url":"https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846neirVSebC0vAHaEi& mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJS.p ng&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3dCb%252 f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5000&q=jpg+nodejs&simid=608055434302923 247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0" }'


#### Response

``postman
{
 "message": {
 "type": "image",
 "format": "jpg",
 "description": "1200 × 1201 png",
 "url": "https://www.bing.com/images/search?view=detailV2&ccid=Tf4BFI68&id=D66EF5BFB7DA0A645A70240C32CB8664E8F8BF09&thid=OIP.Tf4BFI6846neirVSebC0vAHaEi& mediaurl=https%3a%2f%2flogos-download.com%2fwp-content%2fuploads%2f2016%2f09%2fNode_logo_NodeJ S.png&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.4dfe01148ebce3a9de8ab55279b0b4bc%3frik%3d Cb%252f46GSGyzIMJA%26pid%3dImgRaw%26r%3d0&exph=3061&expw=5000&q=jpg+nodejs&simid=608055434302 923247&FORM=IRPRST&ck=2FF3D39CAEF945F20B996CF6042F88A6&selectedIndex=1&ajaxhist=0&ajaxserp=0",
 "uuid": 104851112
 }
}

2.1.4) Delete an object from the bucket

Request | code snippet

``postman curl --location --request DELETE 'http://localhost:4000/dev/delete-object/104851112'
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva G4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c'
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j'
--header 'Content-Type: application/json'


#### Response

```postman
{
"message": "Removed object with uuid 104851112 successfully."
}

Name		Name	Last commit message	Last commit date
Latest commit History 68 Commits
.github/workflows		.github/workflows
doc/assets		doc/assets
src		src
translation		translation
.gitattributes		.gitattributes
.gitignore		.gitignore
.jshintrc		.jshintrc
.prettierrc		.prettierrc
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
serverless.yml		serverless.yml
serverless_ssm.yml		serverless_ssm.yml

License

andresWeitzel/CRUD_Amazon_S3_AWS

Folders and files

Latest commit

History

Repository files navigation

CRUD_Bucket_S3_AWS

Table of Contents 📜

Section 1) Description, Configuration, and Technologies

Section 2) Endpoints and Examples

Section 3) Functionality Testing and References

Section 1) Description, configuration and technologies

1.0) Description 🔝

1.0.0) General Description

1.0.1) Architecture and Operation Description

1.1) Project Execution 🔝

1.2) Setting up the project from scratch 🔝

1.3) Technologies 🔝

Section 2) Endpoints and Examples.

2.0) Endpoints and Resources 🔝

2.1.0) Variables in Postman

2.1.1) Upload an object to the s3 bucket

Request | code snippet

2.1.2) Get an object from the bucket based on its uuuid

Request | code snippet

2.1.3) Update an object in the s3 bucket

Request | code snippet

2.1.4) Delete an object from the bucket

Request | code snippet

Section 3) Functionality Testing and References.

3.0) Functionality Testing 🔝

Types of Operations | Watch

3.1) References 🔝

Bucket configuration

Tools

AWS-SDK

API Gateway

Bookstores

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 4

Packages 0

Contributors 2

Languages

Packages