Refactor storage service to improve modularity and error handling

.env.example

@@ -4,4 +4,4 @@ DOMAIN=localhost
 PORT=3333
 AVAILABLE_BUCKETS=verifiable-credentials,private-verifiable-credentials,epcis-events
 GOOGLE_APPLICATION_CREDENTIALS=/tmp/service-account-file.json
-STORAGE_TYPE=local # local/gcp
+STORAGE_TYPE=local

.github/workflows/test_build.yml

@@ -24,16 +24,17 @@ jobs:
               run: yarn lint
 
             - name: Run tests
-              run: yarn test:ci
-
+              run: |
+                yarn test:ci
+
             - name: Coverage
               uses: artiomtr/jest-coverage-report-action@v2
               with:
-                  test-script: yarn jest
-                  github-token: ${{ secrets.GITHUB_TOKEN }}
-                  coverage-file: report.json
-                  base-coverage-file: report.json
-                  threshold: 80
+                package-manager: yarn
+                github-token: ${{ secrets.GITHUB_TOKEN }}
+                coverage-file: report.json
+                base-coverage-file: report.json
+                threshold: 80
 
             - name: Build
               run: yarn build

.gitignore

@@ -2,6 +2,8 @@
 
 .env
 
+report.json
+
 dist
 coverage
 node_modules

Dockerfile

@@ -1,5 +1,21 @@
 FROM node:18-alpine
 
+# Set the default values for the build arguments
+ARG API_VERSION=v1
+ARG PROTOCOL=http
+ARG DOMAIN=localhost
+ARG PORT=3333
+ARG AVAILABLE_BUCKETS=verifiable-credentials,private-verifiable-credentials,epcis-events
+ARG STORAGE_TYPE=local
+
+# Set the environment variables
+ENV API_VERSION=${API_VERSION}
+ENV PROTOCOL=${PROTOCOL}
+ENV DOMAIN=${DOMAIN}
+ENV PORT=${PORT}
+ENV AVAILABLE_BUCKETS=${AVAILABLE_BUCKETS}
+ENV STORAGE_TYPE=${STORAGE_TYPE}
+
 WORKDIR /app
 
 COPY package*.json ./

README.md

@@ -1,88 +1,187 @@
-# project-identity-resolver
-Reference implementation of UNTP link resolver and credential store
+# Storage Service
 
-* Will implement [ISO-18975](https://www.iso.org/standard/85540.html)
-* Will support any identifier type
-* Will add credential store capabilites to host DPPs etc.
+The storage service directory contains an Express REST API
+that provides endpoints to encrypt and store Verifiable Credentials and documents.
 
-Development work is expected to start in July 2024.
+## Overview
 
-## Storage Service
+The service offers the following functionality:
 
-This repository contains an Express REST API that provides an endpoint to encrypt and store Verifiable Credentials.
-
-The credentials endpoint provides the following functionality:
-
--   **Hash Computation**: It computes the hash of a given document, ensuring data integrity.
--   **Encryption**: The document is encrypted, adding a layer of security to publicly accessible documents.
--   **Storage**: The encrypted document is stored using the specified storage adapter.
--   **Data Retrieval**: Upon successful storage, the service returns three key pieces of information:
+-   **Hash Computation**:
+    Computes the SHA-256 hash of a given document to ensure data integrity.
+-   **Encryption**:
+    Encrypts the document using AES-256-GCM for enhanced security.
+-   **Storage**:
+    Stores the encrypted document using the specified storage adapter
+    (local file system or Google Cloud Storage).
+-   **Data Retrieval**:
+    Upon successful storage, the service returns:
     -   The hash of the original document.
-    -   A decryption key, essential for decrypting the encrypted document.
+    -   A decryption key for the encrypted document (if applicable).
     -   The URI of the stored encrypted document.
 
-### Prerequisites
+## Prerequisites
 
--   [npm](https://www.npmjs.com/) (>= 9.8.1)
--   [yarn](https://yarnpkg.com/) (>= 1.22.21)
+-   [Node.js](https://nodejs.org/) (v18.18.0)
+-   [Yarn](https://yarnpkg.com/) (>= 1.22.21)
 
-### Environment variables
+## Environment Variables
 
-An example env file `.env.example` can be found in the root directory. Duplicate and rename the file to `.env` and modify the variables as required.
+An example environment file `.env.example` is provided in the storage service directory.
+Copy and rename it to `.env`,
+then modify the variables as required.
+The default values should be sufficient for local development.
 
-### Usage
+## Usage
 
 ```bash
 # Install dependencies
 yarn install
 
-# Run the app
-yarn start
+# Run the app and watch for changes
+yarn dev
 
 # Build the app
 yarn build
 
+# Start the server once built
+yarn start
+
 # Run linter
 yarn lint
 
 # Run tests
 yarn test
 ```
 
+## Configuration
+
+Configure the storage service using the following environment variables:
+
+-   `STORAGE_TYPE`:
+    The type of storage to use (`local` or `gcp`).
+-   `LOCAL_DIRECTORY`:
+    The directory for local storage (default: `uploads` in the current directory).
+-   `GOOGLE_APPLICATION_CREDENTIALS`:
+    The path to the GCP service account file (if using GCP).
+
+## Storage Types
+
+### Local Storage
+
+For development purposes,
+use the local storage service,
+which stores files in the local file system.
+
+Example:
+
+```bash
+# Set the storage type to local
+export STORAGE_TYPE=local
+
+# Run the app
+yarn dev
+```
+
+### Google Cloud Storage
+
+For production environments,
+use Google Cloud Storage to store files in a GCP bucket.
+
+Example:
+
+```bash
+# Set the storage type to gcp
+export STORAGE_TYPE=gcp
+
+# Set the path to the GCP service account file
+export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-file.json
+
+# Build the app
+yarn build
+
+# Run the app
+yarn start
+```
+
+## Cryptography
+
+The cryptography service uses the following algorithms:
+
+-   **Hash Algorithm**:
+    SHA-256
+-   **Encryption Algorithm**:
+    AES-256-GCM
+
+## Endpoints
+
+### Store Credential
+
+-   **URL**: `/v1/credentials`
+-   **Method**: `POST`
+-   **Request Body**:
+
+    ```json
+    {
+        "bucket": "verifiable-credentials",
+        "data": {
+            "field1": "value1",
+            "field2": "value2"
+        },
+        "id": "123e4567-e89b-12d3-a456-426614174000" // optional
+    }
+    ```
+
+-   **Response**:
+
+    ```json
+    {
+        "uri": "http://localhost:3333/v1/verifiable-credentials/123e4567-e89b-12d3-a456-426614174000.json",
+        "hash": "computed-hash",
+        "key": "encryption-key"
+    }
+    ```
+
+### Store Document
+
+-   **URL**: `/v1/documents`
+-   **Method**: `POST`
+-   **Request Body**:
+
+    ```json
+    {
+        "bucket": "verifiable-credentials",
+        "data": {
+            "document": "content"
+        },
+        "id": "123e4567-e89b-12d3-a456-426614174000" // optional
+    }
+    ```
+
+-   **Response**:
+
+    ```json
+    {
+        "uri": "http://localhost:3333/v1/verifiable-credentials/123e4567-e89b-12d3-a456-426614174000.json",
+        "hash": "computed-hash"
+    }
+    ```
+
 ## Docker
 
+To run the storage service using Docker:
+
 ```bash
 # Build the image
 docker build -t storage-service:latest .
 
-# Local Storage - Start the container
-docker run -d --env-file .env storage-service:latest
+# Start the container using local storage
+docker run -d --env-file .env -p 3333:3333 \
+storage-service:latest
 
-# Cloud Storage - Start the container 
+# Start the container using Google Cloud Storage
 docker run -d --env-file .env -p 3333:3333 \
+-e STORAGE_TYPE=gcp \
 -v path/to/local/gcp/service-account-file.json:/tmp/service-account-file.json \
 storage-service:latest
 ```
-
-### Example
-
-#### Store Credential
-
-```bash
-# Request
-curl -X POST http://localhost:3333/v1/credentials -H "Content-Type: application/json" -d '{
-    "bucket": "verifiable-credentials",
-    "data": {
-        "field1": "value1",
-        "field2": "value2"
-    },
-    "filename": "myFile"
-}'
-
-# Response
-{
-    "uri": "https://verifiable-credentials.storage.googleapis.com/myFile.json",
-    "hash": "b9ef9745ec5ad95a030de384d6c8713f4e82f45ba9a4e694e0b4f785b3b4f8c61fe6cb5a388d31ee2fb919c00211f4eb55ef9a57f947bcda6545d5276ed0d2d5",
-    "key": "a1b4fa1a361801a9e2f9709c663eba3bea71a455bae08e4acf62f8d596ece669"
-}
-```

jest.config.js

@@ -1,6 +1,7 @@
 module.exports = {
     preset: 'ts-jest',
     testEnvironment: 'node',
+    setupFilesAfterEnv: ['./jest.setup.js'],
     collectCoverage: true,
     testPathIgnorePatterns: ['/node_modules/', '/dist/', '/coverage/'],
     transform: {

jest.setup.js

@@ -0,0 +1 @@
+console.error = jest.fn();

package.json

@@ -7,18 +7,17 @@
         "postinstall": "husky",
         "start": "node dist/index.js",
         "test": "jest",
-        "test:ci": "jest --coverage --testLocationInResults --json --outputFile=report.json",
+        "test:ci": "jest --ci --json --coverage --testLocationInResults --outputFile=report.json",
         "watch:build": "webpack --watch --mode development",
         "watch:server": "nodemon \"./dist/server.js\" --watch \"./dist\""
     },
     "dependencies": {
         "@google-cloud/storage": "^7.7.0",
-        "@govtechsg/oa-encryption": "^1.3.5",
-        "@veramo/utils": "5.5.3",
         "cors": "^2.8.5",
         "dotenv": "^16.4.1",
         "express": "^4.18.2",
-        "lodash": "^4.17.21"
+        "lodash": "^4.17.21",
+        "uuid": "^10.0.0"
     },
     "devDependencies": {
         "@babel/core": "^7.23.9",
@@ -29,6 +28,7 @@
         "@types/jest": "^29.5.12",
         "@types/lodash": "^4.14.202",
         "@types/node": "^20.11.16",
+        "@types/uuid": "^10.0.0",
         "@typescript-eslint/eslint-plugin": "^6.4.0",
         "@typescript-eslint/parser": "^6.21.0",
         "babel-jest": "^29.7.0",