The St. Louis Zoo Video Project is a comprehensive cloud-native solution designed to preserve, catalog, and make accessible decades of valuable animal behavior footage. This system enables efficient large video ingestion, manual and intelligent metadata video tagging, and supports advanced search capabilities for researchers and the general public.
- Enhanced Large Video Upload with both required and optional metadata fields.
- Automated video processing (clipping, thumbnail generation)
- AI-powered metadata extraction (date, timestamps)
- Advanced search functionality with multiple filters
- Preview clip for quick content assessment
- Allows users to request full video access, alerting administrators.
Refer to the provided diagram for a detailed view of the system architecture, which outlines both the Admin and User Flows. This setup integrates various AWS services to handle video uploads, video metadata tagging, storage, and retrieval processes efficiently.
- Login: Admins authenticate through Amazon Cognito to gain system access.
- Video Upload: Admins upload videos via the React Video Uploader App (deployed using AWS cognito).
- API Gateway: Manages the API calls for initiating and finalizing multi-part video uploads.
- Video Upload Lambda Functions:
- Initialize: Triggers when a new upload is started and initialize the multi-part video upload.
- Get Presigned URLs: Generates individual presigned URLs for each part of the video, facilitating the secure and reliable upload of large video files in parts to S3.
- Finalize: Completes the upload process once all parts of the video have been successfully uploaded. This action consolidates the parts into a single file in S3, making it ready for subsequent processing steps.
- Video Processing:
- Once videos are uploaded to s3 bucket, the below step function is triggered automatically using a lambda function.
- Step Functions: Orchestrate the video processing tasks.
- ECS with Fargate Task: Executes the processing tasks such as clipping 10 seconds video, thumbnail generation, and metadata extraction.
- Lambda Function with Amazon Bedrock (Claude 3.5): Generates video date, start time and end time metadata for each video .
- Lambda Function for OpenSearch: Calls the OpenSearch serverless service to store the metadata for each video.
- OpenSearch: Stores all video metadata, enabling efficient and advanced search capabilities.
- Glacier Archive: After processing, videos are archived 2 days after upload date in Glacier for long-term, cost-effective storage.
- Search and Filter Videos: Researchers access a web interface to search for videos using filters like duration, species (scientific/common name), location, group size, and more. Results are displayed, including metadata and links to video clips.
- Request Video Access: Researchers select videos and submit a request for download access. They fill out a form with details such as their name, faculty, and the reason for the request.
- Email Notification: The system sends an email containing the researcher’s request details to an administrator or contact person for review.
Before running the AWS CDK stack, ensure the following are installed and configured:
-
AWS Bedrock Access: Ensure your AWS account has access to the Claude 3.5 model in Amazon Bedrock.
- Request access through the AWS console if not already enabled
-
Download Node.js: Go to nodejs.org and download the latest LTS (Long-Term Support) version of Node.js for your operating system. This version is recommended because it is more stable and well-tested across multiple platforms.
-
Install Node.js: Run the downloaded installer. Follow the installation prompts ensuring that npm is selected to be installed alongside Node.js.
-
Verify Installation: After installation, you can verify that Node.js and npm are installed correctly by running the following commands in your terminal:
node --version npm --version
-
AWS CLI: To interact with AWS services and set up credentials.
- Install and configure AWS CLI
- Generate AWS Access keys and secret key for your AWS Account
- Setup credentials using the command and follow next steps to configure authentication.
aws configure
-
AWS CDK: For defining cloud infrastructure in code.
- Install and bootstrap AWS CDK
-
Docker: Required to build and run Docker images for the ECS tasks while deploying CDK code.
- Install Docker - https://www.docker.com/get-started/
-
AWS Account Permissions: Ensure your AWS account has the necessary permissions to create and manage the required resources (S3, Lambda, Step Functions, ECS, ECR, CloudWatch, Cognito, Opensearch Serverless, Amplify, SES, etc.)
├── backendV2 (Backend CDK code for AWS infrastructure and code that handles data processing and storage)
├── frontendV2 (UI React application for the administrative interface to upload and tag videos.)
└── frontend-user (UI React application that allows end users to search and view video previews.)
Clone the project
git clone https://github.com/ASUCICREPO/St-Louis-Zoo.git
This project is divided into three parts: AdminFrontend, UserFlowFrontend and Backend. You can find detailed instructions for each part in their respective README files.
The backend codebase facilitates large video uploads and handles video processing tasks such as clipping and thumbnail generation. It leverages AI to extract time and date metadata, storing all results in OpenSearch for efficient and quick data retrieval. Additionally, it contains the infrastructure as code (IaC).
For more information on the backend setup, refer to the Backend README.
The frontend provides a user-friendly interface for large video uploads and selecting associated metadata. Built using React, it connects with the backend via API calls to manage video processing tasks efficiently
For more information on the admin frontend setup, refer to the AdminFrontend README.
The frontend provides a user-friendly interface to search and shop videos. Built using React, it connects with the backend via API calls to manage video searching efficiently
For more information on the user flow frontend setup, refer to the UserFlowFrontend README.
Access the AWS Management Console and sign in with your credentials.
- In the AWS Management Console's search bar at the top, type
Cognito. - Select Amazon Cognito from the dropdown menu to open the Cognito dashboard.
- On the Amazon Cognito dashboard, you will see two options: User Pools and Identity Pools.
- Click on User Pools.
- Choose the user pool ZooAdminUserPool
- Click on Users from the left panel
- To add a new user, click Create user.
- Ensure that for field "Alias attributes used to sign in" Email is selected
- Select Send an email invitation
- Fill in the required fields such as username, email (as required by your user pool settings).
- Ensure to set the Mark email as verified if email is a required attribute.
- Enter the password
- Click Create user.
- With the admin user successfully created, you can now use these credentials to log in to the St. Louis Zoo Admin flow.
- Navigate to the login page of the St. Louis Zoo Admin portal and enter the admin username and password you set up.
- Upon successful authentication, you will have access to admin functionalities.
For the St. Louis Zoo project, we have implemented an S3 lifecycle policy to efficiently manage video storage costs and ensure long-term durability. Videos uploaded to the S3 bucket are automatically transitioned to Glacier Deep Archive for cost-effective storage.
- Rule Application: The lifecycle rule applies to all uploaded video files stored in the S3 bucket designated for the zoo's surveillance footage.
- Transition to Glacier Deep Archive: Uploaded videos are moved to Glacier Deep Archive 2 days after their upload date. This transition allows us to benefit from the lower storage costs associated with Glacier Deep Archive while keeping the videos accessible for future retrieval if needed.
- Clips and Thumbnails: To ensure quick access and usability, short clips and thumbnails extracted from the original videos are not archived. These derivatives are stored in standard S3 storage to facilitate immediate access and use in user search
Once Infastructure (backend) and FrontEnd's are deployed:
-
Bulk Upload: Admin can upload up to 20 large video files simultaneously.
-
Batch Tagging: All videos uploaded in a single batch are tagged with the same set of metadata/tags.
- Required Metadata/Tags: It is mandatory for Admin to provide specific metadata/tags that are crucial for the categorization and retrieval of the videos.
- Optional Metadata/Tags: Admin can additionally provide optional metadata/tags to enrich the video data further.
- User Flow: Users can search the video catalog using the smart search feature, view video previews, and retrieve metadata.
Make sure to deploy the backend before working on the frontend! 🛠️ This ensures that all the API endpoints , Lambda functions and all required Services are live and ready for your frontend to interact with.
Contributions to this project are welcome. Please fork the repository and submit a pull request with your changes