v2.1.0 release

# Release Notes

## New Features
- **Add `frames_extractor`**:
  - Added `all_frames` flag to `best_frames_extractor`.
  - Implemented `all_frames` logic in `best_frames_extractor` and updated tests accordingly.
  - Added `all_frames` flag to `service_initializer` and updated tests.

- **New Sections and Enhancements in Documentation**:
  - Added section Architecture and update README
  - Created and updated `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `SECURITY.md`, and `pull_request_template.md`.
  - Added visualizations and demo presentation image to the README.
  - Compressed static files and images for better performance.
  - Updated issue templates for bug reports and feature requests.

## Bug Fixes
- Fixed logo text color.
- Fixed href links in various sections.
- Fixed typos and improved text in both English and Polish READMEs.
- Fixed code_of_conduct href in `CONTRIBUTING.md`.
- Removed non-functional README.md badge.
- Fixed typo in extractors.

## Refactoring and Improvements
- Replaced `|` with `typing.Union`.
- Renamed methods for consistency:
- `get_extractor()` -> `create_extractor()` in `ExtractorFactory`.
- Changed method name in `OpenCVVideo` from `get_next_video_frames` to `get_next_frames`.
- Moved normalizing images from evaluator to extractors.
- Added `normalize_images` method as an abstract method in `image_processors.py`.
- Moved `ServiceShutdownSignal` inside `DockerManager`.
- Made `ServiceInitializer` attributes protected.
- Added license to docstrings.
- Added badges and fixed language links.

## Other Changes
- Created and updated various markdown files in the `.github` directory.
- Improved docker configurations by adding `.dockerignore`.

.github/README.pl.md

@@ -79,6 +79,25 @@
             <p>Output: Obrazy zapisane jako <code>.jpg</code>.</p>
         </ol>
     </details>
+    <br>
+    <details>
+        <summary>
+            <strong>🆕 Frames Extraction 🖼️🖼️🖼️</strong>
+            <blockquote>Zamienia pliki video na klatki.</blockquote>
+        </summary>
+        <p>Modyfikuje <code>best_frames_extractor</code> poprzez pominięcie części z AI/ocenianiem klatek.</p>
+        <code>python start.py best_frames_extractor --all_frames</code>
+        <ol>
+            <p>Input: Folder z plikami video <code>.mp4</code>.</p>
+            <li>Bierze pierwsze video ze wskazanej lokalizacji.</li>
+            <li>
+                Dzieli wideo na klatki. Klatki są brane co 1 sekundę wideo.
+                Klatki są przetwarzane w batchach(seriach).
+            </li>
+            <li>Zapisuje wszystkie klatki w wybranej lokalizacji.</li>
+            <p>Output: Klatki zapisane jako <code>.jpg</code>.</p>
+        </ol>
+    </details>
 </div>
 <div id="installation">
     <h2>💿 Instalacja</h2>
@@ -199,23 +218,32 @@
     <h2>💡O projekcie:</h2>
     <div id="contents">
         <h3>Spis treści:</h3>
-        <a href="#how-it-works">Jak to działa</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#input">Input modelu</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#output">Wyniki oceniania obrazów</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#classes">Klasy estetyczne</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#calculating-mean">Obliczanie ostatecznej oceny obrazu</a><br>
-        <a href="#implementation">Jak to jest zaimplementowane w skrócie</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#model-architecture">Architektura modelu</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#weights">Wagi modelu</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#normalization">Normalizacja obrazów</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#predictions">Przewidywanie przynależności do klas</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#mean-calculation">Obliczanie średniej ważonej</a><br>
-        <a href="#1vs2">v1.0 vs v2.0</a><br>
-        <a href="#build-with">Użyte technologie</a><br>
-        <a href="#tests">Testy</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#unit">jednostkowe</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#integration">integracyjne</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#e2e">e2e</a><br>
+        <ul>
+            <li><a href="#how-it-works">Jak to działa</a></li>
+            <ul>
+                <li><a href="#input">Input modelu</a></li>
+                <li><a href="#output">Wyniki oceniania obrazów</a></li>
+                <li><a href="#classes">Klasy estetyczne</a></li>
+                <li><a href="#calculating-mean">Obliczanie ostatecznej oceny obrazu</a></li>
+            </ul>
+            <li><a href="#implementation">Implementacja w skrócie</a></li>
+            <ul>
+                <li><a href="#model-architecture">Architektura modelu</a></li>
+                <li><a href="#weights">Wagi modelu</a></li>
+                <li><a href="#normalization">Normalizacja obrazów</a></li>
+                <li><a href="#predictions">Przewidywanie przynależności do klas</a></li>
+                <li><a href="#mean-calculation">Obliczanie średniej ważonej</a></li>
+            </ul>
+            <li><a href="#1vs2">v1.0 vs v2.0</a></li>
+            <li><a href="#architecture">Architektura</a></li>
+            <li><a href="#build-with">Użyte technologie</a></li>
+            <li><a href="#tests">Testy</a></li>
+            <ul>
+                <li><a href="#unit">Jednostkowe</a></li>
+                <li><a href="#integration">Integracyjne</a></li>
+                <li><a href="#e2e">E2E</a></li>
+            </ul>
+        </ul>
     </div>
     <div id="how-it-works">
     <h2>📐 Jak to działa</h2>
@@ -372,6 +400,10 @@
         </ul>
         <img src="../static/performance.png" height="200">
     </div>
+    <div id="architecture">
+        <h2>Architektura</h2>
+        <img src="../static/architecture.jpg" width="1000" style="border-radius: 10px;">
+    </div>
     <div id="build-with">
     <h2>🛠️ Użyte technologie</h2>
     <ul>

README.md

@@ -79,6 +79,26 @@
             <p>Output: Images saved as <code>.jpg</code>.</p>
         </ol>
     </details>
+    <br>
+    <details>
+        <summary>
+            <strong>🆕 Frames Extraction 🖼️🖼️🖼️</strong>
+            <blockquote>Extract and return frames from a video.</blockquote>
+        </summary>
+        <p>Modifying <code>best_frames_extractor</code> by skipping AI evaluation part.</p>
+        <code>python start.py best_frames_extractor --all_frames</code>
+        <ol>
+            <p>Input: Folder with <code>.mp4</code> video files.</p>
+            <li>Takes the first video from the specified location.</li>
+            <li>
+                Splits the video into frames.
+                Frames are taken at 1-second intervals.
+                Frames are processed in batches.
+            </li>
+            <li>Saves all frames in the chosen location.</li>
+            <p>Output: Frames saved as <code>.jpg</code>.</p>
+        </ol>
+    </details>
 </div>
 <div id="installation">
     <h2>💿 Installation</h2>
@@ -201,23 +221,32 @@
     <h2>💡 About:</h2>
     <div id="contents">
         <h3>Table of Contents:</h3>
-        <a href="#how-it-works">How it works</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#input">Model Input</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#output">Image Rating Results</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#classes">Aesthetic Classes</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#calculating-mean">Calculating the Final Image Score</a><br>
-        <a href="#implementation">Implementation in Brief</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#model-architecture">Model Architecture</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#weights">Pre-trained Weights</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#normalization">Image Normalization</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#predictions">Class Predictions</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#mean-calculation">Weighted Mean Calculation</a><br>
-        <a href="#1vs2">v1.0 vs v2.0</a><br>
-        <a href="#build-with">Build with</a><br>
-        <a href="#tests">Tests</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#unit">unit</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#integration">integration</a><br>
-        &nbsp&nbsp&nbsp&nbsp<a href="#e2e">e2e</a><br>
+        <ul>
+            <li><a href="#how-it-works">How it works</a></li>
+            <ul>
+                <li><a href="#input">Model Input</a></li>
+                <li><a href="#output">Image Rating Results</a></li>
+                <li><a href="#classes">Aesthetic Classes</a></li>
+                <li><a href="#calculating-mean">Calculating the Final Image Score</a></li>
+            </ul>
+            <li><a href="#implementation">Implementation in Brief</a></li>
+            <ul>
+                <li><a href="#model-architecture">Model Architecture</a></li>
+                <li><a href="#weights">Pre-trained Weights</a></li>
+                <li><a href="#normalization">Image Normalization</a></li>
+                <li><a href="#predictions">Class Predictions</a></li>
+                <li><a href="#mean-calculation">Weighted Mean Calculation</a></li>
+            </ul>
+            <li><a href="#1vs2">v1.0 vs v2.0</a></li>
+            <li><a href="#architecture">Architecture</a></li>
+            <li><a href="#build-with">Build with</a></li>
+            <li><a href="#tests">Tests</a></li>
+            <ul>
+                <li><a href="#unit">unit</a></li>
+                <li><a href="#integration">integration</a></li>
+                <li><a href="#e2e">e2e</a></li>
+            </ul>
+        </ul>
     </div>
     <div id="how-it-works">
     <h2>📐 How it Works</h2>
@@ -352,6 +381,10 @@
         </ul>
         <img src="static/performance.png" height="200">
     </div>
+    <div id="architecture">
+        <h2>Architecture</h2>
+        <img src="static/architecture.jpg" width="1000" style="border-radius: 10px;">
+    </div>
     <div id="build-with">
     <h2>🛠️ Built with</h2>
     <ul>

config.py

@@ -1,4 +1,22 @@
-"""Main configuration dataclass for extractor service manager tool."""
+"""
+Main configuration dataclass for extractor service manager tool.
+LICENSE
+=======
+Copyright (C) 2024  Bartłomiej Flis
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+"""
 from dataclasses import dataclass
 from pathlib import Path
 

extractor_service/app/extractor_manager.py

@@ -1,6 +1,22 @@
 """
 This module provides manager class for running extractors and
 managing extraction process lifecycle.
+LICENSE
+=======
+Copyright (C) 2024  Bartłomiej Flis
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
 """
 import logging
 from typing import Type
@@ -49,7 +65,7 @@ def start_extractor(cls, background_tasks: BackgroundTasks, config: ExtractorCon
         """
         cls._config = config
         cls._check_is_already_extracting()
-        extractor_class = ExtractorFactory.get_extractor(extractor_name)
+        extractor_class = ExtractorFactory.create_extractor(extractor_name)
         background_tasks.add_task(cls.__run_extractor, extractor_class, extractor_name)
         message = f"'{extractor_name}' started."
         return message

extractor_service/app/extractors.py

@@ -5,6 +5,22 @@
     - Extractors:
         - BestFramesExtractor: For extracting best frames from all videos from any directory.
         - TopImagesExtractor: For extracting images with top percent evaluating from any directory.
+LICENSE
+=======
+Copyright (C) 2024  Bartłomiej Flis
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
 """
 from concurrent.futures import ThreadPoolExecutor
 from pathlib import Path
@@ -88,17 +104,17 @@ def _list_input_directory_files(self, extensions: tuple[str],
         logger.debug("Listed file paths: %s", files)
         return files
 
-    def _evaluate_images(self, images: list[np.ndarray]) -> np.array:
+    def _evaluate_images(self, normalized_images: np.ndarray) -> np.array:
         """
         Rating all images in provided images batch using already initialized image evaluator.
 
         Args:
-            images (list[np.ndarray]): List of images in numpy ndarrays.
+            normalized_images (list[np.ndarray]): Already normalized images np.ndarray for evaluating.
 
         Returns:
             np.array: Array with images scores in given images order.
         """
-        scores = np.array(self._image_evaluator.evaluate_images(images))
+        scores = np.array(self._image_evaluator.evaluate_images(normalized_images))
         return scores
 
     @staticmethod
@@ -139,6 +155,21 @@ def _save_images(self, images: list[np.ndarray]) -> None:
             for future in futures:
                 future.result()
 
+    @staticmethod
+    def _normalize_images(images: list[np.ndarray], target_size: tuple[int, int]) -> np.ndarray:
+        """
+        Normalize all images in given list to target size for further operations.
+
+        Args:
+            images (list[np.ndarray]): List of np.ndarray images to normalize.
+            target_size (tuple[int, int]): Images will be normalized to this size.
+
+        Returns:
+            np.ndarray: All images as a one numpy array.
+        """
+        normalized_images = OpenCVImage.normalize_images(images, target_size)
+        return normalized_images
+
     @staticmethod
     def _add_prefix(prefix: str, path: Path) -> Path:
         """
@@ -169,7 +200,7 @@ def _signal_readiness_for_shutdown() -> None:
 class ExtractorFactory:
     """Extractor factory for getting extractors class by their names."""
     @staticmethod
-    def get_extractor(extractor_name: str) -> Type[Extractor]:
+    def create_extractor(extractor_name: str) -> Type[Extractor]:
         """
         Match extractor class by its name and return its class.
 
@@ -221,12 +252,16 @@ def _extract_best_frames(self, video_path: Path) -> list[np.ndarray]:
             list[np.ndarray]: List of best images(frames) from the given video.
         """
         best_frames = []
-        frames_batch_generator = OpenCVVideo.get_next_video_frames(video_path, self._config.batch_size)
+        frames_batch_generator = OpenCVVideo.get_next_frames(video_path, self._config.batch_size)
         for frames in frames_batch_generator:
             if not frames:
                 continue
-            logger.debug("Frames pack generated.")
-            scores = self._evaluate_images(frames)
+            logger.debug("Frames batch generated.")
+            if self._config.all_frames:
+                best_frames.extend(frames)
+                continue
+            normalized_images = self._normalize_images(frames, self._config.target_image_size)
+            scores = self._evaluate_images(normalized_images)
             selected_frames = self._get_best_frames(frames, scores,
                                                     self._config.compering_group_size)
             best_frames.extend(selected_frames)
@@ -268,7 +303,8 @@ def process(self) -> None:
         for batch_index in range(0, len(images_paths), self._config.batch_size):
             batch = images_paths[batch_index:batch_index + self._config.batch_size]
             images = self._read_images(batch)
-            scores = self._evaluate_images(images)
+            normalized_images = self._normalize_images(images, self._config.target_image_size)
+            scores = self._evaluate_images(normalized_images)
             top_images = self._get_top_percent_images(images, scores,
                                                       self._config.top_images_percent)
             self._save_images(top_images)