Hotel Data Integration System

System Architecture

Core Components

1. Supplier Layer

• BaseSupplier - Abstract base class for all suppliers
• Implementations: AcmeSupplier, PatagoniaSupplier, PaperfliesSupplier
• Managed by SupplierManager for unified data fetching.

2. Data Processing Layer

• Normalization: DataNormalizer for data cleaning and standardization
• Merging: DataMerger for combining data from multiple sources
• Validation: Uses Pydantic models in models/hotel.py

3. Storage Layer

• Abstract BaseDB interface
• Implementations: HotelDB and RawHotelDB

4. API Layer

• FastAPI-style routing with HotelAPI
• Endpoint handlers in api/routers

Data Flow

1. Data fetching from suppliers
2. Normalization and cleaning
3. Updating fully normalized data in database
4. Merging with bias handling for multiple suppliers
5. Storage in databases
6. API access to merged data

Design Principles

1. SOLID Principles

• Single Responsibility: Each class has one purpose (e.g., Normalizer)
• Open/Closed: Abstract base classes allow extension (e.g., BaseSupplier)
• Liskov Substitution: All database implementations follow BaseDB contract
• Interface Segregation: Targeted interfaces like AttributeNormalizer
• Dependency Injection: Services accept dependencies in constructors

2. Clean Architecture

• Clear separation between data, domain logic, and presentation
• Domain models independent of external concerns
• Use of interfaces for flexibility

3. Design Patterns

• Strategy: Different merger strategies in services/merger.py
• Factory: Supplier creation in SupplierManager
• Singleton: Logger implementation in utils/logger.py
• Decorator: API routing decorators

Key Features

1. Data Quality

• Robust normalization via HotelCleaner
• Configurable bias handling for data merging
• Validation using Pydantic models

2. Extensibility

• Easy to add new suppliers
• Pluggable normalizers and mergers
• Configurable via configs/config.py

3. Maintainability

• Comprehensive logging system
• Clear error handling with custom exceptions
• Well-defined interfaces

Usage

python main.py hotel_id1,hotel_id2 destination_id1