02.아키텍처_기술스택.md

아키텍처 및 기술스택 선정하기

Architecture Overview

• 사용자
  • MySQL: 사용자 정보 저장
  • Kafka: 사용자 관련 처리를 위한 발송관리용
  • Redis: 사용자 토큰, 사용자 정보 캐싱
• 프로젝트
  • MySQL: 프로젝트 정보 저장
  • Redis: 프로젝트 정보 캐싱
• API 스펙관리
  • MySQL: 스펙 메타정보 저장
  • Kafka: 파일 후처리 전용 (삭제, git/S3/HostFile 동기화)
  • Redis: 스펙정보 캐싱

기술스택 선정

• Language: kotlin
• Web Framework: SpringBoot
• Cache Tool: Redis
• ORM: JPA + query dsl + jooQ
• DBMS: MySQL
• Event Broker: Kafka
• File Storage: S3
• Source Version Control: GitHub
• ETC:
  • Tika: File Media Type Check Library
  • ...

Docker Compose

• Docker Compose는 여러 Docker 컨테이너들을 하나의 환경에서 수행될 수 있도록 컨테이너 관리를 수행할 수 있는 도구이다.

설정

• 우리는 다음과 같은 툴을 설치할 것이다.

• Kafka 설정

  • ZooKeeper: Kafka의 메타정보 및 상태관리
  • Kafka 1 ~ 3: Kafka Broker
  • Kafka-UI: UI for Apache kafka 로 카프카 모니터링 수행

• Redis 설정

• MySQL 설정

version: '3.7'
services:
  zookeeper-1:
    image: confluentinc/cp-zookeeper:5.5.1
    platform: linux/amd64
    ports:
      - '32181:32181'
    environment:
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000

  kafka-1:
    image: confluentinc/cp-kafka:5.5.1
    platform: linux/amd64
    ports:
      - '9092:9092'
    depends_on:
      - zookeeper-1
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:32181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka-1:29092,EXTERNAL://localhost:9092
      KAFKA_DEFAULT_REPLICATION_FACTOR: 3
      KAFKA_NUM_PARTITIONS: 3

  kafka-2:
    image: confluentinc/cp-kafka:5.5.1
    platform: linux/amd64
    ports:
      - '9093:9093'
    depends_on:
      - zookeeper-1
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:32181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka-2:29093,EXTERNAL://localhost:9093
      KAFKA_DEFAULT_REPLICATION_FACTOR: 3
      KAFKA_NUM_PARTITIONS: 3

  kafka-3:
    image: confluentinc/cp-kafka:5.5.1
    platform: linux/amd64
    ports:
      - '9094:9094'
    depends_on:
      - zookeeper-1
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:32181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka-3:29094,EXTERNAL://localhost:9094
      KAFKA_DEFAULT_REPLICATION_FACTOR: 3
      KAFKA_NUM_PARTITIONS: 3

  redis_container:
    image: redis:latest
    container_name: redis_test
    ports:
      - 6379:6379
    volumes:
      - ./redis/data:/data
      - ./redis/conf/redis.conf:/usr/local/conf/redis.conf
    labels:
      - "name=redis"
      - "mode=standalone"
    restart: always
    command: redis-server /usr/local/conf/redis.conf

  mysql:
    image: mysql:latest
    platform: linux/arm64
    restart: always
    volumes:
      - ./db/mysql/data:/var/lib/mysql
      - ./db/mysql/init:/docker-entrypoint-initdb.d
    ports:
      - 3306:3306
    environment:
      MYSQL_ROOT_PASSWORD: root1234
      TZ: Asia/Seoul
      MYSQL_DATABASE: myDb
      MYSQL_USER: testuser
      MYSQL_PASSWORD: testuser1234

  kafka-ui:
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:latest
    ports:
      - 8080:8080
    depends_on:
      - kafka-1
      - kafka-2
      - kafka-3
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: INTERNAL://kafka-1:29092,INTERNAL://kafka-2:29093,INTERNAL://kafka-3:29094
      KAFKA_CLUSTERS_0_ZOOKEEPER: "zookeeper-1:32181"

실행

• docker-compose.yaml 파일이 존재하는 위치로 이동한다.
• 이후 docker-compose -f docker-compose.yaml up -d 를 수행한다.

cd sources/infra-sources

docker-compose -f docker-compose.yaml up -d

• 다음과 같은 결과를 얻을 수 있다.

[+] Running 11/11
 ✔ Container redis_test2                                                                                                                                           Started      3.7s 
 ✔ Container infra-sources-zookeeper-1-1                                                                                                                           Started      3.6s 
 ✔ Container my-dynamodb                                                                                                                                           Started      3.7s 
 ✔ Container infra-sources-mysql-1                                                                                                                                 Started      3.8s 
 ✔ Container redis_test                                                                                                                                            Started      3.7s 
 ! redis_container2 The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested              0.0s 
 ! redis_container The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested               0.0s 
 ✔ Container infra-sources-kafka-3-1                                                                                                                               Started      5.4s 
 ✔ Container infra-sources-kafka-1-1                                                                                                                               Started      5.3s 
 ✔ Container infra-sources-kafka-2-1                                                                                                                               Started      5.3s 
 ✔ Container kafka-ui                                                                                                                                              Started      5.8s

정지하기

• 정지하기 위해서는 다음과 같은 명령어를 수행한다.

docker-compose -f docker-compose.yaml down