Skip to content

A real time Tweet Trend Map and Sentiment Analysis web application with kafka, Angular, Spring Boot, Flink, Elasticsearch, Kibana, Docker and Kubernetes deployed on the cloud

License

Notifications You must be signed in to change notification settings

zmyzheng/TweetMap

Folders and files

NameName
Last commit message
Last commit date

Latest commit

e426ba0 · Feb 16, 2021
Feb 26, 2020
Jan 17, 2021
Jan 10, 2021
Jan 19, 2021
Jan 10, 2021
Feb 16, 2021
Feb 17, 2020
Mar 14, 2020
Jan 18, 2021
Jan 3, 2021
Mar 14, 2020
Mar 14, 2020
Jan 5, 2021
Feb 10, 2020
Nov 16, 2020
Nov 16, 2020
Feb 17, 2020
Mar 14, 2020

Repository files navigation

Tweet Map with Trends

  1. Build a java Tweet Collector to collect real-time tweets with Twitter Streaming API
  2. Push collected tweets to Kafka cluster
  3. Utilize Apache Flink Streaming to process (parse, filter and tranform) tweets
  4. Ingest processed tweets to Elasticsearch for data persistance and index
  5. Develop SpringBoot RESTful API server to query tweets from Elasticsearch
  6. Visualize real time tweet trends with Frontend Angular web application
  7. Show Geographical Tweet Heat Map with Kibana
  8. Create Docker image and deploy microservices to Kubernetes cluster

Project structure

  • tweet-collector: collect real-time tweets with Twitter Streaming API and push to Kafka cluster.

    • To support other social media, implement SocialMediaCollector interface.
    • To support other message queues, implement Sinkable interface.
  • rest-api-server: A RESTful API server querying tweets from Elasticsearch using SpringBoot framework.

  • flink-processor: Streaming process tweets and ingest into Elasticsearch.

  • frontend-website: Visualize Tweet Map with Angular framework. (developing)

Build

JDK version:

  1. master branch and dev-jdk11 branch: JDK 11
  2. dev-jdk8 branch: JDK 8

./gradlew :tweet-collector:clean :tweet-collector:build :tweet-collector:dockerPush

java -jar tweet-collector/build/libs/tweet-collector-1.1-SNAPSHOT.jar

./gradlew :flink-processor:clean :flink-processor:build