Guardian of Waterdrop and Spark
-
Updated
Dec 27, 2022 - Python
Guardian of Waterdrop and Spark
A python library to interact with the Spark History server.
I'll walk you through launching a cluster manually using Spark standalone deploy mode, as well as connecting an app to the cluster, launching the app, where to view the monitoring and logging.
Contains the code and examples for my article on Medium, which explains how to optimize computing data statistics in Apache Spark jobs using the Observations feature.
Add a description, image, and links to the spark-monitor topic page so that developers can more easily learn about it.
To associate your repository with the spark-monitor topic, visit your repo's landing page and select "manage topics."