Skip to content

Commit

Permalink
Merge pull request #3 from MindFlavor/docker
Browse files Browse the repository at this point in the history
Docker support
  • Loading branch information
MindFlavor authored Dec 12, 2022
2 parents 4a1668c + c78dab1 commit 858ce4d
Show file tree
Hide file tree
Showing 6 changed files with 42 additions and 32 deletions.
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
extra/
target/
4 changes: 1 addition & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,10 @@ authors = ["Francesco Cogno <francesco.cogno@gmail.com>"]
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
anyhow = "1.0.66"
chrono = "0.4.22"
clap = { version = "4.0.22", features = ["derive"] }
clap = { version = "4.0.22", features = ["derive", "env"] }
csv = "1.1.6"
goauth = "0.13.1"
google-cloud-googleapis = { version = "0.6.0", features = ["pubsub"] }
Expand Down
16 changes: 10 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
FROM rustlang/rust:nightly as build
FROM ubuntu:latest as build

RUN apt-get update && apt-get install build-essential pkg-config libssl-dev curl -y
# install rust
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y

# create a new empty shell project
RUN USER=root cargo new --bin pgdelaytest
RUN $HOME/.cargo/bin/cargo new --bin pgdelaytest
WORKDIR /pgdelaytest

# copy over your manifests
Expand All @@ -12,17 +16,17 @@ COPY ./Cargo.toml ./Cargo.toml
COPY ./src ./src

# build for release
RUN cargo build --release
RUN $HOME/.cargo/bin/cargo build --release
RUN $HOME/.cargo/bin/cargo build

# our final base
FROM ubuntu:latest

RUN apt-get update
RUN apt-get dist-upgrade -y
RUN apt-get install ca-certificates libssl-dev tini tzdata -y
RUN apt-get update && apt-get dist-upgrade -y && apt-get install ca-certificates libssl-dev tini tzdata -y

# copy the build artifact from the build stage
COPY --from=build /pgdelaytest/target/release/pgdelaytest /usr/local/bin/.
COPY --from=build /pgdelaytest/target/release/pgdelaytest /usr/local/bin/pgdelaytest-debug

ENV RUST_LOG=info
ENTRYPOINT ["/usr/bin/tini", "--", "/usr/local/bin/pgdelaytest"]
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@ CREATE TABLE tbl(id INT, value INT);
INSERT INTO tbl(id, value) VALUES(1, 900);
```

Required parameters for the tool are the two connection strings (in the format specificed here: [https://docs.rs/tokio-postgres/latest/tokio_postgres/config/struct.Config.html](https://docs.rs/tokio-postgres/latest/tokio_postgres/config/struct.Config.html)). You can also optionally change the sleep time between tries and the pub/sub topic to write to. If you specify the pub/sub topic you need to specify a service account key file path that has enough privileges to write to the topic.

Required parameters for the tool are the two connection strings (in the format specificed here: [https://docs.rs/tokio-postgres/latest/tokio_postgres/config/struct.Config.html](https://docs.rs/tokio-postgres/latest/tokio_postgres/config/struct.Config.html)). You can also optionally change the sleep time between tries and the pub/sub topic to write to.

For example this command:

Expand All @@ -27,6 +26,16 @@ Tests the latency between `primary` and `secondary`, publishing the results both

Note that, in order to publish to pub/sub, a valid GCP identity must be available and proper permissions must be granted.

## Usage (Docker)

You can either build the container with `docker build . -t pgdelaytest:latest` or pull it from Docker.io. Then execute it passing env variables. For example:

```bash
docker run -e PRIMARY_CONNECTION_STRING="host=host user=test password=password" -e SECONDARY_CONNECTION_STRING="host=secondary user=test password=password" -e PUB_SUB_TOPIC=topic -e GOOGLE_APPLICATION_CREDENTIALS=/service_account_pvk.json -v /service_account_pvk.json:/service_account_pvk.json pgdelaytest:latest
```

*Note*: this example uses a service account key file, it is not necessary if you don't want to publish to pub/sub or you have default credentials at hand.

## Methodology

The tool updates a row on the primary and right away tries to get the same row from the secondary. If the value matches, the reported latency is zero. If not, the tool keeps querying the same row until the value matches and then reports the time taken as *replication latency*.
Expand Down
16 changes: 11 additions & 5 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use anyhow::{Context, Error};
use clap::Parser;
use event::Event;
use google_cloud_pubsub::client::Client as PubSubClient;
use options::{Commands, Options};
use options::Options;
use tokio_postgres::*;

const ID: i32 = 1;
Expand All @@ -15,13 +15,19 @@ async fn main() -> Result<(), Error> {
let options = Options::parse();
// println!("Using options {:?}", options);

let topic = if let Some(Commands::Publish { pub_sub_topic }) = options.command {
let pubsub = PubSubClient::default().await?;
let topic = pubsub.topic(&pub_sub_topic).new_publisher(None);
Some(topic)
let topic = if let Some(pub_sub_topic) = options.pub_sub_topic {
if pub_sub_topic.is_empty() {
None
} else {
println!("Events will be published to {} topic", pub_sub_topic);
let pubsub = PubSubClient::default().await?;
let topic = pubsub.topic(&pub_sub_topic).new_publisher(None);
Some(topic)
}
} else {
None
};

println!("Openinig connection to primary...");
let (pri_client, pri_connection) =
tokio_postgres::connect(&options.primary_connection_string, NoTls).await?;
Expand Down
23 changes: 7 additions & 16 deletions src/options.rs
Original file line number Diff line number Diff line change
@@ -1,31 +1,22 @@
use clap::{Parser, Subcommand};

#[derive(Subcommand, Debug)]
pub enum Commands {
/// publish events to Google Cloud Pub/Sub
Publish {
/// Pub/sub topic to publish to
#[arg(long)]
pub_sub_topic: String,
},
}
use clap::Parser;

/// Simple program to greet a person
#[derive(Parser, Debug)]
#[command(author, version, about, long_about = None)]
pub struct Options {
/// Primary instance (RW)
#[arg(short, long)]
#[arg(short, long, env = "PRIMARY_CONNECTION_STRING")]
pub primary_connection_string: String,

/// Secondary instance (RO)
#[arg(short, long)]
#[arg(short, long, env = "SECONDARY_CONNECTION_STRING")]
pub secondary_connection_string: String,

/// Sleep time in ms after each evaluation
#[arg(long, default_value_t = 1000)]
#[arg(long, default_value_t = 1000, env = "SLEEP_MS")]
pub sleep_ms: u64,

#[command(subcommand)]
pub command: Option<Commands>,
/// Pub/sub topic to publish to
#[arg(long, env = "PUB_SUB_TOPIC")]
pub pub_sub_topic: Option<String>,
}

0 comments on commit 858ce4d

Please sign in to comment.