whatgotdone

Architecture

Overview

What Got Done has a simple architecture consisting of the following parts:

• What Got Done Backend: A Go HTTP service (running on AppEngine) that handles all HTTP requests, datastore requests, and user authentication (via UserKit).
• What Got Done Frontend: A Vue2 app that renders pages in the user's browser.
• Cloud Firestore: What Got Done's storage provider.
• UserKit: A third-party service that manages What Got Done's user authentication.

Page Rendering Flow

What Got Done uses a somewhat unusual system for rendering pages. The Go backend first pre-renders the page server-side to populate tags related to SEO or social media that need to be set server-side. The Vue2 frontend renders the remainder of the page client-side. To avoid conflicts between the two systems' template syntax, Go uses [[, ]] delimiters, while Vue uses {{, }} delimiters.

The Go backend handles all of What Got Done's /api/* routes. These routes are What Got Done's RESTful interface between the frontend and the backend. These routes never send HTML, and instead only send JSON back and forth.

User authentication

What Got Done uses UserKit for user authentication. For user signup, user login, and password reset, What Got Done loads the UserKit UI widgets in JavaScript. On the backend, the auth package is responsible for translating UserKit auth tokens into What Got Done usernames.

Datastore

What Got Done uses Google Cloud Firestore for data storage.

Only the What Got Done backend can access the Firestore database. Specifically, the datastore package manages all interactions with Firestore.

E2E tests

What Got Done's end-to-end tests use Cypress and follow the testing pattern defined in the article End-to-End Testing Web Apps: The Painless Way. The testing architecture consists of two Docker containers (see docker-compose.yml):

• What Got Done container
• Cypress container

The Cypress container runs a browser to exercise What Got Done's critical functionality. It uses an independent environment and credentials from the production app so that nothing in the E2E tests affect state on production UserKit or Google Cloud Platform.

To run the E2E tests yourself, see the section below.

Contributing to What Got Done

Interested in contributing code or bug reports to What Got Done? That's great! Check our Contibutor Guidelines for more details.

QuickStart

To run What Got Done in a Docker container, run

docker-compose up

What Got Done will be running at http://localhost:3001.

Dev-mode authentication uses UserKit dummy mode. You can log in with any username using the password password.

Development notes

0. Pre-requisites

• Node.js (8.x or higher)
• Go (1.11 or higher)
• Docker (for E2E tests)

1. Start a Firestore emulator

Run the following command to start a Google Cloud Firestore Emulator in a Docker container:

export GOOGLE_CLOUD_PROJECT="dummy-local-gcp-project"
docker run \
  --detach \
  --env "FIRESTORE_PROJECT_ID=${GOOGLE_CLOUD_PROJECT}" \
  --env "PORT=8080" \
  --publish 8080:8080 \
  --name firestore-emulator \
  mtlynch/firestore-emulator

2. Build the frontend

To build the Vue frontend for What Got Done, run the following command:

pushd frontend && \
  npm install && \
  npm run build -- --mode development && \
  popd

3. Run the backend

To run the Go backend server, run the following command:

export USERKIT_SECRET="dummy.dummy"
export FIRESTORE_EMULATOR_HOST="localhost:8080"
mkdir bin
go build --tags 'dev' -o ./bin/main backend/main.go && \
  ./bin/main

What Got Done is now running on http://localhost:3001.

Dev-mode authentication uses UserKit dummy mode. You can log in with any username using the password password.

Optional: Run frontend with hot reloading

If you're making changes to the Vue code, you'll probably want to run the standard Vue HTTP server with hot reloading. Keep the backend running, and in a separate shell session, run the following command:

cd frontend && \
  npm run serve

A hot-reloading Vue server will run on port http://localhost:8085. It will communicate with the What Got Done backend at port 3001.

Quirks of the dev environment

Because the production What Got Done server runs both the frontend and the backend on a single port, there are a few hacks to make a development version work:

• CORS is enabled in dev mode so that the frontend can make CORS requests to the backend from a different HTTP port.
• CSRF protection is disabled in dev mode because the Vue dev server doesn't know how to render the <meta name="csrf-token" /> tag.
• Page titles don't work properly in dev mode because the Vue dev server doesn't know how to render the <title> tag.
• The Content Security Policy header in dev mode needs unsafe-eval and unsafe-inline, whereas we disable this in production.

Optional: Run backend unit tests

Unit tests run in normal Golang fashion:

go test ./...

Optional: Run integration tests

Integration tests run all components together using a local Firestore emulator as the datastore and UserKit dummy mode as authentication:

dev-scripts/run-integration-tests

Optional: Run E2E tests

The E2E tests are a subset of the integration tests, but they use a real GCP Firestore instance instead of a local emulator.

To run the end to end tests, you'll need to create a dedicated GCP project. Specify the GCP project in e2e/docker-compose.yml under GOOGLE_CLOUD_PROJECT.

Then, run the E2E tests as follows:

dev-scripts/run-e2e-tests

Integration tests vs. E2E tests

Integration tests have the advantage of running faster because they have fewer remote dependencies. It's also possible for third-party developers to run them because they require no secrets (this also allows CI to run them on third-party pull requests).

E2E tests are more likely to catch real-world bugs because their configuration more closely matches production infrastructure.

Optional: Enable public analytics from Google Analytics

What Got Done supports pulling metrics from Google Analytics into the page content. To enable this:

1. Enable the Google Analytics Reporting API in your Google Cloud Platform project.
2. Create a service account in Google Cloud Platform console for your What Got Done project.
   1. Assign the service account no permissions/roles, but save its private key as JSON.
   2. Click "Create Key" to create a private key and save it in JSON format as google-analytics-service-account.json in the What Got Done root directory.
3. In Google Analytics, open Admin > View > View User Management and add the email address of the service account you just created (it will have an email like [name]@[project ID].iam.gserviceaccount.com.
   1. Grant the user only "Read & Analyze" permissions.
4. In Google Analytics, open Admin > View > View Settings
5. Save the View ID as an environment variable like export GOOGLE_ANALYTICS_VIEW_ID=12345789