Merge pull request #1839 from EnterpriseDB/release/2021-09-13

Release: 2021-09-13

advocacy_docs/kubernetes/cloud_native_postgresql/architecture.mdx

@@ -4,44 +4,58 @@ originalFilePath: 'src/architecture.md'
 product: 'Cloud Native Operator'
 ---
 
-For High Availability goals, the PostgreSQL database management system provides administrators with built-in **physical replication** capabilities based on **Write Ahead Log (WAL) shipping**.
+For High Availability and Scalability goals, the PostgreSQL database management
+system provides administrators with built-in **physical replication**
+capabilities based on **Write Ahead Log (WAL) shipping**.
 
-PostgreSQL supports both asynchronous and synchronous streaming replication, as well as asynchronous file-based log shipping (normally used as a fallback option, for example, to store WAL files in an object store). Replicas are usually called *standby servers* and can also be used for read-only workloads, thanks to the *Hot Standby* feature.
+PostgreSQL supports both asynchronous and synchronous streaming replication
+over the network, as well as asynchronous file-based log shipping (normally
+used as a fallback option, for example, to store WAL files in an object store).
+Replicas are usually called *standby servers* and can also be used for
+read-only workloads, thanks to the *Hot Standby* feature.
 
-Cloud Native PostgreSQL currently supports clusters based on asynchronous and synchronous streaming replication to manage multiple hot standby replicas, with the following specifications:
+Cloud Native PostgreSQL supports clusters based on asynchronous and synchronous
+streaming replication to manage multiple hot standby replicas within the same
+Kubernetes cluster, with the following specifications:
 
 * One primary, with optional multiple hot standby replicas for High Availability
 * Available services for applications:
     * `-rw`: applications connect to the only primary instance of the cluster
     * `-ro`: applications connect to the only hot standby replicas for read-only-workloads
     * `-r`: applications connect to any of the instances for read-only workloads
 * Shared-nothing architecture recommended for better resilience of the PostgreSQL cluster:
-    * PostgreSQL instances should reside on different Kubernetes worker nodes and share only the network
-    * PostgreSQL instances can reside in different availability zones in the same region
+    * PostgreSQL instances should reside on different Kubernetes worker nodes
+      and share only the network
+    * PostgreSQL instances can reside in different
+      availability zones in the same region
     * All nodes of a PostgreSQL cluster should reside in the same region
 
 !!! Seealso "Replication"
     Please refer to the ["Replication" section](replication.md) for more
-    information about how Cloud Native PostgreSQL relies on PostgreSQL replication.
+    information about how Cloud Native PostgreSQL relies on PostgreSQL replication,
+    including synchronous settings.
 
 ## Read-write workloads
 
-Applications can decide to connect to the PostgreSQL instance elected as *current primary*
-by the Kubernetes operator, as depicted in the following diagram:
+Applications can decide to connect to the PostgreSQL instance elected as
+*current primary* by the Kubernetes operator, as depicted in the following
+diagram:
 
 ![Applications writing to the single primary](./images/architecture-rw.png)
 
 Applications can use the `-rw` suffix service.
 
 In case of temporary or permanent unavailability of the primary, Kubernetes
-will move the `-rw` to another instance of the cluster for high availability
+will move the `-rw` service to another instance of the cluster for high availability
 purposes.
 
 ## Read-only workloads
 
 !!! Important
-    Applications must be aware of the limitations that [Hot Standby](https://www.postgresql.org/docs/current/hot-standby.html)
-    presents and familiar with the way PostgreSQL operates when dealing with these workloads.
+    Applications must be aware of the limitations that
+    [Hot Standby](https://www.postgresql.org/docs/current/hot-standby.html)
+    presents and familiar with the way PostgreSQL operates when dealing with
+    these workloads.
 
 Applications can access hot standby replicas through the `-ro` service made available
 by the operator. This service enables the application to offload read-only queries from the
@@ -51,7 +65,8 @@ The following diagram shows the architecture:
 
 ![Applications reading from hot standby replicas in round robin](./images/architecture-read-only.png)
 
-Applications can also access any PostgreSQL instance at any time through the `-r` service at connection time.
+Applications can also access any PostgreSQL instance through the
+`-r` service.
 
 ## Application deployments
 
@@ -67,35 +82,35 @@ implement a form of Virtual IP as described in the
 ["Service" page of the Kubernetes Documentation](https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies).
 
 !!! Hint
-    It is highly recommended to use those services in your applications,
-    and avoid connecting directly to a specific PostgreSQL instance, as the latter
+    It is highly recommended using those services in your applications,
+    and avoiding connecting directly to a specific PostgreSQL instance, as the latter
     can change during the cluster lifetime.
 
 You can use these services in your applications through:
 
 * DNS resolution
 * environment variables
 
-As far as the credentials to connect to PostgreSQL are concerned, you can
+For the credentials to connect to PostgreSQL, you can
 use the secrets generated by the operator.
 
 !!! Warning
     The operator will create another service, named `[cluster name]-any`. That
     service is used internally to manage PostgreSQL instance discovery.
     It's not supposed to be used directly by applications.
 
-## DNS resolution
+### DNS resolution
 
-You can use the Kubernetes DNS service, which is required by this operator,
-to point to a given server.
-You can do that by just using the name of the service if the application is
+You can use the Kubernetes DNS service to point to a given server.
+The Kubernetes DNS service is required by the operator.
+You can do that by using the name of the service if the application is
 deployed in the same namespace as the PostgreSQL cluster.
 In case the PostgreSQL cluster resides in a different namespace, you can use the
 full qualifier: `service-name.namespace-name`.
 
 DNS is the preferred and recommended discovery method.
 
-## Environment variables
+### Environment variables
 
 If you deploy your application in the same namespace that contains the
 PostgreSQL cluster, you can also use environment variables to connect to the database.
@@ -112,10 +127,10 @@ you can use the following environment variables in your applications:
 * `PG_DATABASE_RW_SERVICE_HOST`: the IP address of the
   service pointing to the *primary* instance of the cluster
 
-## Secrets
+### Secrets
 
-The PostgreSQL operator will generate two secrets for every PostgreSQL cluster
-it deploys:
+The PostgreSQL operator will generate two `basic-auth` type secrets for every
+PostgreSQL cluster it deploys:
 
 * `[cluster name]-superuser`
 * `[cluster name]-app`
@@ -129,3 +144,80 @@ connecting to the PostgreSQL cluster.
 
 The `-superuser` ones are supposed to be used only for administrative purposes.
 
+## Multi-cluster deployments
+
+!!! Info
+    Cloud Native PostgreSQL supports deploying PostgreSQL across multiple
+    Kubernetes clusters through a feature called **Replica Cluster**,
+    which is described in this section.
+
+In a distributed PostgreSQL cluster there can only be a single PostgreSQL
+instance acting as a primary at all times. This means that applications can
+only write inside a single Kubernetes cluster, at any time.
+
+!!! Tip
+   If you are interested in a PostgreSQL architecture where all instances accept writes, 
+   please take a look at  [BDR (Bi-Directional Replication) by EDB](https://www.enterprisedb.com/docs/bdr/latest/). 
+   For Kubernetes, BDR will have its own Operator, expected late in 2021. 
+
+However, for business continuity objectives it is fundamental to:
+
+- reduce global **recovery point objectives** (RPO) by storing PostgreSQL backup data
+  in multiple locations, regions and possibly using different providers
+  (**Disaster Recovery**)
+- reduce global **recovery time objectives** (RTO) by taking advantage of PostgreSQL
+  replication beyond the primary Kubernetes cluster (**High Availability**)
+
+In order to address the above concerns, Cloud Native PostgreSQL introduces the
+concept of a *PostgreSQL Replica Cluster*. Replica clusters are the Cloud
+Native PostgreSQL way to enable multi-cluster deployments in private, public,
+hybrid, and multi-cloud contexts.
+
+A replica cluster is a separate `Cluster` resource:
+
+1. having either `pg_basebackup` or full `recovery` as the `bootstrap`
+   option from a defined external source cluster
+2. having the `replica.enabled` option set to `true`
+3. replicating from a defined external cluster identified by `replica.source`,
+   normally located outside the Kubernetes cluster
+4. replaying WAL information received from the recovery object store
+   (using PostgreSQL's `restore_command` parameter), or via streaming
+   replication (using PostgreSQL's `primary_conninfo` parameter), or any of
+   the two (in case both the `barmanObjectStore` and `connectionParameters`
+   are defined in the external cluster)
+5. accepting only read connections, as supported by PostgreSQL's Hot Standby
+
+!!! Seealso
+    Please refer to the ["Bootstrap" section](bootstrap.md) for more information
+    about cloning a PostgreSQL cluster from another one (defined in the
+    `externalClusters` section).
+
+The diagram below depicts a PostgreSQL cluster spanning over two different
+Kubernetes clusters, where the primary cluster is in the first Kubernetes
+cluster and the replica cluster is in the second. The second Kubernetes cluster
+acts as the company's disaster recovery cluster, ready to be activated in case
+of disaster and unavailability of the first one.
+
+![An example of multi-cluster deployment with a primary and a replica cluster](./images/multi-cluster.png)
+
+A replica cluster can have the same architecture of the primary cluster. In
+place of the primary instance, a replica cluster has a **designated primary**
+instance, which is a standby server with an arbitrary number of cascading
+standby servers in streaming replication (symmetric architecture).
+
+The designated primary can be promoted at any time, making the replica cluster
+a primary cluster capable of accepting write connections.
+
+!!! Warning
+    Cloud Native PostgreSQL does not perform any cross-cluster switchover
+    or failover at the moment. Such operation must be performed manually
+    or delegated to a multi-cluster/federated cluster aware authority.
+    Each PostgreSQL cluster is independent from any other.
+
+The designated primary in the above example is fed via WAL streaming
+(`primary_conninfo`), with fallback option for file-based WAL shipping through
+the `restore_command` and `barman-cloud-wal-restore`.
+
+Cloud Native PostgreSQL allows you to define multiple replica clusters.
+You can also define replica clusters with a lower number of replicas, and then
+increase this number when the cluster is promoted to primary.

advocacy_docs/kubernetes/cloud_native_postgresql/backup_recovery.mdx

@@ -23,11 +23,23 @@ as it is composed of a community PostgreSQL image and the latest
     Barman to Barman Cloud in the future. For the time being, it is your responsibility 
     to configure retention policies directly on the object store. 
 
-## Cloud credentials
+## Cloud provider support
 
-You can archive the backup files in any service whose API is compatible
-with AWS S3. You will need the following information about your
-environment:
+You can archive the backup files in any service that is supported
+by the Barman Cloud infrastructure. That is:
+
+- [AWS S3](https://aws.amazon.com/s3/)
+- [Microsoft Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/).
+
+You can also use any compatible implementation of the
+supported services.
+
+The required setup depends on the chosen storage provider and is
+discussed in the following sections.
+
+### S3
+
+You will need the following information about your environment:
 
 - `ACCESS_KEY_ID`: the ID of the access key that will be used
   to upload files in S3
@@ -50,10 +62,6 @@ kubectl create secret generic aws-creds \
 The credentials will be stored inside Kubernetes and will be encrypted
 if encryption at rest is configured in your installation.
 
-## Configuring the Cluster
-
-### S3
-
 Given that secret, you can configure your cluster like in
 the following example:
 
@@ -113,8 +121,8 @@ spec:
 ### MinIO Gateway
 
 Optionally, you can use MinIO Gateway as a common interface which
-relays backup objects to other cloud storage solutions, like S3, GCS or
-Azure. For more information, please refer to [MinIO official documentation](https://docs.min.io/).
+relays backup objects to other cloud storage solutions, like S3 or GCS.
+For more information, please refer to [MinIO official documentation](https://docs.min.io/).
 
 Specifically, the Cloud Native PostgreSQL cluster can directly point to a local
 MinIO Gateway as an endpoint, using previously created credentials and service.
@@ -231,6 +239,79 @@ spec:
 Verify on `s3://BUCKET_NAME/` the presence of archived WAL files before
 proceeding with a backup.
 
+### Azure Blob Storage
+
+In order to access your storage account, you will need one of the following combinations
+of credentials:
+
+- [**Connection String**](https://docs.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string#configure-a-connection-string-for-an-azure-storage-account)
+- **Storage account name** and [**Storage account access key**](https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage)
+- **Storage account name** and [**Storage account SAS Token**](https://docs.microsoft.com/en-us/azure/storage/blobs/sas-service-create).
+
+The credentials need to be stored inside a Kubernetes Secret, adding data entries only when
+needed. The following command performs that:
+
+```
+kubectl create secret generic azure-creds \
+  --from-literal=AZURE_STORAGE_ACCOUNT=<storage account name> \
+  --from-literal=AZURE_STORAGE_KEY=<storage account key> \
+  --from-literal=AZURE_STORAGE_SAS_TOKEN=<SAS token> \
+  --from-literal=AZURE_STORAGE_CONNECTION_STRING=<connection string>
+```
+
+The credentials will be encrypted at rest, if this feature is enabled in the used
+Kubernetes cluster.
+
+Given the previous secret, the provided credentials can be injected inside the cluster
+configuration:
+
+```yaml
+apiVersion: postgresql.k8s.enterprisedb.io/v1
+kind: Cluster
+[...]
+spec:
+  backup:
+    barmanObjectStore:
+      destinationPath: "<destination path here>"
+      azureCredentials:
+        connectionString:
+          name: azurite
+          key: AZURE_CONNECTION_STRING
+        storageAccount:
+          name: azurite
+          key: AZURE_STORAGE_ACCOUNT
+        storageKey:
+          name: azurite
+          key: AZURE_STORAGE_KEY
+        storageSasToken:
+          name: azurite
+          key: AZURE_STORAGE_SAS_TOKEN
+```
+
+When using the Azure Blob Storage, the `destinationPath` fulfills the following
+structure:
+
+```
+<http|https>://<account-name>.<service-name>.core.windows.net/<resource-path>
+```
+
+where `<resource-path>` is `<container>/<blob>`. The **account name**,
+which is also called **storage account name**, is included in the used host name.
+
+### Other Azure Blob Storage compatible providers
+
+If you are using a different implementation of the Azure Blob Storage APIs,
+the `destinationPath` will have the following structure:
+
+```
+<http|https>://<local-machine-address>:<port>/<account-name>/<resource-path>
+```
+
+In that case, `<account-name>` is the first component of the path.
+
+This is required if you are testing the Azure support via the Azure Storage
+Emulator or [Azurite](https://github.com/Azure/Azurite).
+
 ## On-demand backups
 
 To request a new backup, you need to create a new Backup resource
@@ -334,6 +415,12 @@ spec:
 
 The proposed specification will schedule a backup every day at midnight.
 
+ScheduledBackups can be suspended if needed by setting `.spec.suspend: true`,
+this will stop any new backup to be scheduled as long as the option is set to false.
+
+In case you want to issue a backup as soon as the ScheduledBackup resource is created
+you can set `.spec.immediate: true`.
+
 ## WAL archiving
 
 WAL archiving is enabled as soon as you choose a destination path
@@ -436,4 +523,4 @@ manager running in the Pods.
 You can optionally specify a `recoveryTarget` to perform a point in time
 recovery. If left unspecified, the recovery will continue up to the latest
 available WAL on the default target timeline (`current` for PostgreSQL up to
-11, `latest` for version 12 and above).
+11, `latest` for version 12 and above).

advocacy_docs/kubernetes/cloud_native_postgresql/before_you_start.mdx

@@ -35,6 +35,9 @@ Cloud Native PostgreSQL requires Kubernetes 1.16 or higher.
 | Replica | A PostgreSQL instance replicating from the only primary instance in a cluster and is kept updated by reading a stream of Write-Ahead Log (WAL) records. A replica is also known as *standby* or *secondary* server. PostgreSQL relies on physical streaming replication (async/sync) and file-based log shipping (async). |
 | Hot Standby | PostgreSQL feature that allows a *replica* to accept read-only workloads. |
 | Cluster | To be intended as High Availability (HA) Cluster: a set of PostgreSQL instances made up by a single primary and an optional arbitrary number of replicas. |
+| Replica Cluster | A Cloud Native PostgreSQL `Cluster` that is in continuous recovery mode from a selected PostgreSQL cluster, normally residing outside the Kubernetes cluster. It is a feature that enables multi-cluster deployments in private, public, hybrid, and multi-cloud contexts. |
+| Designated Primary | A PostgreSQL standby instance in a replica cluster that is in continuous recovery from another PostgreSQL cluster and that is designated to become primary in case the replica cluster becomes primary. |
+| Superuser | In PostgreSQL a *superuser* is any role with both `LOGIN` and `SUPERUSER` privileges. For security reasons, Cloud Native PostgreSQL performs administrative tasks by connecting to the `postgres` database as the `postgres` user via `peer` authentication over the local Unix Domain Socket. |
 
 ## Cloud terminology
 
@@ -46,4 +49,4 @@ Cloud Native PostgreSQL requires Kubernetes 1.16 or higher.
 ## What to do next
 
 Now that you have familiarized with the terminology, you can decide to
-[test Cloud Native PostgreSQL on your laptop using a local cluster](quickstart.md) before deploying the operator in your selected cloud environment.
+[test Cloud Native PostgreSQL on your laptop using a local cluster](quickstart.md) before deploying the operator in your selected cloud environment.