Architecture v1

Consider these main architectural aspects when deploying EDB Postgres Distributed in Kubernetes.

EDB Postgres Distributed for Kubernetes is a Kubernetes operator designed to deploy and manage EDB Postgres Distributed clusters running in private, public, hybrid, or multi-cloud environments.

Relationship with EDB Postgres Distributed

EDB Postgres Distributed (PGD) is a multi-master implementation of Postgres designed for high performance and availability. PGD generally requires deployment using Trusted Postgres Architect (TPA), a tool that uses Ansible to provision and deploy PGD clusters.

EDB Postgres Distributed for Kubernetes offers a different way of deploying PGD clusters, leveraging containers and Kubernetes. The advantages are that the resulting architecture:

• Is self-healing and robust.
• Is managed through declarative configuration.
• Takes advantage of the vast and growing Kubernetes ecosystem.

Relationship with EDB Postgres for Kubernetes

A PGD cluster consists of one or more PGD groups, each having one or more PGD nodes. A PGD node is a Postgres database. EDB Postgres Distributed for Kubernetes internally manages each PGD node using the Cluster resource as defined by EDB Postgres for Kubernetes, specifically a cluster with a single instance (that is, no replicas).

You can configure the single PostgreSQL instance created by each Cluster in the .spec.cnp section of the PGD Group spec.

In EDB Postgres Distributed for Kubernetes, as in EDB Postgres for Kubernetes, the underlying database implementation is responsible for data replication. However, it's important to note that failover and switchover work differently, entailing Raft election and nominating new write leaders. EDB Postgres for Kubernetes handles only the deployment and healing of data nodes.

Managing PGD using EDB Postgres Distributed for Kubernetes

The EDB Postgres Distributed for Kubernetes operator can manage the complete lifecycle of PGD clusters. As such, in addition to PGD nodes (represented as single-instance Clusters), it needs to manage other objects associated with PGD.

PGD relies on the Raft algorithm for distributed consensus to manage node metadata, specifically agreement on a write leader. Consensus among data nodes is also required for operations such as generating new global sequences or performing distributed DDL.

These considerations force additional actors in PGD above database nodes.

EDB Postgres Distributed for Kubernetes manages the following:

• Data nodes. A node is a database and is managed by EDB Postgres for Kubernetes, creating a Cluster with a single instance.
• Witness nodes are basic database instances that don't participate in data replication. Their function is to guarantee that consensus is possible in groups with an even number of data nodes or after network partitions. Witness nodes are also managed using a single-instance Cluster resource.
• PGD proxies act as Postgres proxies with knowledge of the write leader. PGD proxies need information from Raft to route writes to the current write leader.

Proxies and routing

PGD groups assume full mesh connectivity of PGD nodes. Each node must be able to connect to every other node using the appropriate connection string (a libpq-style DSN). Write operations don't need to be sent to every node. PGD takes care of replicating data after it's committed to one node.

For performance, we often recommend sending write operations mostly to a single node, the write leader. Raft is used to identify which node is the write leader and to hold metadata about the PGD nodes. PGD proxies are used to transparently route writes to write leaders and to quickly pivot to the new write leader in case of switchover or failover.

It's possible to configure Raft subgroups, each of which can maintain a separate write leader. In EDB Postgres Distributed for Kubernetes, a PGD group containing a PGD proxy comprises a Raft subgroup.

Two kinds of routing are available with PGD proxies:

• Global routing uses the top-level Raft group and maintains one global write leader.
• Local routing uses subgroups to maintain separate write leaders. Local routing is often used to achieve geographical separation of writes.

In EDB Postgres Distributed for Kubernetes, local routing is used by default, and a configuration option is available to select global routing.

For more information, see the PGD documentation of routing with Raft.

PGD architectures and high availability

EDB proposes several recommended architectures to make good use of PGD's distributed multi-master capabilities and to offer high availability.

The Always On architectures are built from either one group in a single location or two groups in two separate locations. See Choosing your architecture in the PGD documentation for more information.

Deploying PGD on Kubernetes

EDB Postgres Distributed for Kubernetes leverages Kubernetes to deploy and manage PGD clusters. As such, some adaptations are necessary to translate PGD into the Kubernetes ecosystem.

Images and operands

You can configure PGD to run one of three Postgres distributions. See the PGD documentation to understand the features of each distribution.

To function in Kubernetes, containers are provided for each Postgres distribution. These are the operands. In addition, the operator images are kept in those same repositories.

See EDB private image registries for details on accessing the images.

Kubernetes architecture

Some of the points of the PG4K document on Kubernetes architecture are reproduced here. See the PG4K documentation for details.

Kubernetes natively provides the possibility to span separate physical locations. These physical locations are also known as data centers, failure zones, or, more frequently, availability zones. They are connected to each other by way of redundant, low-latency, private network connectivity.

Being a distributed system, the recommended minimum number of availability zones for a Kubernetes cluster is three. This minimum makes the control plane resilient to the failure of a single zone. This means that each data center is active at any time and can run workloads simultaneously.

You can install EDB Postgres Distributed for Kubernetes in a single Kubernetes cluster or across multiple Kubernetes clusters.

Single Kubernetes cluster

A multi-availability-zone Kubernetes architecture is typical of Kubernetes services managed by cloud providers. Such an architecture enables the EDB Postgres Distributed for Kubernetes and the EDB Postgres for Kubernetes operators to schedule workloads and nodes across availability zones, considering all zones active.

PGD clusters can be deployed in a single Kubernetes cluster and take advantage of Kubernetes availability zones to enable high-availability architectures, including the Always On recommended architectures.

You can realize the Always On Single Location architecture shown in Choosing your architecture in the PGD documentation on a single Kubernetes cluster with three availability zones.

The EDB Postgres Distributed for Kubernetes operator can control the scheduling of pods (that is, which pods go to which data center) using affinity, tolerations, and node selectors, as is the case with EDB Postgres for Kubernetes. Individual scheduling controls are available for proxies as well as nodes.

See the Kubernetes documentation on scheduling, and Scheduling in the EDB Postgres for Kubernetes documentation for more information.

Multiple Kubernetes clusters

PGD clusters can also be deployed in multiple Kubernetes clusters that can reliably communicate with each other.

Always On multi-location PGD architectures can be realized on multiple Kubernetes clusters that meet the connectivity requirements.

For more information, see "Connectivity".

• On this page
• Relationship with EDB Postgres Distributed
• Relationship with EDB Postgres for Kubernetes
• Managing PGD using EDB Postgres Distributed for Kubernetes
• Deploying PGD on Kubernetes