The Crunchy PostgreSQL Operator 4.0 provides an open source PostgreSQL-as-a-Service for Kubernetes platform.
I had previously explored how to build an active-active PostgreSQL cluster using Symmetric-DS. The steps are essentially the same on Kubernetes:
- Start 2 PostgreSQL pods
- Create a user and a database on each pod/instance
- Start the primary symmetric-ds pod
- Add the symmetric-ds triggers and routes
- Open registration
- Start the secondary symmertic-ds pod
However, there are some interesting nuances I discovered while building out this setup, which I will discuss below. Before diving into the details, I'd also like to credit my colleagues Patrick McLaughlin and Simon Nielly as co-authors for helping me to put together this guide.
Anyway, let's explore how to deploy an active-active PostgreSQL cluster on Kubernetes.
An important part of running a production PostgreSQL database system (and for that matter, any database software) is to ensure you are prepared for disaster. There are many ways to go about preparing your system for disaster, but one of the simplest and most effective ways to do this is by taking periodic backups of your database clusters.
How does one typically go about setting up taking a periodic backup? If you’re running PostgreSQL on a Linux based system, the solution is to often use cron, and setting up a crontab entry similar to this in your superuser account:
# take a daily base backup at 1am to a mount point on an external disk # using pg_basebackup 0 1 * * * /usr/bin/env pg_basebackup –D /your/external/mount/
However, if you’re managing tens, if not hundreds and thousands of PostgreSQL databases, this very quickly becomes an onerous task and you will need some automation to help you scale your disaster recovery safely and efficiently.
Automating Periodic Backups
The Crunchy PostgreSQL Operator, an application for managing PostgreSQL databases in a Kubernetes-based environment in is designed for managing thousands of PostgreSQL database from a single interface to help with challenges like the above. One of the key features of the PostgreSQL Operator is to utilize Kubernetes Labels to apply commands across many PostgreSQL databases. Later in this article, we will see how we can take advantage of labels in order to set backup policies across many clusters.
Interested in running PostgreSQL natively on Kubernetes? Let's look at a few quick steps to get up and running with the open source Crunchy PostgreSQL Operator for Kubernetes on your choice of Kubernetes deployment.
The Crunchy PostgreSQL Operator (aka "pgo") provides a quickstart script to automate the deployment of the Crunchy PostgreSQL Operator to a number of popular Kubernetes environments, including Google Kubernetes Engine (GKE), OpenShift Container Platform (OCP) and Pivotal Container Service (PKS). You can download the quickstart script here:
The script will deploy the operator to a GKE Kubernetes cluster or an OpenShift Container Platform cluster. The script assumes you have a StorageClass defined for persistence.
Running PostgreSQL databases in containerized environments is more popular than ever and is moving beyond running only in local, development environments and into large scale production environments. To answer the need to orchestrate complex, database workloads, the Crunchy Data team created the PostgreSQL Operator to automate many typical database administrator tasks at scale:
- Provisioning new PostgreSQL clusters
- Scaling up replicas
- Setup and manage disaster recovery, high-availability, and monitoring
- Allocate resources (memory, CPU, etc.) and suggest nodes for databases to run on
- Mass applying user policies
- Performing major/minor upgrades
and more. In order to keep all of its PostgreSQL databases up and running, the PostgreSQL Operator uses Kubernetes Deployments, which provides an API to manage replicated applications. In order to understand why this is, first we need to understand about running stateful applications with Kubernetes.
Crunchy Data recently released version 2.6 of the PostgreSQL Kubernetes Operator, a powerful controller that follows the Kubernetes Operator pattern that enables users to provision and manage thousands of PostgreSQL databases at scale. We thought that we would demonstrate some of the features in this latest version over the next few weeks, which includes support for manual database failover, storage selection, node (or server) selection preference, and many goodies that make it easier to manage a large PostgreSQL cluster.
Crunchy Data is pleased to announce an initial implemention of a PostgreSQL Operator for Kubernetes to build on our work with PostgreSQL Containers. This initial implementation provides a user with the ability to perform certain PostgreSQL functions including creating PostgreSQL clusters, performing database backup and restores and viewing persistent volume claims.
This blog provides guidance on how to build a PostgreSQL cluster using the new Kubernetes feature - StatefulSet. Using this StatefulSet capability provides a very simple, Kubernetes native, mechanism to make clustering decisions when deploying a PostgreSQL cluster.