Deploying CloudNativePG on Civo Kubernetes

PostgreSQL is a powerful, open-source relational database that has been a cornerstone for modern applications across industries. With the rise of cloud-native architectures, organizations are increasingly seeking ways to run databases like PostgreSQL on Kubernetes, leveraging the scalability, portability, and automation provided by container orchestration platforms. However, while Kubernetes is well-suited for stateless applications, deploying and managing stateful services like databases comes with unique challenges.

This tutorial will guide you through deploying and managing a PostgreSQL cluster on the Civo Kubernetes service using the CloudNativePG operator. By adopting this approach, you can take advantage of Kubernetes’ ecosystem to simplify database lifecycle management, achieve high availability, and ensure data persistence in a dynamic, scalable environment.

Introduction to PostgreSQL on Kubernetes

Running PostgreSQL on Kubernetes is an increasingly popular choice for organizations aiming to combine the scalability and flexibility of container orchestration with robust database management. Kubernetes excels at handling stateless applications, but managing stateful services like databases introduces unique complexities that require thoughtful solutions.

Deploying PostgreSQL on Kubernetes involves addressing critical aspects such as persistent storage, high availability, and data integrity. Key Kubernetes resources like StatefulSets are essential for managing the lifecycle of PostgreSQL pods, ensuring each pod retains a stable, unique identity and access to persistent storage even after restarts.

Persistent Volumes (PVs) ensure data durability, while ConfigMaps and Secrets simplify managing database configurations and securely storing sensitive information like credentials. These tools collectively enhance the reliability, security, and manageability of PostgreSQL in a Kubernetes environment, making it a compelling solution for modern cloud-native applications.

Why use operators for PostgreSQL?

While Kubernetes provides foundational building blocks for deploying stateful applications, managing databases like PostgreSQL involves much more than simply running pods and attaching storage. Ensuring high availability, automating backups, handling failovers, and maintaining data integrity during scaling or infrastructure changes requires specialized expertise and tools.

This is where database operators, like CloudNativePG, come into play. Operators extend Kubernetes functionality to automate complex operational tasks that are otherwise manual and error-prone. By translating operational knowledge into software, operators reduce the burden on DevOps teams and allow them to focus on higher-value tasks.

Deployment of CloudNativePG

Prerequisites

While this tutorial assumes basic familiarity with Kubernetes concepts such as Pods, Services, and ConfigMaps, you will need the following to complete this tutorial:

• A Civo account
• Kubectl installed (to manage namespaces and secrets on the cluster)
• Create a kubernetes cluster

Installing the CloudNativePG Operator

CloudNativePG can be installed using Helm or by directly applying the YAML manifests.

1kubectl apply --server-side -f https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.24/releases/cnpg-1.24.1.yaml

Check if the operator is running correctly:

1kubectl get pods -n cnpg-system
2
3NAME                                       READY   STATUS    RESTARTS   AGE
4cnpg-controller-manager-7fd67898bb-2lc5x   1/1     Running   1          53s

Deploying a PostgreSQL cluster

A cluster in CloudNativePG is a set of PostgreSQL instances managed in Kubernetes, providing high availability and automatic replication. To install it, you apply the operator's manifests and create a PostgreSQLCluster resource using a YAML file, which configures and deploys the cluster automatically.

Create a file named pgcluster.yaml and add the following content:

1apiVersion: postgresql.cnpg.io/v1
2kind: Cluster
3metadata:
4  name: civo-postgres-cluster
5spec:
6  instances: 3  # number of replicas
7  storage:
8    size: 1Gi

Apply this manifest to deploy the cluster:

1kubectl apply -f pgcluster.yaml 
2
3cluster.postgresql.cnpg.io/civo-postgres-cluster created

After installing the PostgreSQL cluster using CloudNativePG, you can see the instances distributed as pods within your Kubernetes cluster. Each PostgreSQL instance runs in its own pod, and you can check them using the kubectl get pods command to view the status and distribution of the database instances across your cluster.

1kubectl get pods
2
3NAME                                 READY   STATUS      RESTARTS   AGE
4install-cert-manager-cloudst-qg82f   0/1     Completed   0          20h
5civo-postgres-cluster-1              1/1     Running     0          3m28s
6civo-postgres-cluster-2              1/1     Running     0          114s
7civo-postgres-cluster-3              1/1     Running     0          48s

If you want to configure the cluster during boot initialization, you can do so using the BootstrapConfiguration. This allows you to set custom configurations at the time of cluster creation, such as defining users and databases and applying specific initialization scripts.

For each instance created by the CloudNativePG, a Persistent Volume Claim (PVC) is generated in Kubernetes. In our demo, the volumes are securely stored in Civo Cloud. To view the created volumes, simply log into your Civo account, click on the "Volumes" option, and you'll see them listed there.

Connecting to the PostgreSQL cluster

Now, it's time to connect to the database, which we can do through one of the services created by the cluster operator. Since I didn't specify a username and password for my cluster, the operator generated these credentials automatically and stored them in a Kubernetes Secret.

Let's take a look at the Secrets to retrieve the connection details:

1kubectl get secrets
2
3NAME                              TYPE                     DATA   AGE
4civo-postgres-cluster-ca          Opaque                   2      16m
5civo-postgres-cluster-server      kubernetes.io/tls        2      16m
6civo-postgres-cluster-replication kubernetes.io/tls        2      16m
7civo-postgres-cluster-app         kubernetes.io/basic-auth 9      16m

To retrieve the username and password that the operator created for you, run the following commands:

1#Get the password
2kubectl get secret civo-postgres-cluster-app \ -o=jsonpath='{.data.password}' | base64 -d
3
4#Get the username
5kubectl get secret civo-postgres-cluster-app \ -o=jsonpath='{.data.username}' | base64 -d

Expose the PostgreSQL service so that you can connect from your local machine:

1kubectl port-forward svc/civo-postgres-cluster-rw 5432:5432

Connect via psql or pgadmin using the username and password obtained above. In my case, the username was “app” and the password was “ANe9uuSy1LFPnR0LqOFYziiHL4nJO6QnKI2c0l5hzDwGr3bupzuGrIxDNq2dAUob”. To log in using these credentials, run the following command:

1psql -h localhost -p 5432 -U app

You will be asked to enter the password, and after that, you will be ready to use your database:

1psql (14.13 (Ubuntu 14.13-0ubuntu0.22.04.1), server 17.0 (Debian 17.0-1.pgdg110+1))
2WARNING: psql major version 14, server major version 17.
3         Some psql features might not work.
4SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
5Type "help" for help.
6
7app=> 

It's the perfect time to dive into CloudNativePG! If you're eager to explore advanced features such as automated backups, scaling, and monitoring, check out some of these resources:

• PostgreSQL Docs on Civo
• Beginners guide to PersistentVolume and PersistentVolumeClaim
• Creating a Kubernetes Operator with Kubebuilder
• Tutorials on Backups