Hardening your infrastructure as code pipeline with self-hosted OpenTaco (formerly Digger)

In part 1 of our tutorial series, we set up a simple Digger pipeline to perform infrastructure as code (IaC) deployments, all within a GitHub pipeline. But what if we wanted to expand such a deployment for a larger installation or want it hardened for production deployments? We will look at deploying the orchestrator backend.

Utilizing the orchestrator backend will allow for efficient management of CI pipelines, especially for teams working on multiple PRs simultaneously. For this reason, this tutorial will dive into configuring and deploying the Digger orchestrator backend, setting up secure webhooks using cert-manager, and integrating it with GitHub to manage larger infrastructure environments efficiently.

What is the orchestrator backend?

The orchestrator backend is a service that triggers Pipeline Runs. Events within the source code management system usually trigger most CI pipeline runs, code commits, pull requests, and other internal activity. Most CI systems also provide a way for CI activities to be triggered externally. This is what the orchestrator uses to help build a configuration that will work more efficiently with multiple users and larger environments.

Some of the other core points include:

• Quicker response to PR comments and faster status check updates
• Parallelization where appropriate
• Appropriate queuing with multiple PRs
• PR level locks

In OpenTaco the orchestrator is no longer the whole story. It is the piece that talks to GitHub, but it now sits alongside services for state management, drift detection, token issuance, and remote runs.

The OpenTaco architecture

Before we install anything, here is what the umbrella chart actually deploys, so the moving parts are not a surprise.

• UI: The public entrypoint. It serves the dashboard and proxies the backend APIs.
• Orchestrator: Processes GitHub webhooks and creates Terraform or OpenTofu workflow runs. This is the former digger-backend.
• Drift: Runs drift detection and sends notifications.
• Statesman: A Terraform Cloud compatible state backend, backed by S3-compatible object storage.
• Token service: Issues API tokens for Statesman authentication.
• Sidecar: Manages remote run sandboxes. Optional, and we leave it off in this guide.

Three of these services need a database, and Statesman needs object storage. The platform reference chart below provides both, so you have a working baseline to build on.

Two third-party dependencies are worth knowing up front. OpenTaco uses WorkOS for authentication, so you will need a WorkOS account to log in. It also uses GitHub for PR automation, which we wire up near the end.

Prerequisites

• A verified Civo account
• The Civo CLI installed and authenticated
• kubectl and Helm installed
• A WorkOS account for authentication
• A GitHub organization or account where the app will be installed

Creating a Civo Kubernetes cluster

We will provision a small cluster from the command line. Two medium nodes are enough for the whole platform:

1civo kubernetes create taco-demo \
2  --size g4s.kube.medium \
3  --nodes 2 \
4  --create-firewall \
5  --wait --save --switch

The --save --switch flags merge the new kubeconfig and make it your active context. Confirm the nodes are ready:

1kubectl get nodes

Every Civo cluster gets a DNS name of the form <cluster-id>.k8s.civo.com. We use it as the public URL for OpenTaco, so record it now:

1civo kubernetes show taco-demo
2# Note the "DNS A record" value, then:
3export HOST=<your-cluster-id>.k8s.civo.com

Installing the platform baseline

OpenTaco ships an optional platform reference chart that provisions Traefik for ingress, MinIO for object storage, and a CloudNativePG PostgreSQL cluster with the three databases the services need. Treat it as a reference baseline, not a production-hardening blueprint. For a real deployment you would bring your own managed database and object storage, and on Civo you could point Statesman at Civo object storage instead of MinIO.

Create the namespaces:

1kubectl create namespace opentaco
2kubectl create namespace traefik

Install the platform reference chart in two steps. The chart bundles the CloudNativePG operator and a PostgreSQL cluster resource in one release, so a single install fails on CRD ordering. Install with the database disabled first, let the operator come up, then enable it:

1cat > platform-values.yaml <<'EOF'
2cnpg:
3  nameOverride: cloudnative-pg
4postgresql:
5  enabled: false
6EOF
7
8
9helm upgrade --install opentaco-platform-reference \
10  oci://ghcr.io/diggerhq/helm-charts/opentaco-platform-reference \
11  --version 0.1.0 -n opentaco -f platform-values.yaml
12
13
14kubectl rollout status deployment/opentaco-platform-reference-cloudnative-pg \
15  -n opentaco --timeout=180s
16
17
18sed -i '' 's/enabled: false/enabled: true/' platform-values.yaml
19helm upgrade opentaco-platform-reference \
20  oci://ghcr.io/diggerhq/helm-charts/opentaco-platform-reference \
21  --version 0.1.0 -n opentaco -f platform-values.yaml

Wait for the PostgreSQL cluster and MinIO to report ready:

1kubectl get pods -n opentaco
2kubectl get cluster.postgresql.cnpg.io -n opentaco

Setting up cert-manager

As we will expose the UI on a public address over HTTPS, we need a signed TLS certificate. To configure one, leverage the cert-manager Civo Marketplace app. The core cert-manager is installed, but we still need to configure it:

1civo kubernetes applications add cert-manager --cluster taco-demo

The following will set up a cluster-issuer to allow us to continue:

1kubectl apply -f - <<EOF
2apiVersion: cert-manager.io/v1
3kind: ClusterIssuer
4metadata:
5  name: letsencrypt-prod
6spec:
7  acme:
8    email: your-email@example.com
9    preferredChain: ""
10    privateKeySecretRef:
11      name: prod-letsencrypt-key
12    server: https://acme-v02.api.letsencrypt.org/directory
13    solvers:
14      - selector: {}
15        http01:
16          ingress:
17            class: traefik
18EOF

Configuring WorkOS authentication

WorkOS must be configured before anyone can sign in. In the WorkOS dashboard, enable AuthKit, then set the following against your application, substituting your cluster host:

• Redirect URI: https://${HOST}/api/auth/callback
• Sign-out redirect: https://${HOST}/logout
• CORS allowed origin: https://${HOST}
• A webhook for the user.created event pointing at https://${HOST}/api/auth/workos/webhooks

Then collect three values you will need for the secrets below: the Client ID, an API key, and the webhook signing secret.

Creating the application secrets

OpenTaco reads its configuration from Kubernetes Secrets. Rather than putting values inline in the chart, we pre-create the secrets and reference them, which keeps credentials out of your values file and works cleanly with tools like the External Secrets Operator later.

Generate the shared internal secrets once and create the UI, orchestrator, statesman, and drift secrets. The stringData fields below take plain text and the API server encodes them:

1INTERNAL_SECRET=$(openssl rand -base64 36)
2ENCRYPTION_SECRET=$(openssl rand -base64 24)
3STATE_KEY=$(openssl rand -base64 36)
4
5
6kubectl apply -f - <<EOF
7apiVersion: v1
8kind: Secret
9metadata: { name: ui-secrets, namespace: opentaco }
10type: Opaque
11stringData:
12  WORKOS_CLIENT_ID: "<your-workos-client-id>"
13  WORKOS_API_KEY: "<your-workos-api-key>"
14  WORKOS_WEBHOOK_SECRET: "<your-workos-webhook-secret>"
15  WORKOS_COOKIE_PASSWORD: "$(openssl rand -base64 24)"
16  WORKOS_REDIRECT_URI: https://${HOST}/api/auth/callback
17  PUBLIC_URL: https://${HOST}
18  ALLOWED_HOSTS: ${HOST},localhost
19  ORCHESTRATOR_BACKEND_URL: http://opentaco-taco-orchestrator-web:3000
20  ORCHESTRATOR_BACKEND_SECRET: ${INTERNAL_SECRET}
21  STATESMAN_BACKEND_URL: http://opentaco-taco-statesman:8080
22  TOKENS_SERVICE_BACKEND_URL: http://opentaco-taco-token-service:8081
23  DRIFT_REPORTING_BACKEND_URL: http://opentaco-taco-drift:3004
24EOF

Create three more secrets the same way, named taco-orchestrator-secrets, statesman-secrets, and drift-secrets, wiring each service to its database in the shared PostgreSQL cluster. The full set of variables is listed in the OpenTaco configuration docs. One consistency rule matters most: the orchestrator's DIGGER_INTERNAL_SECRET must equal the ORCHESTRATOR_BACKEND_SECRET we just set in ui-secrets, which is why we generated INTERNAL_SECRET once and reuse it.

Installing OpenTaco

With the baseline and secrets in place, install the umbrella chart. We start from the platform-reference values, which pre-wire every service to the in-cluster PostgreSQL and MinIO, then layer our own overrides on top. Helm deep-merges values files in the order they are passed, so the two files stay cleanly separated.

Fetch the platform-reference values from the OpenTaco repo:

1curl -sO https://raw.githubusercontent.com/diggerhq/digger/develop/self-hosting/kubernetes/helm-charts/opentaco/values.platform-reference.yaml

Then create values-opentaco.yaml, which points each service at its pre-created secret, sets the public URL, and enables the UI ingress:

1taco-orchestrator:
2  digger:
3    secret:
4      useExistingSecret: true
5      existingSecretName: taco-orchestrator-secrets
6
7
8taco-statesman:
9  taco:
10    publicBaseUrl: https://<your-cluster-id>.k8s.civo.com
11    secretKey: <generate with: openssl rand -base64 36>
12    secret:
13      useExistingSecret: true
14      existingSecretName: statesman-secrets
15
16
17taco-drift:
18  drift:
19    secret:
20      useExistingSecret: true
21      existingSecretName: drift-secrets
22
23
24taco-sidecar:
25  enabled: false
26
27
28taco-ui:
29  ui:
30    secret:
31      useExistingSecret: true
32      existingSecretName: ui-secrets
33    ingress:
34      enabled: true
35      className: traefik
36      annotations:
37        cert-manager.io/cluster-issuer: letsencrypt-prod
38      hosts:
39        - host: <your-cluster-id>.k8s.civo.com
40          paths:
41            - path: /
42              pathType: Prefix
43      tls:
44        - secretName: opentaco-ui-tls
45          hosts:
46            - <your-cluster-id>.k8s.civo.com

Install:

1helm upgrade --install opentaco \
2  oci://ghcr.io/diggerhq/helm-charts/opentaco \
3  --version 0.1.2-public \
4  -f values.platform-reference.yaml \
5  -f values-opentaco.yaml \
6  -n opentaco

Watch the pods settle:

1kubectl get pods -n opentaco

Once all pods are running, the cert-manager certificate is issued and the UI is reachable over HTTPS:

1curl -sI "https://${HOST}/"

Open https://${HOST} in a browser and you are met with the OpenTaco sign-in screen.

The OpenTaco landing and sign-in page, served over HTTPS on the Civo cluster.

Sign up the first user through the WorkOS-backed login, and you land on the dashboard.

The OpenTaco dashboard after signing in, with the Units, Repos, Projects, and Drift sections in the sidebar.

Configuring GitHub integration

The orchestrator processes GitHub webhooks, so it needs a GitHub App whose events point at your cluster. A self-hosted deployment creates its own app rather than installing the hosted one, so that the app's webhooks reach your orchestrator and its credentials belong to you.

The intended flow is to open the repository setup wizard in the dashboard, click "Connect with GitHub", and complete the GitHub App creation.

The repository setup wizard: connect GitHub, add the workflow, add digger.yml, then Terraform.

You then copy the five values the app produces into your orchestrator and drift secrets:

• GITHUB_APP_ID
• GITHUB_APP_CLIENT_ID
• GITHUB_APP_CLIENT_SECRET
• GITHUB_APP_PRIVATE_KEY_BASE64
• GITHUB_WEBHOOK_SECRET

After saving them, restart the orchestrator and drift services so the new credentials load, then install the app on the repositories you want Digger to manage.

Running Digger in the pipeline

Once the GitHub App is connected, configure Digger in the pipeline to use your self-hosted orchestrator. It is the same as in part 1, with one parameter added for the orchestrator URL:

1- name: digger run
2  uses: diggerhq/digger@latest
3  with:
4    digger-hostname: https://<your-cluster-id>.k8s.civo.com

Cleaning up

A running cluster costs money, so remove it when you are done:

1civo kubernetes remove taco-demo

If this was a throwaway run, also tidy the external accounts: delete the GitHub App from your organization settings and remove the WorkOS application or environment you configured.

Summary

You have now self-hosted OpenTaco, the platform Digger became. We deployed the orchestrator and its companion services with the umbrella chart, gave them a database and object storage with the platform reference chart, secured the UI with cert-manager and a Let's Encrypt certificate, and wired authentication through WorkOS.

Now that the platform is running, you can explore options for scaling and hardening this setup:

• The database and object storage here run inside Kubernetes as a reference baseline. For critical environments, move PostgreSQL and state storage onto managed, backed-up infrastructure. On Civo, Statesman can use Civo object storage for state.
• Keep all secrets in a secret manager rather than inline values, and treat the shared internal secrets as a single rotation unit.