Persistent LangGraph Agent Loop on Civo Kubernetes with Managed PostgreSQL

When a Kubernetes pod restarts, everything stored in memory disappears. For a simple chatbot, that is usually fine. But for an agent running a multi-step research task, it means losing every finding, tool result, and decision made along the way. The task has to start from scratch.

Most agent demos avoid this issue because they are stateless. They process a single request, return a response, and exit. There is no long-running workflow, accumulated context, or state to recover. In production, that quickly becomes a limitation.

A common solution is to run a database inside the cluster, but that adds the overhead of managing infrastructure before you even start building the agent. Another option is storing state in files or object storage, which means writing custom serialization and recovery logic.

A better approach is automatic checkpointing. After every step, the agent saves its state to external storage and can resume from the latest checkpoint if the pod restarts. This tutorial shows you how to build that with LangGraph, Civo Managed PostgreSQL, and relaxAI.

What you'll build

The architecture consists of a LangGraph research agent running as a Kubernetes Deployment on Civo, with Civo Managed PostgreSQL providing persistent state.

The agent starts with a high-level research prompt and runs autonomously. It searches the web, fetches and reads relevant pages, saves its progress to PostgreSQL after every step through LangGraph checkpointing, and decides when it has gathered enough information to generate a final report.

Every action is streamed to the pod logs, making it easy to follow the agent's progress. If the pod restarts at any point, LangGraph restores the latest checkpoint from PostgreSQL, allowing the workflow to continue without repeating completed steps.

The stack includes:

• A Civo Kubernetes cluster running the agent as a single Deployment on a CPU node
• Civo Managed PostgreSQL for persistent agent state
• A LangGraph agent with three tools: web search, URL fetch, and report generation
• relaxAI providing LLM reasoning through a standard chat completions API 

Once deployed, the only manual step is tailing the pod logs to watch the agent work.

Prerequisites

Before you begin, make sure you have the required accounts and tools installed.

Accounts

• A Civo account with access to Kubernetes and Managed PostgreSQL
• A relaxAI API key

Local tools

• Civo CLI for provisioning the cluster and database
• kubectl for communicating with the cluster
• Docker for building and pushing the container image
• Python 3.11 for running the agent locally before containerizing

Tested versions

Project structure

1langgraph-civo/
2├── agent/
3│   ├── agent.py            # LangGraph agent definition
4│   ├── tools.py            # tool definitions
5│   └── requirements.txt    # Python dependencies
6├── Dockerfile              # container image
7└── kubernetes/
8    ├── namespace.yaml      # namespace for all resources
9    └── deployment.yaml     # agent Deployment manifest

Once all tools are installed and both API keys are confirmed, you have everything needed to follow this tutorial from start to finish.

How it fits together

Before starting the deployment, it is useful to understand the role of each component and how they work together. The setup is divided into four main parts, with each handling a specific responsibility.

The Civo Kubernetes cluster runs the agent as a single Deployment on a CPU node. Kubernetes restarts the pod if it fails, but the agent does not lose its progress because its state is stored outside the cluster.

Inside the pod, LangGraph manages the agent loop. It decides which tool to call, processes the result, and checks whether the task is complete. After each step, it saves the current state to PostgreSQL via LangGraph checkpointing. If the pod restarts, LangGraph loads the latest checkpoint and resumes from where it stopped.

Civo Managed PostgreSQL stores the agent checkpoints. It runs outside the cluster, requires no in-cluster database deployment or persistent volumes, and is accessed through a standard PostgreSQL connection string.

relaxAI handles all LLM requests through an OpenAI-compatible API. The agent sends prompts to relaxAI and receives the next action without running a model inside the cluster.

The workflow is simple:

1. The agent starts and loads the latest checkpoint from PostgreSQL
2. It sends the current state to relaxAI and receives the next action
3. It executes the selected tool and saves the updated state
4. The loop repeats until the agent finishes the task and generates a final report

Each component has a single role: Kubernetes runs the workload, LangGraph manages the workflow, PostgreSQL stores the state, and relaxAI provides the reasoning.

Provisioning the Database

Before creating the cluster, create the Managed PostgreSQL instance first. The agent requires the database connection string to configure its Kubernetes secret, so the database must be available before deployment.

Authenticate the Civo CLI:

1civo apikey save my-key YOUR_CIVO_API_KEY
2civo apikey use my-key

Create the PostgreSQL instance:

1civo database create langgraph-db \
2  --size g3.db.small \
3  --software PostgreSQL \
4  --region NYC1 \
5  --wait

The --wait flag holds the terminal until the instance is fully provisioned. Expect under 5 minutes.

Database provisioned and ready

Once provisioned, retrieve the connection string:

1civo db credential langgraph-db

Copy the connection string. It looks like this:

1postgresql://civo:password@host:5432

Append /postgres?sslmode=require to get the full URL:

1postgresql://civo:password@host:5432/postgres?sslmode=require

To confirm the database is accepting connections, run: 

1civo database show langgraph-db --region NYC1 | grep Status

Save the full connection string. You'll use it later when creating the Kubernetes secret. The agent uses this connection string to read and write its checkpoints.

Creating the Cluster

With the database ready, create the Kubernetes cluster. The agent runs on a single CPU node, so no GPU resources are needed.

Create the cluster:

1civo kubernetes create langgraph-cluster \
2  --size g4s.kube.small \
3  --nodes 1 \
4  --region NYC1 \
5  --wait

If the CLI times out, run civo kubernetes ls --region NYC1 and confirm the cluster shows ACTIVE before moving on.

Save the kubeconfig so kubectl can connect to the cluster:

1civo kubernetes config langgraph-cluster --region NYC1 --save --switch

Verify kubectl is connected:

1kubectl cluster-info

Expected:

Cluster connected successfully

Verify the node is Ready:

1kubectl get nodes

Expected:

Node Ready

The node must show Ready before moving on.

Building the LangGraph Agent

The agent is split across three files: requirements.txt defines the dependencies, tools.py defines what the agent can do, and agent.py wires everything together into a LangGraph graph.

agent/requirements.txt

1langgraph
2langgraph-checkpoint-postgres
3langchain
4langchain-openai
5langchain-community
6psycopg[binary]
7python-dotenv
8requests
9beautifulsoup4
10ddgs

agent/tools.py

The agent has three tools. web_search queries DuckDuckGo and returns the top five results. fetch_url retrieves and cleans the text content of a page, returning a descriptive error message if the page is unavailable so the agent knows to search for an alternative. write_report saves the final report to report.md.

1import requests
2from bs4 import BeautifulSoup
3from ddgs import DDGS
4from langchain_core.tools import tool
5@tool
6def web_search(query: str) -> str:
7    """Search the web for a given query and return the top 5 results."""
8    with DDGS() as ddgs:
9        results = ddgs.text(query, max_results=5)
10        if not results:
11            return "No results found. Try a different query."
12        output = []
13        for i, r in enumerate(results, 1):
14            output.append(f"[{i}] Title: {r['title']}\nURL: {r['href']}\nSnippet: {r['body']}\n")
15        return "\n".join(output)
16@tool
17def fetch_url(url: str) -> str:
18    """Fetch the full content of a URL. Returns cleaned text or an error if unavailable."""
19    try:
20        response = requests.get(url, timeout=10, headers={"User-Agent": "Mozilla/5.0"})
21        if response.status_code == 404:
22            return f"ERROR_404: Page not found at {url}. Do not use this source. Search for an alternative."
23        if response.status_code != 200:
24            return f"ERROR_{response.status_code}: Failed to fetch {url}. Search for an alternative."
25        soup = BeautifulSoup(response.text, "html.parser")
26        for tag in soup(["script", "style", "nav", "footer", "header", "aside"]):
27            tag.decompose()
28        text = soup.get_text(separator="\n", strip=True)
29        lines = [line for line in text.splitlines() if len(line.strip()) > 40]
30        return "\n".join(lines)[:8000]
31    except Exception as e:
32        return f"ERROR: Failed to fetch {url}: {str(e)}. Search for an alternative."
33@tool
34def write_report(content: str) -> str:
35    """Write the final research report to report.md."""
36    with open("report.md", "w", encoding="utf-8") as f:
37        f.write(content)
38    return "Report written to report.md

agent/agent.py

The following snippets are broken into logical parts for clarity. In the actual file, they form one continuous script.

Part 1: Imports and state definition

1import os
2import uuid
3import json
4import time
5import psycopg
6from dotenv import load_dotenv
7from langchain_openai import ChatOpenAI
8from langgraph.graph import StateGraph, END
9from langgraph.checkpoint.postgres import PostgresSaver
10from langgraph.prebuilt import ToolNode
11from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage
12from typing import TypedDict, Annotated
13from langgraph.graph.message import add_messages
14from tools import web_search, fetch_url, write_report
15
16load_dotenv()
17
18class AgentState(TypedDict):
19    messages: Annotated[list, add_messages]

Part 2: Model and system prompt

The model is pointed at the relaxAI API using an OpenAI-compatible client. The system prompt enforces the research workflow in three phases: search, read, then write.

1model = ChatOpenAI(
2    base_url="https://api.relax.ai/v1",
3    api_key=os.getenv("RELAX_API_KEY"),
4    model="Nemotron-3-Super",
5    temperature=0,
6    max_tokens=4096,
7)
8
9tools = [web_search, fetch_url, write_report]
10model_with_tools = model.bind_tools(tools)
11tool_node = ToolNode(tools)
12
13SYSTEM_PROMPT = """You are an expert autonomous research agent.
14
15You MUST follow these steps in order:
161. Call web_search 3 times with different queries
172. Call fetch_url on 2 URLs from the results
183. Call write_report with a structured report
19
20Report structure:
21# [Topic] Research Report
22## Executive Summary
23## Recent Developments
24## Key Features
25## Real-World Applications
26## Limitations
27## Future Direction
28## Sources
29
30Do not call write_report before completing steps 1 and 2."""

Part 3: Helper functions

safe_trim keeps the message history within the model context window while preserving tool call pairs. is_completed queries PostgreSQL to check if this thread already finished, so the agent does not repeat work after a pod restart.

1def safe_trim(messages, max_messages=10):
2    if len(messages) <= max_messages:
3        return messages
4    trimmed = messages[-max_messages:]
5    while trimmed and isinstance(trimmed[0], ToolMessage):
6        trimmed = trimmed[1:]
7    return trimmed
8
9
10def is_completed(db_url: str, thread_id: str) -> bool:
11    try:
12        with psycopg.connect(db_url) as conn:
13            cur = conn.cursor()
14            cur.execute(
15                "SELECT checkpoint FROM checkpoints WHERE thread_id = %s ORDER BY checkpoint_id DESC LIMIT 1;",
16                (thread_id,)
17            )
18            row = cur.fetchone()
19            if row and row[0]:
20                checkpoint = json.loads(row[0]) if isinstance(row[0], str) else row[0]
21                messages = checkpoint.get("channel_values", {}).get("messages", [])
22                for msg in messages:
23                    if isinstance(msg, dict) and msg.get("name") == "write_report":
24                        return True
25        return False
26    except Exception:
27        return False

Part 4:  Graph definition

The graph has two nodes: agent and tools. The agent node calls the model and returns the next action. The tools node executes the tool call and returns the result. The graph loops between them until the agent calls write_report or the message count exceeds 30.

1def agent_node(state: AgentState):
2    trimmed = safe_trim(state["messages"])
3    messages = [SystemMessage(content=SYSTEM_PROMPT)] + trimmed
4    response = model_with_tools.invoke(messages)
5    return {"messages": [response]}
6
7
8def should_continue(state: AgentState):
9    last_message = state["messages"][-1]
10    if not last_message.tool_calls:
11        return END
12    return "tools"
13
14
15def after_tools(state: AgentState):
16    for msg in state["messages"]:
17        if hasattr(msg, "name") and msg.name == "write_report":
18            return END
19    if len(state["messages"]) > 30:
20        return END
21    return "agent"
22
23
24def build_graph(checkpointer):
25    graph = StateGraph(AgentState)
26    graph.add_node("agent", agent_node)
27    graph.add_node("tools", tool_node)
28    graph.set_entry_point("agent")
29    graph.add_conditional_edges("agent", should_continue)
30    graph.add_conditional_edges("tools", after_tools)
31    return graph.compile(checkpointer=checkpointer)

Part 5: Entry point

On startup, the agent checks PostgreSQL for an existing completed checkpoint. If found, it sleeps to keep the pod alive without repeating the work. If not found, it runs the full research loop and sleeps after completion.

1if __name__ == "__main__":
2    db_url = os.getenv("DATABASE_URL")
3    prompt = os.getenv("RESEARCH_PROMPT", "Research the latest trends in Kubernetes and cloud native computing")
4    thread_id = os.getenv("THREAD_ID", str(uuid.uuid4()))
5
6    print(f"Starting research: {prompt}")
7    print(f"Thread ID: {thread_id}\n")
8
9    if is_completed(db_url, thread_id):
10        print("Research already completed for this thread. Sleeping to keep pod alive.")
11        while True:
12            time.sleep(3600)
13
14    try:
15        with PostgresSaver.from_conn_string(db_url) as checkpointer:
16            checkpointer.setup()
17            graph = build_graph(checkpointer)
18            config = {"configurable": {"thread_id": thread_id}}
19            for event in graph.stream(
20                {"messages": [HumanMessage(content=prompt)]},
21                config=config,
22                stream_mode="values"
23            ):
24                last_message = event["messages"][-1]
25                last_message.pretty_print()
26        print("\nResearch completed successfully.")
27        print("Sleeping to keep pod alive for log inspection.")
28        while True:
29            time.sleep(3600)
30    except Exception as e:
31        print(f"\nAgent failed with error: {e}")
32        raise

Combine all five parts into a single agent/agent.py file in the order shown above. 

Containerizing the Agent

With the agent code in place, the next step is packaging it into a container image that Kubernetes can run.

Create the Dockerfile at the project root:

1FROM python:3.11-slim
2
3WORKDIR /app
4
5COPY agent/requirements.txt .
6
7RUN pip install --no-cache-dir -r requirements.txt
8
9COPY agent/ .
10
11CMD ["python", "agent.py"]

Build the image:

1docker build -t your-dockerhub-username/langgraph-agent:latest

Push it to Docker Hub:

1docker push your-dockerhub-username/langgraph-agent:latest

Docker push output

The image is now available for Kubernetes to pull when the Deployment starts.

Configuring the Kubernetes manifests

Two manifest files define the full agent deployment. Create them inside the kubernetes/ folder before applying anything.

kubernetes/namespace.yaml

A dedicated namespace keeps all agent resources isolated from system workloads and makes cleanup a single command.

1apiVersion: v1
2kind: Namespace
3metadata:
4  name: langgraph

kubernetes/deployment.yaml

The Deployment runs a single replica of the agent pod. Credentials are injected from the Kubernetes secret rather than hardcoded into the manifest. The restartPolicy is set to Always so Kubernetes restarts the pod if it crashes before completing. Once the agent finishes and enters its sleep loop, the pod stays alive with zero restarts.

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: langgraph-agent
5  namespace: langgraph
6spec:
7  replicas: 1
8  selector:
9    matchLabels:
10      app: langgraph-agent
11  template:
12    metadata:
13      labels:
14        app: langgraph-agent
15    spec:
16      containers:
17        - name: langgraph-agent
18          image: your-dockerhub-username/langgraph-agent:latest
19          env:
20            - name: RELAX_API_KEY
21              valueFrom:
22                secretKeyRef:
23                  name: langgraph-secret
24                  key: RELAX_API_KEY
25            - name: DATABASE_URL
26              valueFrom:
27                secretKeyRef:
28                  name: langgraph-secret
29                  key: DATABASE_URL
30            - name: RESEARCH_PROMPT
31              valueFrom:
32                secretKeyRef:
33                  name: langgraph-secret
34                  key: RESEARCH_PROMPT
35            - name: THREAD_ID
36              valueFrom:
37                secretKeyRef:
38                  name: langgraph-secret
39                  key: THREAD_ID
40          resources:
41            requests:
42              memory: 256Mi
43              cpu: 250m
44            limits:
45              memory: 512Mi
46              cpu: 500m
47      restartPolicy: Always

Replace your-dockerhub-username with your actual Docker Hub username before applying.

Deploying and watching the agent run

With the manifests in place, apply them in order. The namespace must exist before the secret, and the secret must exist before the Deployment tries to read from it.

Apply the namespace:

1kubectl apply -f kubernetes/namespace.yaml

Create the Kubernetes secret with your credentials:

1kubectl create secret generic langgraph-secret \
2  --from-literal=RELAX_API_KEY=your_relaxai_api_key \
3  --from-literal=DATABASE_URL=postgresql://civo:password@host:5432/postgres?sslmode=require \
4  --from-literal=RESEARCH_PROMPT="Research the latest developments in LangGraph and autonomous agents" \
5  --from-literal=THREAD_ID=research-k8s-001 \
6  --namespace=langgraph

Apply the Deployment:

1kubectl apply -f kubernetes/deployment.yaml

Watch the pod start:

1kubectl get pods -n langgraph -w

Pod running

Tail the logs to watch the agent work in real time:

1kubectl logs -f deployment/langgraph-agent -n langgraph

Agent calling web_search three times with different queries

The agent starts by calling web_search three times in parallel with different queries, covering different angles of the research topic. The search results come back with titles, URLs, and snippets from across the web. 

Agent calling fetch_url and reading page content

From the search results, the agent picks two URLs and calls fetch_url on each. When a page returns a 403 or 404, the tool returns a descriptive error message, and the agent searches for an alternative source rather than stopping. 

write_report call and Agent completed successfully

After gathering enough information, the agent calls write_report to generate a structured report covering the executive summary, recent developments, key features, real-world applications, limitations, future direction, and sources. The tool saves the report to report.md, and the agent logs that the research completed successfully.

After finishing the task, the pod enters a sleep loop and remains running without restarts. The workflow is complete: the agent finished its work, PostgreSQL stores the checkpoint history, and the Kubernetes Deployment stays healthy.

Verifying state persistence 

Once the agent finishes, the complete research report appears in the pod logs under the write_report tool call. The report includes an executive summary, recent developments, key features, real-world applications, limitations, future direction, and the sources used during the research.

To confirm that the agent saved its state after each step, query the checkpoints table directly using the PostgreSQL connection string from the credentials you retrieved earlier:

1python3 -c "
2import psycopg
3conn = psycopg.connect('postgresql://civo:password@host:5432/postgres?sslmode=require')
4cur = conn.cursor()
5cur.execute('SELECT thread_id, checkpoint_id FROM checkpoints ORDER BY checkpoint_id DESC LIMIT 10;')
6for row in cur.fetchall():
7    print(row)
8conn.close()
9"

Checkpoint query output

The output shows one row for each agent step. Every search, fetch, and reasoning decision was saved to PostgreSQL as a checkpoint. If the pod had been stopped during the run, Kubernetes would restart it, and the agent would load the latest checkpoint and continue from where it stopped without repeating completed work.

That is the difference between a stateless script and an agent workflow that can maintain progress.

What's next

The deployment in this tutorial covers the core of what you need to run a persistent agent on Kubernetes. From here, you can extend it in several ways:

• Run scheduled tasks by replacing the Deployment with a Kubernetes CronJob. The agent can run automatically, store each run under a new thread ID in PostgreSQL, and exit when finished.
• Add an HTTP endpoint by wrapping the agent with FastAPI and exposing it through a Civo LoadBalancer. Users can send a research prompt, receive a job ID, and check the result later.
• Run multiple research tasks at once by scaling the Deployment with multiple replicas, each using a different THREAD_ID. PostgreSQL handles the state for each agent separately.
• Use a self-hosted model by adding a GPU node pool and connecting the agent to a vLLM inference server. The agent code stays the same; only the model endpoint changes.
• Add tracing and monitoring by connecting to LangSmith. Agent steps, tool calls, and workflow progress can be tracked by adding the required environment variable.

These changes do not require modifying the core agent logic. The graph, tools, and checkpointing system remain the same. Only the infrastructure around the agent changes.

Key takeaways

The agent state survives pod restarts because it is stored in PostgreSQL instead of process memory. Every search, page fetch, and reasoning step is saved after it completes via LangGraph checkpointing. If the pod is stopped during a run, Kubernetes restarts it, and the agent loads the latest checkpoint and continues from where it stopped.

CPU-only compute is enough for this setup. relaxAI handles the LLM requests through an OpenAI-compatible API, so no GPU node is required. The entire stack runs on a single small Kubernetes node, keeping the setup simple and the cost low.

Everything is managed through one Civo account. The Kubernetes cluster, Managed PostgreSQL instance, and networking are provisioned from the same CLI and appear on the same bill. There is no need to manage multiple providers or separate database infrastructure.