Build a GPU-Accelerated OCR API with PaddleOCR on Civo Kubernetes

At some point, the scanned documents start piling up: invoices from suppliers, intake forms from clients, printed reports from a system that does not have an export button. The only way to get the data out of them is to open each one and type what you see into something else, and that works fine until the volume makes it unsustainable.

The obvious next move is an LLM, send the image, ask for the fields, parse what comes back, and for a while that seems reasonable until you get a wrong total or a date that got transposed, and for a pipeline feeding a financial system, those are not small errors because a language model predicts what the text probably says, not reads it.

PaddleOCR locates text on the page and reads it directly. For most documents, the same image produces the same JSON output every time.

In this tutorial, you will deploy PaddleOCR on a Civo L40S GPU pod with a Watcher pod that monitors a Civo Object Store bucket:

• A FastAPI service running on a Civo L40S GPU pod that returns structured JSON with every line of text and its bounding box
• A Watcher pod that monitors a Civo Object Store bucket and automatically sends new images to the OCR service
• A fully automated pipeline where you drop an image into the pending folder, and JSON lands in the results folder with no manual API calls or client code

By the end, you will have a fully automated document extraction pipeline running on Civo Kubernetes.

What you’ll build

A fully automated document extraction pipeline on Civo Kubernetes that picks up scanned images from a bucket and returns structured JSON with every line of text and its position on the page.

The pipeline works in four steps:

1. You upload an image to Civo Object Store into the pending folder
2. A Watcher pod notices the new file and forwards it to the OCR service over an internal ClusterIP connection
3. The OCR service runs PaddleOCR on a Civo L40S GPU pod and pulls out every line of text with its bounding box
4. The result lands back in the results folder in Object Store as structured JSON

The core components:

• Civo Kubernetes cluster with separate CPU and GPU node pools
• A FastAPI OCR service running on an L40S GPU pod, handling text detection and recognition
• A Watcher pod running on the CPU node that watches the bucket and triggers the OCR service automatically
• Civo Object Store as both the input source and the JSON result destination

Dropping an image into the bucket is the only manual step in the entire pipeline. 

Prerequisites

Before starting, make sure you have the following:

• A Civo account with Kubernetes and GPU access. Sign up at civo.com if you do not have an account yet.
• Civo CLI, kubectl, Docker, and s3cmd installed locally
• A container registry account, Docker Hub works fine
• A scanned document, invoice, or printed page in PNG or JPEG format to test with

Create the following folder structure:

Everything else gets created as part of the tutorial.

How it fits together

When you drop an image into the pending folder of your Civo Object Store bucket, the Watcher pod notices it. The Watcher runs on the CPU node, polls the pending folder on a fixed interval, and when it finds a new file, it calls the OCR service internally, passing the object key.

The OCR service fetches the image from Object Store and hands it to PaddleOCR, which runs two models back to back:

• A detection model that locates where text sits on the page
• A recognition model that reads it

The result gets written to the results folder in Object Store as a JSON file and returned to the Watcher. The ClusterIP service makes this internal communication work by giving the OCR service a stable hostname inside the cluster. Nothing in this pipeline is public-facing.

Image by Author

Everything in this pipeline runs inside Civo. The cluster, the Object Store bucket, and the network between the two services are all in the same environment, so there is nothing external to configure, no third-party storage to connect, and no traffic leaving the platform to get to the inference layer. 

Cluster sizing

This tutorial uses two node pools: a small CPU node for the Watcher pod and system workloads, and a single L40S GPU node for the OCR service at $1.29 per hour. One GPU node is enough because this pipeline runs a single always-on pod, not a parallel batch job.

Both node pools are small by design. The Watcher barely touches the CPU node, and the OCR service runs as a single always-on pod on the GPU. One node each is all this pipeline needs.

Creating the cluster

Before deploying anything, you need a Kubernetes cluster with two node pools: one CPU node for the Watcher pod and one L40S GPU node for the OCR service.

Authenticate the Civo CLI and create the cluster:

1civo apikey save my-key YOUR_API_KEY_HERE
2civo apikey use my-key
3
4civo kubernetes create ocr-cluster \
5  --size=g4s.kube.small \
6  --nodes=1 \
7  --region=NYC1 \
8  --wait

If the CLI times out, run civo kubernetes ls --region=NYC1 and confirm the cluster shows ACTIVE with All Workers Up: True before moving on.

Save the kubeconfig so kubectl can connect to the cluster:

# Linux / macOS

1civo kubernetes config ocr-cluster --region NYC1 --save --switch

# Windows

1civo kubernetes config ocr-cluster --region NYC1 --save --switch --local-path $env:USERPROFILE\.kube\config

Now add the L40S GPU node pool:

1civo kubernetes node-pool create ocr-cluster \
2  --size=an.g1.l40s.kube.x1 \
3  --nodes=1 \
4  --region=NYC1

Wait 3 minutes, then verify both nodes are Ready:

1kubectl get nodes

Installing the NVIDIA GPU Operator

Without the GPU Operator, nvidia.com/gpu: 1 in the deployment YAML is not recognized, and the OCR service pod will stay in Pending indefinitely.

1helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
2helm repo update
3
4helm upgrade --install gpu-operator \
5  -n gpu-operator --create-namespace \
6  nvidia/gpu-operator \
7  --set driver.enabled=true \
8  --set toolkit.enabled=false \
9  --set devicePlugin.enabled=true \
10  --set gfd.enabled=true \
11  --set operator.defaultRuntime=containerd \
12  --set validator.cuda.runtimeClassName=nvidia

Wait 5 minutes, then verify all pods are Running or Completed:

1kubectl get pods -n gpu-operator

Verify the GPU is allocatable:

1kubectl get nodes -o custom-columns="NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"

The GPU node must show 1 before moving on.

Setting up Civo Object Store

The Object Store bucket is the entry point for the entire pipeline. Images go into the pending/ folder, and JSON results come back into the results/ folder. Nothing in the pipeline moves without the bucket in place first.

Create the bucket credentials and the bucket itself:

1civo objectstore credential create ocr-creds --region=NYC1
2civo objectstore create ocr-jobs --region=NYC1 --size=500

Verify the bucket is ready:

1civo objectstore ls --region=NYC1

Get the access key:

1civo objectstore show ocr-jobs --region=NYC1

Copy the Access Key from the output, then get the secret:

1civo objectstore credential secret --access-key=YOUR_ACCESS_KEY --region=NYC1

Save both values somewhere safe. You will use them in the deploy step to create the Kubernetes Secret that injects the credentials into both pods at runtime. 

Now configure s3cmd to talk to the bucket:

1s3cmd --configure

Fill in the following:

• Access Key: Your bucket access key
• Secret Key: Your bucket secret key
• Default Region: us-east-1
• S3 Endpoint: objectstore.nyc1.civo.com
• DNS-style bucket: objectstore.nyc1.civo.com
• Encryption password: Leave empty
• Use HTTPS: Yes

When asked to test, say Yes. You should see:

Now, create the folder structure inside the bucket:

1New-Item -Path .keep -ItemType File
2s3cmd put .keep s3://ocr-jobs/pending/.keep
3s3cmd put .keep s3://ocr-jobs/results/.keep
4Remove-Item .keep

Upload your test images to the pending folder:

1s3cmd put .\test-images\invoice.jpg s3://ocr-jobs/pending/invoice.jpg
2s3cmd put .\test-images\resume.jpg s3://ocr-jobs/pending/resume.jpg
3s3cmd put .\test-images\form.jpg s3://ocr-jobs/pending/form.jpg

Verify everything is in place:

1s3cmd ls s3://ocr-jobs/pending/

With the bucket ready and the test images uploaded, the next step is building the two services that will process them.

Building the OCR service

The OCR service is where the actual work happens. It runs as a single FastAPI pod on the L40S GPU node, stays alive between requests, and every time the Watcher finds a new image it calls this service with the object key and waits for the result. 

The /ocr endpoint accepts the object key, fetches the image from Object Store, runs PaddleOCR, writes the result JSON to the results/ folder, and returns the output to the Watcher.

Initialization

The first thing main.py does is initialize the app and the OCR engine:

1from fastapi import FastAPI, HTTPException
2from pydantic import BaseModel
3import boto3
4import os
5import tempfile
6from paddleocr import PaddleOCR
7import json
8
9app = FastAPI()
10ocr = PaddleOCR(use_angle_cls=True, lang="en")

PaddleOCR initializes once at startup, loads both models into GPU memory, and stays ready. Every request goes straight to inference 

Object Store client

Next, the service connects to Object Store using credentials from environment variables:

1from botocore.config import Config
2
3s3 = boto3.client(
4    "s3",
5    endpoint_url=os.environ["OBJECT_STORE_ENDPOINT"],
6    aws_access_key_id=os.environ["OBJECT_STORE_ACCESS_KEY"],
7    aws_secret_access_key=os.environ["OBJECT_STORE_SECRET_KEY"],
8    config=Config(
9        signature_version="s3v4",
10        request_checksum_calculation="when_required"
11    )
12)

Nothing is hardcoded. The Kubernetes Secret injects all four values at runtime.

The endpoint

The request model and the /ocr endpoint:

1class OCRRequest(BaseModel):
2    object_key: str
3
4@app.post("/ocr")
5def run_ocr(request: OCRRequest):
6    try:
7        with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as tmp:
8            s3.download_fileobj(BUCKET, request.object_key, tmp)
9            tmp_path = tmp.name
10
11        result = ocr.ocr(tmp_path, cls=True)
12
13        lines = []
14        for line in result[0]:
15            bbox, (text, confidence) = line
16            lines.append({
17                "text": text,
18                "bbox": bbox,
19                "confidence": round(confidence, 4)
20            })
21
22        output = {
23            "object_key": request.object_key,
24            "lines": lines
25        }
26
27        result_key = request.object_key.replace("pending/", "results/").replace(".png", ".json").replace(".jpg", ".json")
28
29        s3.put_object(
30            Bucket=BUCKET,
31            Key=result_key,
32            Body=json.dumps(output, indent=2),
33            ContentType="application/json"
34        )
35
36        return output
37
38    except Exception as e:
39        raise HTTPException(status_code=500, detail=str(e))

The endpoint fetches the image into a temp file, runs PaddleOCR, builds the output, writes the result JSON to the results/ folder by mirroring the input key, and returns the JSON to the Watcher.

Dependencies

ocr-service\requirements.txt:

1fastapi
2uvicorn
3paddleocr==2.7.3
4paddlepaddle-gpu==2.6.1
5boto3
6botocore
7pydantic
8numpy==1.24.3
9opencv-python-headless==4.8.0.76

Expected output

When the endpoint processes an image, this is what the JSON looks like:

1{
2  "object_key": "pending/invoice.jpg",
3  "lines": [
4    {
5      "text": "Invoice #1042",
6      "bbox": [[10, 20], [200, 20], [200, 45], [10, 45]],
7      "confidence": 0.98
8    },
9    {
10      "text": "Total excl. VAT",
11      "bbox": [[10, 300], [150, 300], [150, 320], [10, 320]],
12      "confidence": 0.97
13    }
14  ]
15}

text is what PaddleOCR reads, bbox is the four corner coordinates of where the text sits on the page, and confidence is how certain the model is on a scale of 0 to 1.

Building the Watcher

The Watcher does not do much, and that is the point. It runs on the CPU node, checks the pending/ folder every 10 seconds, and when it finds something new it forwards it to the OCR service and waits. It is the lightest possible piece of the pipeline. 

Initialization

watcher\watcher.py starts by setting up logging and the Object Store client:

1import boto3
2import os
3import time
4import requests
5import logging
6
7logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
8
9s3 = boto3.client(
10    "s3",
11    endpoint_url=os.environ["OBJECT_STORE_ENDPOINT"],
12    aws_access_key_id=os.environ["OBJECT_STORE_ACCESS_KEY"],
13    aws_secret_access_key=os.environ["OBJECT_STORE_SECRET_KEY"],
14)
15
16BUCKET = os.environ["OBJECT_STORE_BUCKET"]
17OCR_SERVICE_URL = os.environ["OCR_SERVICE_URL"]
18POLL_INTERVAL = int(os.environ.get("POLL_INTERVAL", 10))

OCR_SERVICE_URL points to the ClusterIP service name. Kubernetes resolves it internally so the Watcher calls http://ocr-service/ocr without ever knowing the IP. POLL_INTERVAL defaults to 10 seconds but can be changed via environment variable. 

Tracking processed files

The Watcher keeps an in-memory set of files it has already sent to the OCR service so the same image does not get processed twice:

1processed = set()

Polling and processing

1def list_pending():
2    response = s3.list_objects_v2(Bucket=BUCKET, Prefix="pending/")
3    if "Contents" not in response:
4        return []
5    return [obj["Key"] for obj in response["Contents"] if obj["Key"].endswith((".png", ".jpg", ".jpeg"))]
6
7def process(object_key):
8    logging.info(f"Detected new file: {object_key}")
9    try:
10        response = requests.post(OCR_SERVICE_URL, json={"object_key": object_key})
11        response.raise_for_status()
12        logging.info(f"Processed {object_key} successfully")
13        processed.add(object_key)
14    except Exception as e:
15        logging.error(f"Failed to process {object_key}: {e}")
16
17def watch():
18    logging.info("Watcher started, polling every %s seconds", POLL_INTERVAL)
19    while True:
20        files = list_pending()
21        for key in files:
22            if key not in processed:
23                process(key)
24        time.sleep(POLL_INTERVAL)
25
26if __name__ == "__main__":
27    watch()

list_pending lists new image files in the pending/ folder, process calls the OCR service and marks the file as done, and watch runs the loop on repeat.

Dependencies

watcher\requirements.txt:

1boto3
2requests

The Watcher only needs boto3 to talk to Object Store and requests to call the OCR service. No GPU libraries, no heavy dependencies, which is why it runs on the CPU node and builds into a container under 100MB. 

Containerizing the Services

Each service gets its own Dockerfile. The OCR service needs a GPU-capable base image with CUDA and cuDNN pre-installed. The Watcher needs nothing more than a slim Python image.

OCR Service

ocr-service\Dockerfile:

1FROM paddlepaddle/paddle:2.6.1-gpu-cuda11.7-cudnn8.4-trt8.4
2
3WORKDIR /app
4
5COPY requirements.txt .
6RUN pip install --upgrade pip && \
7    pip install -r requirements.txt
8
9COPY main.py .
10
11CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

The base image comes with CUDA 11.7, cuDNN 8.4, and TensorRT 8.4 pre-installed, so PaddleOCR runs on the L40S without any manual driver setup.

On first startup, PaddleOCR downloads its models at runtime, not during the build. Expect a minute or two before the pod starts accepting requests. Every restart after that uses the cached models.

Watcher

watcher\Dockerfile:

1FROM python:3.11-alpine
2
3WORKDIR /app
4
5COPY requirements.txt .
6RUN pip install --upgrade pip && \
7    pip install -r requirements.txt
8
9COPY watcher.py .
10
11CMD ["python", "watcher.py"]

Alpine works here because the Watcher has no native GPU dependencies. The result is a container under 100MB compared to the OCR service image, which is around 7GB.

Build and push both images

1docker build -t YOUR_REGISTRY/ocr-service:latest .\ocr-service docker push YOUR_REGISTRY/ocr-service:latest 

1docker build -t YOUR_REGISTRY/watcher:latest .\watcher docker push YOUR_REGISTRY/watcher:latest 

Both images are on Docker Hub. The OCR service image carries the full PaddlePaddle GPU stack, and the Watcher image is a lightweight polling script under 100MB. The next step is the Kubernetes manifests that wire them together.

Deploying to Civo Kubernetes

With both images in the registry, three manifests handle the deployment: a Deployment and a ClusterIP Service for the OCR service, and a Deployment for the Watcher. Credentials go in separately via the CLI so nothing sensitive touches a file. 

Kubernetes Secret

A Kubernetes Secret injects sensitive values into pods as environment variables at runtime. Nothing sensitive lives in the deployment YAML or the container image.

Create the Secret using the credentials you saved in the Object Store setup step:

1kubectl create secret generic object-store-credentials `
2  --from-literal=OBJECT_STORE_ENDPOINT=https://objectstore.nyc1.civo.com `
3  --from-literal=OBJECT_STORE_ACCESS_KEY=YOUR_ACCESS_KEY `
4  --from-literal=OBJECT_STORE_SECRET_KEY=YOUR_SECRET_KEY `
5  --from-literal=OBJECT_STORE_BUCKET=ocr-jobs

Verify the Secret was created:

1kubectl get secret object-store-credentials

Expected:

1NAME                       TYPE     DATA   AGE
2object-store-credentials   Opaque   4      1s

OCR Service Deployment

kubernetes\ocr-deployment.yaml:

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: ocr-service
5spec:
6  replicas: 1
7  selector:
8    matchLabels:
9      app: ocr-service
10  template:
11    metadata:
12      labels:
13        app: ocr-service
14    spec:
15      containers:
16        - name: ocr-service
17          image: YOUR_REGISTRY/ocr-service:latest
18          ports:
19            - containerPort: 8000
20          envFrom:
21            - secretRef:
22                name: object-store-credentials
23          resources:
24            requests:
25              memory: "8Gi"
26              cpu: "2"
27            limits:
28              nvidia.com/gpu: 1
29              memory: "8Gi"
30              cpu: "2"
31          readinessProbe:
32            httpGet:
33              path: /docs
34              port: 8000
35            initialDelaySeconds: 60
36            periodSeconds: 10
37      nodeSelector:
38        nvidia.com/gpu.present: "true"
39
40

A few things worth noting:

• nvidia.com/gpu: 1 tells Kubernetes this pod needs a GPU, without it the pod lands on the CPU node
• nodeSelector pins the pod to the L40S node, the GPU Operator adds this label automatically
• readinessProbe gives PaddleOCR 60 seconds to download its models before the pod starts accepting requests
• envFrom pulls all credentials from the Secret so nothing is hardcoded

ClusterIP Service

A ClusterIP service gives the OCR pod a stable internal hostname so the Watcher can call it by name rather than by IP address. It has no external IP and is not reachable from outside the cluster.

kubernetes\ocr-service.yaml:

1apiVersion: v1
2kind: Service
3metadata:
4  name: ocr-service
5spec:
6  selector:
7    app: ocr-service
8  ports:
9    - protocol: TCP
10      port: 80
11      targetPort: 8000
12  type: ClusterIP

Port 80 on the service maps to port 8000 on the container, so the Watcher calls http://ocr-service/ocr not http://ocr-service:8000/ocr.

Watcher Deployment

kubernetes\watcher-deployment.yaml:

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: watcher
5spec:
6  replicas: 1
7  selector:
8    matchLabels:
9      app: watcher
10  template:
11    metadata:
12      labels:
13        app: watcher
14    spec:
15      containers:
16        - name: watcher
17          image: YOUR_REGISTRY/watcher:latest
18          env:
19            - name: OCR_SERVICE_URL
20              value: "http://ocr-service/ocr"
21          envFrom:
22            - secretRef:
23                name: object-store-credentials
24          resources:
25            requests:
26              cpu: "100m"
27              memory: "128Mi"
28            limits:
29              cpu: "200m"
30              memory: "256Mi"
31      nodeSelector:
32        kubernetes.io/os: linux
33
34

OCR_SERVICE_URL is set to http://ocr-service/ocr. Kubernetes DNS resolves ocr-service to the correct pod IP automatically so the Watcher never needs to know the actual IP address.

Apply the manifests

Apply them in this order. The Secret needs to exist before the pods start otherwise they will fail to read the credentials on startup: 

1kubectl apply -f kubernetes\ocr-deployment.yaml
2kubectl apply -f kubernetes\ocr-service.yaml
3kubectl apply -f kubernetes\watcher-deployment.yaml

Verify all resources are running:

1kubectl get all
2

If the OCR service pod stays in Pending run:

1kubectl describe pod ocr-service-xxxxx
2

Check the Events section at the bottom for scheduling errors. The most common cause is the GPU Operator not finishing initialization. Give it a few more minutes and try again. 

Verify the OCR service loaded PaddleOCR successfully:

1kubectl logs deployment/ocr-service

Once the Watcher logs show all three files processed, check the results folder:

1s3cmd ls s3://ocr-jobs/results/

Download and inspect one result to verify the output:

1s3cmd get s3://ocr-jobs/results/invoice.json -

Finally, verify the full bucket structure to confirm input and output are both in place:

1s3cmd ls --recursive s3://ocr-jobs/

The Watcher detected all three images, sent them to the OCR service, and the results landed in the results/ folder without any manual intervention. Every line of text from every document is now structured JSON, ready for any downstream system to consume. 

If the Watcher logs don't show the expected output, here's what to check:

• Watcher logs show no files detected: The pending/ folder is empty or the .keep placeholder is the only object. Run s3cmd ls s3://ocr-jobs/pending/ to confirm the images are there.
• Watcher logs show a connection error calling the OCR service: The ClusterIP service isn't reachable. Run kubectl get svc and confirm ocr-service is listed. If the OCR pod isn't ready yet, the Watcher will keep retrying on the next poll.
• OCR service pod is stuck in Pending: Almost always a GPU scheduling issue. Run kubectl describe pod ocr-service-xxxxx and check the Events section. If it says insufficient nvidia.com/gpu, the GPU Operator hasn't finished initializing. Give it a few more minutes.
• Watcher logs show a 500 from the OCR service: The image may be corrupt or in an unsupported format. Check kubectl logs deployment/ocr-service for the full error. PaddleOCR supports PNG and JPEG, other formats will fail.
• Results folder is empty after processing: The OCR service ran but failed to write to Object Store. Check the credentials in the Secret match what civo objectstore show returns.

Clean up

To remove everything when you're done:

1kubectl delete deployment ocr-service watcher
2kubectl delete service ocr-service
3kubectl delete secret object-store-credentials
4

Then from the Civo dashboard, delete the Kubernetes cluster and the Object Store bucket. The GPU node bills by the hour, so delete the cluster as soon as you're finished to avoid ongoing charges.

Key takeaways 

PaddleOCR is the right tool when your documents are typed, printed, and reasonably clean. The output is deterministic, which matters when the pipeline feeds a database or financial system. Low resolution scans, dense multi-column layouts, and non-English documents are where it struggles, and an LLM with vision is the better call in those cases.

The most natural extension from here is PDF support: split pages into images with pdf2image, send each to /ocr, and merge the results. The Watcher needs one change to detect .pdf files in the pending/ folder.

Additional resources

• Civo Kubernetes documentation
• Civo Object Store documentation
• Civo CLI documentation
• PaddleOCR documentation
• PaddleOCR GitHub
• NVIDIA GPU Operator