How to build a fully sovereign AI stack in India using Civo’s sovereign region

As organizations move AI systems from experimentation to production, especially on sensitive datasets like financial transactions and healthcare records, infrastructure choices are now as critical as model selection. The focus has shifted from simply building AI to deciding where and how data is processed and protected.

In India, this shift is accelerating. The demand for AI infrastructure is rising in parallel with a sharper focus on data sovereignty and operational control. For many organizations, sovereignty is no longer just a compliance checkbox; it is now central to how systems are architected and managed.

A sovereign AI stack integrates compute, storage, and networking within a defined regional boundary. This approach gives organizations full control over both infrastructure and data. By keeping all components local, you can run AI workloads with predictable performance, data locality, and strict access controls.

In this tutorial, you will set up a foundational sovereign AI stack in Civo’s India region using core cloud services: compute, storage, and private networking. This setup offers a practical starting point for running AI workloads today, with a clear path to scaling for production.

Prerequisites

Before you begin, ensure you have the following:

• A Civo account with access to the dashboard
• A system with a terminal (Linux, macOS, or Windows with SSH support)
• Basic familiarity with Linux command-line usage
• An SSH key configured on your system (recommended for instance access)
• The Civo CLI installed

Architecture overview

This setup brings together compute, storage, and networking within Civo’s India region to create a practical foundation for a sovereign AI stack.

At a high level, the system follows a straightforward flow:

A request is sent to the public endpoint, which is handled by a compute instance running the AI inference service. The service processes the request using locally available models or data, while all internal communication remains within a private network.

Each component serves a specific role in the architecture:

• Compute acts as the inference layer, executing AI models and managing incoming API requests.
• Storage ensures persistent access to model artifacts and data. Object storage handles large assets, while block storage supports low-latency local access.
• Networking defines the sovereignty boundary, keeping all internal data transit isolated within the India region and under strict access controls.

Separating compute, storage, and networking follows a proven pattern in production AI deployments. This approach allows each layer to scale and operate independently.

By keeping all components within the same regional environment and limiting external exposure, this architecture gives you practical control over infrastructure, data locality, and access boundaries for AI workloads.

Create a private network

To keep the stack isolated, create a dedicated private network for the resources used in this guide.

1. In the Civo dashboard, go to VPC → Networks and click Create network.
2. Enter a name such as sovereign-ai-network.
3. If you expand Show advanced options, you will see fields for Nameserver and CIDR. For this setup, leave those values unchanged and use the defaults.
4. Click Create. Once the network appears in the list, use it for the compute and storage resources you create in the following steps.

Create a firewall

Securing your AI infrastructure starts with controlling network access. By setting up a custom firewall, you can restrict external traffic to only what your AI stack needs, reducing unnecessary exposure.

1. In the Civo dashboard, navigate to VPC → Firewalls, then click Create a firewall.
2. Enter a name such as sovereign-ai-firewall, and select the private network you created earlier.
3. After creating the firewall, open it to configure its rules. By default, new firewalls include rules that allow all traffic. Remove these rules to restrict access, then add only the required inbound rules:

• TCP port 22 for SSH access.
• TCP port 11434 for the AI inference service.

4. Set the source to 0.0.0.0/0 for general access, or limit it to your specific IP address for stricter control.

Create a compute instance

In this step, you will provision a compute instance that runs your AI inference service.

1. In the Civo dashboard, navigate to Compute → Instances, then click Create new instance.
2. Enter a hostname and keep the instance count as 1.
3. Instance Size: Select a plan with at least 4 vCPUs and 8 GB RAM (e.g., g4s.large).
4. Image: Choose Ubuntu (latest LTS).
5. Initial User: Set to civo.
6. Networking:

• Select your custom private network (sovereign-ai-network).
• Enable a public IP address.
• Attach your custom firewall (sovereign-ai-firewall).

7. Authentication: Select your SSH key for secure access.
8. Click Create instance. Note the assigned public IP address once provisioned.

For automated workflows, you can also use the Civo CLI:

1civo instance create \
2  --hostname sovereign-ai \
3  --size g4s.large \
4  --diskimage ubuntu-noble \
5  --initialuser civo \
6  --network sovereign-ai-network \
7  --firewall sovereign-ai-firewall \
8  --publicip create \
9  --sshkey <SSH_KEY_NAME_OR_ID> \
10  --region mum1 \
11  --wait

Create and attach block storage

Block storage provides persistent, high-performance storage that remains available independently of your compute instance. In this setup, it is used to store AI models and datasets separately from the system disk, making the environment easier to manage and scale.

1. In the Civo dashboard, navigate to Volumes and click Create a volume.
2. Name: ai-model-storage.
3. Network: Select the same private network used by your instance.
4. Size: Choose 40 GB or higher.
5. Click Create. Once provisioned, navigate to your instance overview and attach the volume in the Volumes section.

Prepare and mount the volume

Connect to your instance using SSH:

1ssh civo@<YOUR_PUBLIC_IP>

Verify that the new disk is available:

1lsblk

You should see an additional disk (for example, /dev/sda) that is not part of the system partitions.

The device name may vary depending on your instance configuration.

Create a filesystem on the new volume:

1sudo mkfs.ext4 /dev/sda

Create a mount point:

1sudo mkdir -p /mnt/ai-storage

Mount the volume:

1sudo mount /dev/sda /mnt/ai-storage

Verify that the volume is mounted:

1df -h

At this stage, the volume is ready to use and can store AI models and data.

Configure persistent mounting

Retrieve the volume UUID:

1sudo blkid

You will see output similar to:

Use your actual volume UUID in the following command to add an entry to /etc/fstab:

1echo 'UUID=xxxx-xxxx-xxxx-xxxx /mnt/ai-storage ext4 defaults,nofail 0 2' | sudo tee -a /etc/fstab

Reload systemd configuration to apply the updated mount settings:

1sudo systemctl daemon-reload

Test the mount configuration

Unmount the volume to simulate a system restart:

1sudo umount /mnt/ai-storage

This temporarily removes the mounted volume so you can verify that it is correctly defined in /etc/fstab.

Mount all filesystems defined in /etc/fstab:

1sudo mount -a

If no errors appear, the configuration is correct, and the volume will be mounted automatically on reboot.

Create object storage

Object storage is used to store large files such as AI models, datasets, and backups. It is accessed over an API and is designed for scalable, long-term storage.

1. In the Civo dashboard, navigate to Object Store → Credentials and click Create a new Credential. Name it ai-storage-credentials.
2. Go to the Object Store and click Create an Object Store.
3. Name: ai-data-store.
4. Credential: Select the credentials you just created.
5. Click Create store.
6. Inside the store, you can create folders (e.g., ai-models) to organize your assets.

Install and configure the AI inference service

Install and configure an AI inference service on your compute instance. This guide uses Ollama, which provides a simple way to run large language models locally.

By default, Ollama stores models on the system disk. To improve scalability and keep model data separate from the operating system, configure it to use the attached block storage volume.

Install Ollama

Update the system package index:

1sudo apt update

Install Ollama using the official installation script:

1curl -fsSL https://ollama.com/install.sh | sh

Verify the installation:

1ollama --version

Verify the Ollama service

Ollama runs as a system service after installation. Verify that the service is active:

1sudo systemctl status ollama

If the service is not running, restart it:

1sudo systemctl restart ollama

Configure model storage location

Create a directory inside the mounted volume:

1sudo mkdir -p /mnt/ai-storage/ollama

Set ownership so the Ollama service can access the directory:

1sudo chown -R ollama:ollama /mnt/ai-storage/ollama

Expose the Ollama API and configure storage

By default, Ollama listens only on the local interface (127.0.0.1). To allow external access and ensure models are stored on the attached volume, update the service configuration.

Create a systemd override for the Ollama service:

1sudo systemctl edit ollama

This creates an override file at /etc/systemd/system/ollama.service.d/override.conf. You do not need to edit the main service file directly.

Add the following configuration:

1[Service]
2Environment="OLLAMA_HOST=0.0.0.0:11434"
3Environment="OLLAMA_MODELS=/mnt/ai-storage/ollama"

Reload the systemd configuration to apply the updated service settings:

1sudo systemctl daemon-reload

Restart the Ollama service to apply the changes:

1sudo systemctl restart ollama

This configuration ensures that the service listens on all interfaces and stores model data on the attached volume.

Run a model

Start a model to verify the setup:

1ollama run llama3

The first run downloads the model to the configured storage location. This may take some time depending on network speed.

Once the model is ready, you can interact with it directly in the terminal. 

Press Ctrl + D to exit the session.

Test the API endpoint

Verify that the Ollama API is responding locally on the server:

1curl http://127.0.0.1:11434/api/generate \
2  -d '{
3    "model": "llama3",
4    "prompt": "Explain data sovereignty in simple terms",
5    "stream": false
6  }'

If the configuration is correct, the API returns a generated response from the model.

Test external access from your local machine or another terminal using the instance's public IP:

1curl http://<YOUR_PUBLIC_IP>:11434/api/generate \
2  -d '{
3    "model": "llama3",
4    "prompt": "Explain data sovereignty in simple terms",
5    "stream": false
6  }'

If this request succeeds and returns a valid JSON response, the inference service is reachable through the public endpoint and using the configured model storage.

Verify the sovereign setup

Verify that your infrastructure and data handling are configured correctly and operating within the intended regional and security boundaries.

Verify resource placement

In the Civo dashboard, confirm that your resources are deployed in the same region:

• Compute instance
• Private network
• Object storage

All components should be created in the India (mum1) region to ensure data locality.

Verify storage persistence

Confirm that the attached block storage is mounted and in use:

1df -h /mnt/ai-storage

Verify that model data is stored on the mounted volume:

1ls -lh /mnt/ai-storage/ollama

If model files are present in this directory, the storage layer is correctly configured and decoupled from the system disk.

Verify network access controls

Ensure that only the required ports are accessible externally.

From your local machine, test a non-configured port:

1curl --connect-timeout 5 http://<YOUR_PUBLIC_IP>:8080

The request should fail or time out, confirming that unsolicited access is blocked.

Verify API accessibility

Confirm that the AI inference service is reachable and responding correctly:

1curl http://<YOUR_PUBLIC_IP>:11434/api/generate \
2  -d '{
3    "model": "llama3",
4    "prompt": "test",
5    "stream": false
6  }'

A valid JSON response confirms that the service is accessible and functioning correctly.

Conclusion

In this guide, we walked through building a sovereign AI stack on Civo’s India region. By combining compute, block storage, object storage, and private networking, you can deploy an AI inference service that remains fully within a defined regional boundary.

Configuring dedicated storage for model data and applying explicit firewall rules helps maintain predictable performance, enforces data locality, and limits service exposure to only what is necessary.

This approach provides a practical foundation for running AI workloads that require controlling both infrastructure placement and data handling.

You can extend this setup by integrating managed Kubernetes clusters, automating deployments with tools like Terraform, or scaling the inference layer to support production workloads as your requirements grow.