Fine-tune Llama 4 with LoRA on a Civo H100 cluster using Unsloth
Fine-tune Llama 4 Scout with QLoRA on a single Civo H100 node, from a custom JSONL dataset in Object Store to a trained LoRA adapter on Hugging Face Hub, with live loss curves in Weights & Biases throughout.
Written by
Software Engineer at GoCardless
Written by
Software Engineer at GoCardless
A base model is trained on broad internet-scale data, which makes it useful across a wide range of tasks but not deeply specialized in any of them. It understands your domain in general terms, but it does not know your specific terminology, your edge cases, or the exact response patterns your use case requires. At some point, adding more instructions to the prompt stops making a difference, and that is usually the signal that the model's underlying behavior needs to change, not just the prompt.
That is where fine-tuning comes in. LoRA makes it practical on a single GPU by training a small set of adapter weights while keeping the base model frozen. Unsloth handles the training pipeline with custom CUDA kernels that cut memory usage significantly. It is currently the only framework that supports 4-bit QLoRA fine-tuning for Llama 4 Scout on a single GPU.
This tutorial walks through fine-tuning Llama 4 Scout on a custom dataset using a single Civo H100 node, with the adapter pushed to Hugging Face Hub and loss curves streaming to Weights & Biases in real time:
- Build a custom JSONL dataset and upload it to Civo Object Store
- Run an Unsloth training job with QLoRA fine-tuning on the H100 node
- Push the trained adapter to Hugging Face Hub automatically at the end of training
- Run an inference check to confirm the fine-tuned model produces the expected output format
You will get a trained LoRA adapter pushed to Hugging Face Hub and inference results that clearly demonstrate what changed in the model.
Prerequisites
Before starting, make sure you have the following:
Accounts
- A Civo account with Kubernetes and H100 GPU access
- A Hugging Face account with Llama 4 Scout access approved and a token with write scope
- A Weights & Biases account (free tier works)
H100 GPU availability
H100 GPU nodes are available in select Civo regions. This tutorial uses LON1. If you are in a different region, check GPU node availability in the Civo dashboard before starting. If the an.g1.h100.kube.x1 size is not listed, contact Civo support to request access.
Local environment
- Civo CLI installed and authenticated
- kubectl installed
- Helm installed
- s3cmd installed and configured
- Python 3.10+ for the eval script
Tested with
- Civo CLI v1.1.6
- Kubernetes v1.35.0
- Unsloth 2026.5.9
- PyTorch 2.10.0
- CUDA 12.8
Project structure
llama4-finetune/├── dataset/│ ├── train.jsonl│ └── eval.jsonl├── scripts/│ └── train.py├── kubernetes/│ └── training-job.yaml└── eval/└── infer.py
dataset/contains the training and eval JSONL filesscripts/contains the Unsloth training script and inference check scriptkubernetes/contains the Kubernetes Job manifesteval/contains the Python script for base vs fine-tuned comparison
How it fits together
The pod loads Llama 4 Scout from Hugging Face and the dataset from Civo Object Store. Unsloth attaches QLoRA adapters to the frozen model and trains on the H100 node. Loss curves stream to Weights & Biases throughout training. When training completes, the adapter is pushed to Hugging Face Hub automatically.
Each component has a specific role:
- Civo Object Store stores the dataset and training script. The pod pulls both at startup with no manual transfer needed after the initial upload
- The H100 node pool runs Unsloth as a Kubernetes Job, not a Deployment. The job runs to completion and terminates, with automatic retries on failure
- Hugging Face Hub supplies the base model at job start and receives the trained adapter when training completes
- Weights & Biases streams training metrics in real time without SSH access or manual log inspection
The only external dependency is the model on Hugging Face.
Creating the cluster
Before deploying anything, you need a Kubernetes cluster with two node pools: one CPU node for system workloads and one H100 node for the training job.
Authenticate the Civo CLI and create the cluster:
civo apikey save my-key YOUR_API_KEY_HEREcivo apikey use my-keycivo kubernetes create llama4-cluster \--size=g4s.kube.small \--nodes=1 \--region=LON1 \--wait
Save the kubeconfig so kubectl can connect to the cluster:
civo kubernetes config llama4-cluster --region LON1 --save --switch
Verify the cluster is reachable:
kubectl cluster-info
Once the control plane URL appears, the cluster is ready.
Adding the H100 node pool
civo kubernetes node-pool create llama4-cluster \--size=an.g1.h100.kube.x1 \--nodes=1 \--region=LON1
Wait 5 minutes, then verify both nodes are ready:
kubectl get nodes
If the H100 node stays NotReady after 5 minutes, check the Events section:
kubectl describe node NODE_NAME
Installing the NVIDIA GPU operator
Without the GPU Operator, nvidia.com/gpu: 1 is ignored, and the pod remains Pending. Single H100 nodes on Civo also need an NVLink workaround; otherwise, CUDA fails to initialize. Create the ConfigMap before installing the operator:
kubectl create namespace gpu-operator --dry-run=client -o yaml | kubectl apply -f -kubectl -n gpu-operator create configmap nvidia-kernel-config \--from-literal=nvidia.conf='options nvidia NVreg_NvLinkDisable=1' \--dry-run=client -o yaml | kubectl apply -f -
Add the Helm repo and install with the NVLink fix:
helm repo add nvidia https://helm.ngc.nvidia.com/nvidiahelm repo updatehelm upgrade --install gpu-operator \-n gpu-operator --create-namespace \nvidia/gpu-operator \--set driver.enabled=true \--set driver.kernelModuleConfig.name=nvidia-kernel-config \--set toolkit.enabled=false \--set devicePlugin.enabled=true \--set gfd.enabled=true \--set operator.defaultRuntime=containerd \--set validator.cuda.runtimeClassName=nvidia
Why toolkit.enabled=false?
Civo's H100 nodes come with the NVIDIA Container Runtime pre-installed on the host. Setting toolkit.enabled=false tells the GPU Operator not to install or manage it, which prevents the Operator from conflicting with the existing installation. The nvidia runtime class required by validator.cuda.runtimeClassName is also pre-registered on Civo nodes, so no additional setup is needed.
The first installation takes 15 to 30 minutes due to kernel module compilation. Wait, then verify all pods are Running or Completed:
kubectl get pods -n gpu-operator
You may see nvidia-cuda-validator in CrashLoopBackOff. This is a known timing issue with GPU Operator on H100: the validator pod tests GPU access using a runtime class that may not be fully ready at first boot, and it keeps restarting until the test passes. The device plugin, which is the component that actually allocates the GPU to your training pod, runs independently and is not affected. Do not wait for the validator to clear before continuing; confirm GPU access is working using the verification step below instead.
Verify the GPU is allocatable:
kubectl get nodes -o custom-columns="NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"
The H100 node must show 1 before moving on.
Object Store setup
The Object Store bucket stores the training dataset and training script. The pod pulls both at startup without any manual file transfer after the initial upload.
Create the bucket credentials and the bucket:
civo objectstore credential create llama4-creds --region=LON1civo objectstore create llama4-finetune --region=LON1 --size=500
Verify the bucket is ready:
civo objectstore ls --region=LON1
Get the access key:
civo objectstore show llama4-finetune --region=LON1
Copy the Access Key from the output, then get the secret:
civo objectstore credential secret --access-key=YOUR_ACCESS_KEY --region=LON1
Save both values. You will use them in the next step to create the Kubernetes Secret.
Now configure s3cmd to talk to the bucket:
s3cmd --configure
Fill in the following:
- Access Key: your bucket access key
- Secret Key: your bucket secret key
- Default Region:
us-east-1 - S3 Endpoint:
objectstore.lon1.civo.com - DNS-style bucket:
objectstore.lon1.civo.com - Encryption password: leave empty
- Use HTTPS: Yes
When asked to test, say Yes.
Verify the bucket is accessible:
s3cmd ls
The bucket llama4-finetune should appear in the list.
Dataset prep
The dataset is a JSONL file with instruction and output fields. It pairs alert scenarios with structured six-step kubectl runbooks for DevOps incident response.
The dataset ships with 700 training records in train.jsonl and 50 held-out records in eval.jsonl. Place both files in the dataset/ folder, then create the bucket folder structure and upload everything:
touch .keeps3cmd put .keep s3://llama4-finetune/dataset/.keeps3cmd put .keep s3://llama4-finetune/scripts/.keeps3cmd put .keep s3://llama4-finetune/checkpoints/.keeprm .keeps3cmd put dataset/train.jsonl s3://llama4-finetune/dataset/train.jsonls3cmd put dataset/eval.jsonl s3://llama4-finetune/dataset/eval.jsonls3cmd put scripts/train.py s3://llama4-finetune/scripts/train.py
Upload the eval script to Object Store now so it is ready for the inference check later:
s3cmd put eval/infer.py s3://llama4-finetune/scripts/infer.py
Verify everything landed in the bucket:
s3cmd ls --recursive s3://llama4-finetune/
You should see dataset/train.jsonl, dataset/eval.jsonl, scripts/train.py, and the .keep files in the checkpoints/ folder. With the data in place, the next step is wiring up the credentials.
The 700-record dataset is single-domain and fixed-format. Use 500-2000 examples for style changes, 2000 10000 for domain knowledge, and 5000+ for complex behavior. The Civo H100 pipeline does not change: replace train.jsonl, update num_train_epochs if needed, and rerun the Job.
Kubernetes secrets
A Kubernetes Secret stores sensitive values outside the Job manifest and injects them into the pod as environment variables at runtime. Nothing sensitive touches a file on disk or lives inside the container image.
Create the Secret with your five credentials:
kubectl create secret generic training-secrets \--from-literal=HF_TOKEN=YOUR_HF_TOKEN \--from-literal=WANDB_API_KEY=YOUR_WANDB_KEY \--from-literal=OBJECT_STORE_ACCESS_KEY=YOUR_ACCESS_KEY \--from-literal=OBJECT_STORE_SECRET_KEY=YOUR_SECRET_KEY \--from-literal=OBJECT_STORE_ENDPOINT=objectstore.lon1.civo.com
Verify the Secret was created:
kubectl get secret training-secrets
The Job manifest uses secretKeyRef to map each key. At runtime, Kubernetes injects them as environment variables, letting the training script access them directly with no hardcoded values.
The training script
With Unsloth, training configuration lives in a Python script rather than a YAML file. This makes the setup more explicit and easier to debug since every parameter is visible in one place.
Create scripts/train.py. The full file is below, broken into sections.
Imports and credentials
import osos.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"import wandbfrom datasets import load_datasetfrom unsloth import FastLanguageModelfrom trl import SFTTrainer, SFTConfigHF_TOKEN = os.environ["HF_TOKEN"]WANDB_API_KEY = os.environ["WANDB_API_KEY"]
HF_HUB_ENABLE_HF_TRANSFER enables the Rust-based downloader before imports run. Credentials come from Kubernetes Secret environment variables, keeping sensitive data out of the script.
Config
Replace YOUR_HF_USERNAME in HUB_MODEL_ID with your actual Hugging Face username. The trained adapter will be pushed to this repository at the end of training. If this value is wrong or the repository does not exist under your account, the push will fail.
MODEL_NAME = "unsloth/Llama-4-Scout-17B-16E-Instruct-unsloth-dynamic-bnb-4bit"DATASET_PATH = "/data/train.jsonl"OUTPUT_DIR = "/checkpoints/llama4-devops-lora"HUB_MODEL_ID = "YOUR_HF_USERNAME/llama4-devops-lora"MAX_SEQ_LEN = 1024
The -unsloth-dynamic- infix is required. Unsloth’s dynamic 4-bit quantization leaves the MoE router and vision layers unquantized, allowing Scout to fit on a single H100. A -bnb-4bit model name points to a different repository.
W&B initialization
wandb.login(key=WANDB_API_KEY)wandb.init(project="llama4-devops-finetune", name="lora-run-1")
This connects to W&B before the model loads. Loss curves start streaming as soon as the first training step runs.
Model loading
print("Loading model...")model, tokenizer = FastLanguageModel.from_pretrained(model_name=MODEL_NAME,max_seq_length=MAX_SEQ_LEN,load_in_4bit=True,token=HF_TOKEN,)
FastLanguageModel.from_pretrained is Unsloth’s patched loader. It applies optimized kernels, handles Llama 4’s MoE layers natively, and loads weights in 4-bit. The initial model download is about 50GB and typically takes 30-60 minutes.
LoRA adapters
print("Adding LoRA adapters...")model = FastLanguageModel.get_peft_model(model,r=16,lora_alpha=32,lora_dropout=0,bias="none",target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],use_gradient_checkpointing="unsloth",random_state=42,)
The adapter targets only the four attention projection layers, keeping VRAM usage well below 80GB. use_gradient_checkpointing="unsloth" offloads activations to CPU RAM during backpropagation.
Dataset formatting
def format_example(ex):messages = [{"role": "user", "content": ex["instruction"]},{"role": "assistant", "content": ex["output"]},]return {"text": tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=False)}dataset = load_dataset("json", data_files=DATASET_PATH, split="train")dataset = dataset.map(format_example, remove_columns=dataset.column_names)
tokenizer.apply_chat_template formats each record in Llama 4’s native chat template and automatically adds the <|eot|> stop token, so no manual EOS handling is needed.
Trainer
trainer = SFTTrainer(model=model,processing_class=tokenizer,train_dataset=dataset,args=SFTConfig(output_dir=OUTPUT_DIR,dataset_text_field="text",max_length=MAX_SEQ_LEN,packing=True,per_device_train_batch_size=1,gradient_accumulation_steps=8,num_train_epochs=1,learning_rate=1e-4,lr_scheduler_type="cosine",warmup_ratio=0.1,optim="adamw_8bit",bf16=True,logging_steps=10,save_strategy="epoch",save_total_limit=1,report_to="wandb",seed=42,),)trainer.train()
If processing_class raises a TypeError, use tokenizer=tokenizer. packing=True packs multiple records into each sequence window, while adamw_8bit reduces optimizer memory use.
Save and push
print("Saving and pushing adapter...")model.save_pretrained(OUTPUT_DIR)tokenizer.save_pretrained(OUTPUT_DIR)model.push_to_hub(HUB_MODEL_ID, token=HF_TOKEN)tokenizer.push_to_hub(HUB_MODEL_ID, token=HF_TOKEN)print("Training complete.")
The adapter saves locally first, then pushes to Hugging Face Hub. The push happens at the end of training, so a failed run does not leave a partial adapter on the Hub.
Once the file is saved, upload it to Object Store:
s3cmd put scripts/train.py s3://llama4-finetune/scripts/train.py
Any changes to the script require a re-upload before the next run since the job pulls it fresh from the bucket at startup.
Training job
The training job runs as a Kubernetes Job rather than a Deployment. A Job is designed for tasks that run once and are complete, which is exactly what a training run is. Kubernetes automatically retries the Job up to three times if it fails before finishing.
The full manifest lives in kubernetes/training-job.yaml.
Image and startup script
The container uses the official Unsloth image and a bash script to install s3cmd, download the dataset and training script from Object Store, run training, and sync checkpoints back when finished.
apiVersion: batch/v1kind: Jobmetadata:name: llama4-finetunespec:backoffLimit: 3template:spec:restartPolicy: OnFailurecontainers:- name: trainerimage: unsloth/unsloth:latestcommand: ["/bin/bash", "-c"]args:- |set -eecho "Installing s3cmd..."pip install s3cmd -qecho "Downloading dataset from Object Store..."s3cmd --access_key=$OBJECT_STORE_ACCESS_KEY \--secret_key=$OBJECT_STORE_SECRET_KEY \--host=$OBJECT_STORE_ENDPOINT \--host-bucket=$OBJECT_STORE_ENDPOINT \get s3://llama4-finetune/dataset/train.jsonl /data/train.jsonlecho "Downloading training script from Object Store..."s3cmd --access_key=$OBJECT_STORE_ACCESS_KEY \--secret_key=$OBJECT_STORE_SECRET_KEY \--host=$OBJECT_STORE_ENDPOINT \--host-bucket=$OBJECT_STORE_ENDPOINT \get s3://llama4-finetune/scripts/train.py /workspace/train.pyecho "Starting training..."python3 /workspace/train.pyecho "Syncing checkpoints to Object Store..."s3cmd --access_key=$OBJECT_STORE_ACCESS_KEY \--secret_key=$OBJECT_STORE_SECRET_KEY \--host=$OBJECT_STORE_ENDPOINT \--host-bucket=$OBJECT_STORE_ENDPOINT \sync /checkpoints/ s3://llama4-finetune/checkpoints/echo "Training complete."
The Unsloth image is approximately 7GB. There is no model config patching step because Unsloth handles Llama 4's MoE architecture natively through its own patched loader.
Resource requests and node selector
Without nvidia.com/gpu: 1, the pod schedules without GPU access and fails immediately. The node selector pins it to the H100 node.
resources:limits:nvidia.com/gpu: 1env:- name: HF_TOKENvalueFrom:secretKeyRef:name: training-secretskey: HF_TOKEN- name: WANDB_API_KEYvalueFrom:secretKeyRef:name: training-secretskey: WANDB_API_KEY- name: OBJECT_STORE_ACCESS_KEYvalueFrom:secretKeyRef:name: training-secretskey: OBJECT_STORE_ACCESS_KEY- name: OBJECT_STORE_SECRET_KEYvalueFrom:secretKeyRef:name: training-secretskey: OBJECT_STORE_SECRET_KEY- name: OBJECT_STORE_ENDPOINTvalueFrom:secretKeyRef:name: training-secretskey: OBJECT_STORE_ENDPOINT- name: HF_HOMEvalue: /hf-cachevolumeMounts:- name: hf-cachemountPath: /hf-cache- name: workspacemountPath: /workspace- name: datamountPath: /data- name: checkpointsmountPath: /checkpointsvolumes:- name: hf-cacheemptyDir:sizeLimit: 120Gi- name: workspaceemptyDir: {}- name: dataemptyDir: {}- name: checkpointsemptyDir: {}nodeSelector:nvidia.com/gpu.present: "true"
All volumes use emptyDir and are created fresh each run. hf-cache is sized at 120Gi for the model download. The adapter pushes to Hugging Face Hub at the end, so nothing needs to persist.
Launching and monitoring
Apply the manifest:
kubectl apply -f kubernetes/training-job.yaml
Watch the pod come up:
kubectl get pods -w
The pod stays in ContainerCreating while the Unsloth image pulls. Once it is Running, tail the logs:
kubectl logs -f job/llama4-finetune
The job downloads the dataset and script from Object Store, then pulls approximately 50GB of model weights from Hugging Face. Expect 15 to 25 minutes of silence before the first loss line appears.
Reading the W&B loss curve
Open your W&B dashboard and navigate to your project. You should see a live run with a loss curve decreasing over steps and a learning rate curve following the cosine schedule.
train/loss drops from 2.5 to 0.5 by step 40. On a 700-record dataset that is an expectation of memorization of the output format. train/grad_norm dropping from 2.5 to under 0.5 confirms the adapter weights are converging cleanly
Eval
After training, the adapter is pushed to Hugging Face Hub automatically. The eval script runs on an interactive GPU pod on the H100 node since loading the full 50GB base model requires more VRAM than any local machine is likely to have.
Launch an interactive eval pod on the H100 node:
POD_OVERRIDES='{"spec":{"nodeSelector":{"nvidia.com/gpu.present":"true"},"containers":[{"name":"llama4-eval","image":"unsloth/unsloth:latest","stdin":true,"tty":true,"resources":{"limits":{"nvidia.com/gpu":"1"}},"command":["/bin/bash"],"env":[{"name":"HF_TOKEN","value":"YOUR_HF_TOKEN_HERE"}]}]}}'kubectl run llama4-eval --rm -it \--image=unsloth/unsloth:latest \--restart=Never \--overrides="$POD_OVERRIDES" \-- /bin/bash--restart=Never \--overrides=$PodOverrides \-- /bin/bash
Inside the pod, download the eval script from Object Store and run it:
pip install s3cmd -qs3cmd --access_key=YOUR_ACCESS_KEY \--secret_key=YOUR_SECRET_KEY \--host=objectstore.lon1.civo.com \--host-bucket=objectstore.lon1.civo.com \get s3://llama4-finetune/scripts/infer.py /workspace/infer.pypython3 /workspace/infer.py
The full script lives in eval/compare.py. It has four parts.
Imports and prompts
The five held-out prompts are DevOps alerts that the model never saw during training:
Replace YOUR_HF_USERNAME in ADAPTER_PATH with your actual Hugging Face username. This must match the HUB_MODEL_ID value you set in train.py. If it points to the wrong repository, the adapter will not load.
import osimport torchfrom unsloth import FastLanguageModelfrom peft import PeftModelEVAL_PROMPTS = ["High CPU usage detected on payment-service in production. CPU utilization has exceeded 92% for the past 15 minutes. Current value: 97%.","Pod auth-service in prod-us has been in CrashLoopBackOff state for 20 minutes. Restart count: 8.","Service api-gateway in production is returning 34% error rate. Threshold is 5%. Affected endpoints detected in last 10 minutes.","Database connection pool exhausted for order-service in production. Available connections: 2/50. Requests are being queued.","Disk usage on billing-service persistent volume in prod-eu has reached 91%. Volume will be full in approximately 3 hours.",]BASE_MODEL = "unsloth/Llama-4-Scout-17B-16E-Instruct-unsloth-dynamic-bnb-4bit"ADAPTER_PATH = os.environ.get("ADAPTER_PATH", "YOUR_HF_USERNAME/llama4-devops-lora")HF_TOKEN = os.environ.get("HF_TOKEN")
Loading the models
The base model loads through Unsloth's patched loader. The fine-tuned model wraps the base with the LoRA adapter:
def load_base_model():print("Loading base model...")model, tokenizer = FastLanguageModel.from_pretrained(model_name=BASE_MODEL,max_seq_length=1024,load_in_4bit=True,token=HF_TOKEN,)FastLanguageModel.for_inference(model)return model, tokenizerdef load_finetuned_model(base_model, tokenizer):print("Loading fine-tuned adapter...")model = PeftModel.from_pretrained(base_model, ADAPTER_PATH)FastLanguageModel.for_inference(model)return model
Inference function
The prompt runs through the same Llama 4 chat template used during training so the model sees the format it was trained on:
def generate(model, tokenizer, prompt, max_new_tokens=300):messages = [{"role": "user", "content": prompt}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)with torch.no_grad():outputs = model.generate(input_ids=inputs,max_new_tokens=max_new_tokens,temperature=0.1,do_sample=True,)return tokenizer.decode(outputs[0][inputs.shape[-1]:],skip_special_tokens=True).strip()
Eval loop
Each prompt runs against both models and prints side by side:
def run_eval():base_model, tokenizer = load_base_model()finetuned_model = load_finetuned_model(base_model, tokenizer)for i, prompt in enumerate(EVAL_PROMPTS, 1):print(f"\n{'='*60}")print(f"PROMPT {i}:")print(prompt)print(f"\nBASE MODEL:")print(generate(base_model, tokenizer, prompt))print(f"\nFINE-TUNED MODEL:")print(generate(finetuned_model, tokenizer, prompt))print(f"{'='*60}")if __name__ == "__main__":run_eval()
The base model responds with general advice. The fine-tuned model responds with a numbered kubectl runbook in the same structure as the training data. That difference is the adapter working correctly.
The base model responds with headings, possible causes, and general recommendations. None of them are commands you can run immediately.
The fine-tuned model produces six kubectl commands scoped to the exact service and namespace from the alert, ending with an escalation condition. Every step maps directly to the alert that triggered it.
The base model knows what high CPU usage means. The fine-tuned model knows what to do about it.
What's next
The adapter is a lightweight set of weights on top of a frozen base model. A few directions from here:
- Merge and serve: Unsloth can merge the LoRA adapter into the base model to produce a single standalone file. Serve it with vLLM on a Civo GPU node for production inference without the PEFT dependency.
- Retrain on your own data: Replace train.jsonl with your own instruction and output pairs, upload it to the same bucket path, and rerun the job. Everything else stays the same.
- Extend the adapter: The current adapter covers attention layers only at rank 16. For stronger domain adaptation, increase the rank or add more epochs.
Summary
Fine-tuning is the right tool when prompting is no longer enough. A fine-tuned adapter trained on your data adjusts the weights directly so the model follows your examples rather than giving general answers.
Llama 4 Scout's MoE architecture requires a framework that handles it natively. Unsloth is currently the only framework that supports 4-bit QLoRA for Llama 4, keeping the full 109B sparse model within the 80GB HBM3 budget of a single H100.
LoRA keeps the process lightweight. You train a small set of adapter weights while the base model stays frozen. The trained adapter is a few hundred megabytes on top of a 50GB base and can be shared across deployments.
Loss curves confirm the adapter is learning. Held-out prompts confirm it learned the right thing. Both matter.
Software Engineer at GoCardless
Mostafa Ibrahim is a software engineer and technical writer specializing in developer-focused content for SaaS and AI platforms. He currently works as a Software Engineer at GoCardless, contributing to production systems and scalable payment infrastructure.
Alongside his engineering work, Mostafa has written more than 200 technical articles reaching over 500,000 readers. His content covers topics including Kubernetes deployments, AI infrastructure, authentication systems, and retrieval-augmented generation (RAG) architectures.
Share this article