Kubecost is a cost management platform built on open source that helps teams running Kubernetes maximize the efficiency of their spending.

Many teams using Kubernetes struggle to understand where their cloud spend is going, i.e. which team, project, or application is spending how much and on which infrastructure services. This is where Kubecost comes in: based on open source tools for collecting and aggregating the right set of metrics from the cluster and workloads, Kubecost can tell you the cost for the microservice or application that you are running in the cluster.

Kubecost helps developers with systems to track overruns and nudge developers and tools to show the savings of proper capacity management.

Kubecost dashboard showing an example spending report

The key focus areas for Kubecost are:

  • Spend visibility: insights on the amounts teams are spending on their workloads.

  • Insights and optimization: after getting the visibility, infrastructure optimization and application performance improvements are the next step.

  • Governance: after finding the right balance between the cost spend and performance, running it over time with changing equations falls under this focus area.

Kubecost architecture

Kubecost architecture diagram, showing Kubecost reading from Prometheus and serving this data through its own front-end and cost model calculations

The main parts of the architecture are:

  • Kubecost Pod: This will take the metrics from Kubernetes API and use the billing data from the configuration provided by the user or taken automatically from the public clouds. With this information, the pod generates a set of metrics that Prometheus can scrape for use.
  • Kube-state metrics - Kubernetes API metrics, e.g. resource requests.
  • node-exporter - node-level utilization metrics.
  • cadvisor - for utilization metrics (CPU/Memory for containers).
  • network metrics: Daemonset for collecting metrics.
  • Prometheus - metrics data store.

Kubecost on Civo Kubernetes

Kubecost can be installed on any Kubernetes distribution. For this tutorial we will install Kubecost on a Civo Kubernetes cluster.

Cluster creation

You can create the cluster from the UI or from the Civo CLI. For this tutorial, let's create using the CLI.

$ civo k3s create kubecost-demo
The cluster kubecost-demo (9fe52857-f03e-451a-9c69-7125b06dcbbd) has been created

Above will create a 3 node cluster named kubecost-demo

We will need to get the Kubeconfig for the cluster and save to our desired location. If you do not specify a path, it will save it to ~/.kube/config.

$ civo k3s config kubecost-demo --save --local-path /Users/saiyampathak/civo/test/kubecost-demo.config

Access your cluster with:
KUBECONFIG=/Users/saiyampathak/civo/test/kubecost-demo.config kubectl get node

Let's make sure that kubectl knows to use our cluster's configuration file:

$ export KUBECONFIG=/Users/saiyampathak/civo/test/kubecost-demo.config

$ kubectl get nodes
NAME                                   STATUS   ROLES    AGE   VERSION
k3s-kubecost-demo-0edfef9e-node-pool-b3fe   Ready    <none>   39s   v1.20.2+k3s1
k3s-kubecost-demo-0edfef9e-node-pool-e43a   Ready    <none>   38s   v1.20.2+k3s1
k3s-kubecost-demo-0edfef9e-node-pool-9209   Ready    <none>   38s   v1.20.2+k3s1

Kubecost installation

Kubecost can be installed easily via helm or directly with a kubectl apply command For this tutorial we will install via helm

helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm upgrade -i --create-namespace kubecost kubecost/cost-analyzer --namespace kubecost --set kubecostToken="aGVsbUBrdWJlY29zdC5jb20=xm343yadf98"

NAME: kubecost
LAST DEPLOYED: Mon Jul 19 23:01:32 2021
NAMESPACE: kubecost
STATUS: deployed
--------------------------------------------------Kubecost has been successfully installed. When pods are Ready, you can enable port-forwarding with the following command:

    kubectl port-forward --namespace kubecost deployment/kubecost-cost-analyzer 9090

Next, navigate to http://localhost:9090 in a web browser.

You will be able to see all the components deployed in the kubecost namespace

kubectl get pods -n kubecost
NAME                                           READY   STATUS    RESTARTS   AGE
kubecost-prometheus-node-exporter-7twvw        1/1     Running   0          2m40s
kubecost-prometheus-node-exporter-846qc        1/1     Running   0          2m40s
kubecost-prometheus-node-exporter-xcnt5        1/1     Running   0          2m40s
kubecost-kube-state-metrics-6c4df87fb6-bwbkd   1/1     Running   0          2m39s
kubecost-grafana-779599f58d-qlbll              3/3     Running   0          2m39s
kubecost-prometheus-server-6b5884f9d5-qjvzw    2/2     Running   0          2m39s
kubecost-cost-analyzer-69f485d5bd-d2rvn        3/3     Running   0          2m39s

Setting the right parameters

Once Kubecost is deployed onto the cluster, you can forward the port from the cluster and access the kubecost UI at http://localhost:9090 with teh following command:

kubectl port-forward --namespace kubecost deployment/kubecost-cost-analyzer 9090

You will be able to see the cluster on the page:

Cluster information shown on Kubecost's user interface

By default, Kubecost has some cost values set for the workloads in terms of CPU/Memory, etc. Since this is a Civo Kubernetes cluster, we will adjust the cost accordingly in the settings section.

In the settings section enable custom pricing:

"Enable custom pricing" toggle in Kubecost settings

We will be using the pricing for a Civo Kubernetes cluster of this size, as mentioned on the pricing page.

Edit in the costs based on charges from the pricing page. The specific charges for RAM and persistent volumes come out as: Storage per GB - $0.10 Ram per GB - $4

Adding custom values to Kubecost's custom pricing panel

After hitting save, let's start exploring Kubecost features

Kubecost Features

Now we have Kubecost deployed, we can start seeing the overall cost for the workloads, namespaces etc. Let's explore different Kubecost features.

Cost overview

Overview page of Kubecost, showing graphs and projected cost information This is the default overview page when you open the kubecost dashboard for a particular cluster. It calculates our costs based on custom pricing inputs we provided in the settings for civo Kubernetes. It provides a high-level view of cluster spend, efficiency, and key cost drivers.

Cost Allocation

Below is the cost allocation view per namespace where the first six columns show the actual cost within the Kubernetes cluster. Kubecost can present cost by any Kubernetes concept, so you can view cost by label, annotation, service, and more.

Cost allocation view showing cost per namespace over the past seven days

Next is the shared costs that can be defined in the settings. There can be various use cases for this, like having a cost shared in multiple namespaces with a specific label, or evenly distributing the cost for a particular product/service for all the teams.

Kubecost shared costs settings page

You can also define external costs that the Kubernetes cluster is consuming. After drilling down into a particular namespace you will be able to see the pods running in that namespace and the costs associated with them.

Cost view showing impact of particular pods in a namespace on the cost accrued

Cost view showing the graph of the cumulative cost accrued by a particular deployment


Assets give you the total cost view and adjustments if you have defined any discounts in the settings.

Stacked bar graph in Kubecost showing costs by type and asset


All the allocations views can be saved as reports and shared with the teams and can be also useful for keeping a track on the spend per team or whatever criteria chosen from the filters provided.

Kubecost report creation screen


You can set up Alerts and notifications based on the thresholds defined.

Kubecost alert management dashboard

Kubecost-kubectl integration

Kubecost has a krew plugin as well. As the repository says it:

provides command-line access to Kubernetes cost allocation metrics via the kubecost APIs. It allows developers, devops teams, and others to quickly determine the cost & efficiency for any Kubernetes workload.

You need to have krew installed on your system, after which you can install the cost plugin using:

kubectl krew install cost
Updated the local copy of plugin index.
Installing plugin: cost
Installed plugin: cost
 | Use this plugin:
 |  kubectl cost
 | Documentation:
 |  https://github.com/kubecost/kubectl-cost
 | Caveats:
 | \
 |  | Requires Kubecost (a cluster-side daemon) to be installed in your cluster.
 |  | See https://www.kubecost.com/install for installation instructions.
 | /
WARNING: You installed plugin "cost" from the krew-index plugin repository.
   These plugins are not audited for security by the Krew maintainers.
   Run them at your own risk.

Now you can use this plugin to find the cost of a namespace or a pod directly using a kubectl command:

 kubectl cost pod -n kubecost
| NAMESPACE | POD                                          | MONTHLY RATE (ALL) | COST EFFICIENCY |
| kubecost  | kubecost-cost-analyzer-69f485d5bd-d2rvn      |           3.844531 |        0.000000 |
|           | kubecost-prometheus-server-6b5884f9d5-qjvzw  |           3.200000 |        0.000000 |
|           | kubecost-grafana-779599f58d-qlbll            |           0.000000 |        0.000000 |
|           | kubecost-prometheus-node-exporter-846qc      |           0.000000 |        0.000000 |
|           | kubecost-prometheus-node-exporter-xcnt5      |           0.000000 |        0.000000 |
|           | kubecost-kube-state-metrics-6c4df87fb6-bwbkd |           0.000000 |        0.000000 |
|           | kubecost-prometheus-node-exporter-7twvw      |           0.000000 |        0.000000 |
| SUMMED    |                                              |       USD 7.044531 |                 |

This becomes very handy for the CLI lovers to get the cost of particular resources immediately with additional parameters.

Wrapping up

Kubecost looks to be a great open-source tool to measure costs for your workloads running on Kubernetes cluster. The best part is that you can configure the costs depending on where the cluster is running, and that it can be used to measure the cost of any Kubernetes cluster given its customisable pricing settings.

Let us know on Twitter @Civocloud and @SaiyamPathak if you try Kubecost out on Civo Kubernetes or even bare metal servers.

Live stream recording