Harvester is an Open source HCI (Hyper-converged software) that enables the creation and management of virtual machines (VM) using the power of Kubernetes. It is based on Kubevirt to provide the virtualization layer for the VMs, and Longhorn as a persistent storage solution for the machines. One of the goals for Harvester is to make it easy for the user to run and manage VMs without knowledge of the tech stack (Kubernetes, Kubevirt, etc.) behind it.

In this tutorial, we will try to understand

  • Harvester Architecture
  • What does Harvester offer?
  • How to install Harvester on Civo Kubernetes
  • Launching a VM using Harvester
  • Install K3s on a newly launched VM created via Harvester to experience "Kube-ception"

For more context, listen to this stream by Sheng from SUSE to learn more about Harvester and its bare metal setup.

Harvester Architecture

Major componenets of Harvester are Kubevirt, Longhorn, and Multus for putting together VLANs (Virtual Local Area Network) and the management network.

Harvester Architecture Illustration

As you can see this is Hyperconverged Infrastructure mode architecture, where you have

  • Bare-metal nodes
  • K3OS running on top of the nodes (note that this might change when Harvester goes into General Availability)
  • Longhorn and Kubevirt
  • VM creation
  • VMs are attached to a VLAN and management network, or just the management network

Harvester Features/Use-cases

Harvester use cases

Here are some notable features of the Harvester (from the docs)

  • VM lifecycle management including SSH-Key injection, Cloud-init and, graphic and serial port console
  • VM live migration support
  • VM backup and restore support
  • Distributed block storage
  • Multiple NICs in the VM connecting to the management network or VLANs
  • Virtual Machine and cloud-init templates
  • Built-in Rancher integration and the Harvester node driver
  • PXE/iPXE boot support

Raw Block Device Support

With the release of version 0.2.0, Harvester has already shown impressive improvements, like the removal of a dependency on Minio dependency, as Longhorn is used to manage the VM image using the Backing Image feature. Now there is less time in importing the image and more time on the first VM boot from that image as it gets stored in Longhorn.

VM live migration and backup support

VM Live Migration and Backup support are also key features in v0.2.0. You can now migrate a VM from one node to another if needed.

Backup and migration options in Harvester

You also have a feature to take backups of virtual machines to a target outside of the cluster. In order to use this, you need to add a backup target as S3 compatible endpoint or NFS server.

Backup target settings in Harvester

Rancher integration and PXE/iPXE boot support

With Harvester v0.2.0 you get PXE boot support to provision bare metal nodes with Operating Systems in an automated way, and Rancher integration to create Kubernetes clusters on top of your bare metal nodes.

Harvester Installation

Harvester can be installed in two modes: ISO mode and App mode

  • The HCI mode can be used to install Harvester directly on bare-metal to create a Harvester cluster. It will result in this layer of architecture:

Harvester HCI architecture illustration

  • App mode is used to install Harvester as a Helm chart onto an existing Kubernetes cluster

Below we will go through the App Mode installation method, as we will be installing Harvester onto a Civo Kubernetes cluster.

Create Civo Kubernetes cluster

Create a new cluster from the Kubernetes menu on Civo (you can also use Civo CLI). Once ready, you should see the cluster with ready nodes.

I created a big Extra Large Nodes cluster:

New Kubernetes Cluster on Civo

Cluster running on Civo

Make sure you have both Helm and kubectl installed on your machine, and the KUBECONFIG file for your cluster downloaded so that you can run kubectl get nodes and get details of the cluster you just created:

$ kubectl get nodes
NAME                                    STATUS   ROLES    AGE     VERSION
k3s-harvester-a2481fd0-node-pool-8553   Ready    <none>   2m34s   v1.20.2+k3s1
k3s-harvester-a2481fd0-node-pool-6a18   Ready    <none>   2m34s   v1.20.2+k3s1
k3s-harvester-a2481fd0-node-pool-6cd6   Ready    <none>   2m34s   v1.20.2+k3s1
k3s-harvester-a2481fd0-node-pool-e8be   Ready    <none>   2m33s   v1.20.2+k3s1
k3s-harvester-a2481fd0-node-pool-65d3   Ready    <none>   2m33s   v1.20.2+k3s1
k3s-harvester-a2481fd0-node-pool-8b06   Ready    <none>   2m33s   v1.20.2+k3s1

Install Harvester via helm chart

Create a namespace for Harvester:

$ kubectl create ns harvester-system
namespace/harvester-system created

Install Harvester

$ helm install harvester harvester --namespace harvester-system --set longhorn.enabled=true,minio.persistence.storageClass=longhorn,service.harvester.type=NodePort

NAME: harvester
LAST DEPLOYED: Tue May 4 21:27:14 2021
NAMESPACE: harvester-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The Harvester has been installed into "harvester-system" namespace with "harvester" as the release name.
- [x] KubeVirt Operator
- [x] KubeVirt Resource named "kubevirt"
- [x] KubeVirt Containerized Data Importer Operator
- [x] KubeVirt Containerized Data Importer(CDI) Resource named "cdi"
- [x] Minio
- [x] Longhorn
- [ ] Multus-cni
Please make sure there is a default StorageClass in the Kubernetes cluster.
To learn more about the release, try:
$ helm status harvester
$ helm get all harvester
$ helm history harvester -n harvester-system
REVISION UPDATED                  STATUS   CHART           APP VERSION DESCRIPTION
1        Fri May  7 11:17:09 2021 deployed harvester-0.2.0 v0.2.0      Install complete

In the harvester-system namespace, a bunch of operators will be deployed:

$ kubectl get deploy -n harvester-system
NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
harvester-network-controller-manager   2/2     2            2           16d
harvester                              3/3     3            3           16d
virt-operator                          1/1     1            1           16d
cdi-apiserver                          1/1     1            1           16d
cdi-uploadproxy                        1/1     1            1           16d
virt-api                               2/2     2            2           16d
cdi-operator                           1/1     1            1           16d
cdi-deployment                         1/1     1            1           16d
virt-controller                        2/2     2            2           16d

You can access the harvester UI via NodePort or you can also create an ingress object for the same.

$ kubectl get svc -n harvester-system
NAME                          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
harvester-minio               ClusterIP   10.43.254.67    <none>        9000/TCP         16d
harvester                     NodePort    10.43.18.240    <none>        8443:31594/TCP   16d
cdi-api                       ClusterIP   10.43.81.71     <none>        443/TCP          16d
cdi-prometheus-metrics        ClusterIP   10.43.44.66     <none>        443/TCP          16d
cdi-uploadproxy               ClusterIP   10.43.63.49     <none>        443/TCP          16d
kubevirt-prometheus-metrics   ClusterIP   10.43.135.213   <none>        443/TCP          16d
virt-api                      ClusterIP   10.43.206.174   <none>        443/TCP          16d
kubevirt-operator-webhook     ClusterIP   10.43.18.154    <none>        443/TCP          16d

Launching a VM from Harvester UI

First, log in to the Harvester UI with default credentials (admin/password)

Harvester login screen

You should see Harvester 0.2.0 with default view:

Harvester Dashboard

We will need to add an image from which the VMs can be created.

Let's add an Ubuntu image using https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img in the "Image: Create" menu.

adding an image to our Harvester options

Stored Ubuntu image on Harvester

Now this image can be used to create the Virtual Machine!

Create VM

Create the virtual machine from the Harvester user interface under "Virtual Machine: Create"

VM Creation on Harvester

After selecting CPU, Memory, Image, put in the cloud config in the Advanced Options screen

VM Cloud Config on Harvester

Once you click to create the VM will be created successfully:

Virtual Machine creation on Harvester successful

VM Details screen on Harvester

Kube-ception Mode

Now that the VM is created successfully you can login to the console directly from the UI. Once you select the console option from the VM you will be prompted with login and password.

Use credentials ubuntu/password to log in (which we set in the Cloud Config above)

Ubuntu terminal login

Successful log in to Ubuntu VM

install K3s

Install K3s on the newly created VM using the curl command

curl -sfL https://get.k3s.io | sh -

You can see the beautiful inception of Kubernetes within Kubernetes within Kubernetes:

Kubeception

Wrapping up

Harvester looks to be promising open-source hyperconverged infrastructure (HCI) software with the combination of technologies that it uses and the number of features that are already built. This will be very well adopted and used by the community in my opinion.

Let us know on Twitter @Civocloud and @SaiyamPathak if you try Harvester out on Civo Kubernetes or even bare metal servers.