Introduction - Hashicorp Waypoint

Yesterday (15 October 2020) Hashicorp launched an open source tool named Waypoint. Let us try to understand the problem statement and see how Waypoint aims to fix those.

HashiCorp Co-Founder and CTO Mitchell Hashimoto very rightly said during the 2020 HashiConf keynote that traditional software developer lifecycle includes different phases: Code, Test, Build, Deploy, Release, Operate and Measure. Out of this list, commonly-accepted tools for Code, Test, Operate and Measure exist, but the three areas of Build, Deploy and Relese have some challenges that Waypoint aims to solve.

There are build systems that help you build and then deploy to specific platforms, but due to wide variety of platforms out there for application deployment, you will have to use and learn different build/deploy methods specific to the different platforms. The same goes for Release management. Now, in order to have a common workflow with ease of use across all platforms, Hashicorp has introduced Waypoint, with the aim of "providing a consistent workflow to build, deploy and release an application to any platform".

From the docs - "Waypoint is a tool that enables developers to describe how to get their applications from development to production in a single file and deploy using a single command: waypoint up".

Waypoint comes with logs and exec tools that helps you check if there are any issues with the proposed deployment. It is highly pluggable and extensible: in the image below below you can see some of the existing plugins for Docker and Kubernetes that are there by default - and the community can create and contribute more plugins.

Waypoint Plugins

So, based on the plugins and with minimal configuration, Waypoint lets you build, deploy and release applications across platforms, saving you the hassle of having to write more lines of configuration files in different languages for different platforms. Here are some of the resources to get you started - Docs Github

In this post I will show you how you can install and use Waypoint to build, deploy and release a sample application to a Civo Kubernetes cluster. If you have not yet signed up, you can do so here - you'll get $70 a month credit while the service is in beta.

Demo

Deploying a cluster

First, let's create a Civo cluster. It's easiest using the Civo CLI tool, but you can also use the Kubernetes web UI in your Civo account to create it.

Download civo cli from here

Configure to use by providing your API key, which you will find here.

Create cluster using the below command:

civo k3s create --wait --save                           
The cluster polished-tree (6c8d2b30-496a-46d0-91bf-22124fc14f21) has been created in 2 min 53 sec

Installing Waypoint onto our machine

You can download and install Waypoint on any platform. from your local machine to a virtual machine in the cloud. This example shows Waypoint installation on a CentOS7 box running on Civo.

Installing git, docker, and kubectl. Make sure you have downloaded the cluster configuration in the previous step to ~/.kube/config (if you used the command-line tool above, it will be saved in the correct place).

yum install git -y

yum install docker -y
systemctl start docker 

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl 
mv kubectl /usr/local/bin/

#Check if cluster is runnning 
kubectl get nodes
NAME               STATUS     ROLES    AGE     VERSION
kube-node-1c23     Ready      <none>   110s    v1.18.6+k3s1
kube-node-3e04     Ready      <none>   110s    v1.18.6+k3s1
kube-master-5fdc   Ready      master   3m52s   v1.18.6+k3s1

Install and configure Waypoint for a sample repository:


sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo yum -y install waypoint

git clone https://github.com/saiyam1814/waypoint-demo.git
cd waypoint-demo

waypoint install --platform=kubernetes -accept-tos
service/waypoint created
statefulset.apps/waypoint-server created
Waypoint server successfully installed and configured!

The CLI has been configured to connect to the server automatically. This
connection information is saved in the CLI context named "install-1602830444".
Use the "waypoint context" CLI to manage CLI contexts.

The server has been configured to advertise the following address for
entrypoint communications. This must be a reachable address for all your
deployments. If this is incorrect, manually set it using the CLI command
"waypoint server config-set".

Advertise Address: 91.211.154.49:9701
HTTP UI Address: 91.211.154.49:9702

Check that Waypoint is Running

kubectl get pods 
NAME                   READY   STATUS    RESTARTS   AGE
svclb-waypoint-qltwk   2/2     Running   0          93s
svclb-waypoint-cgvdb   2/2     Running   0          93s
svclb-waypoint-d2m4v   2/2     Running   0          93s
waypoint-server-0      1/1     Running   0          93s

Deploying our application

Now that we have Wavpoint installed, let's initiate if for the application that is to be installed.

 waypoint init
Initial Waypoint configuration created!
No Waypoint configuration was found in this directory.

A sample configuration has been created in the file "waypoint.hcl". This
file is heavily commented to help you get started.

Once you've setup your initial configuration, run "waypoint init" again to
validate the configuration and initialize your project.

This creates a waypoint.hcl file with basic structure that needs to be configured as per the application. Now for our current example project, there is already a waypoint.hcl present in waypoint-demodirectory

project = "saiyam-waypoint"

app "saiyam-waypoint" {
  labels = {
      "service" = "saiyam-waypoint",
      "env" = "dev"
  }

  build {
   use "docker" {}
    registry {
        use "docker" {
          image = "saiyam911/cd-demo"
          tag = "1"
  }
    }
 }

  deploy { 
    use "kubernetes" {
        probe_path = "/"
        service_port = 8080
}
  }

  release {
    use "kubernetes" {
      node_port = 31769
      port = 8080
    }
  }
}

Let's run waypoint init

waypoint init
✓ Configuration file appears valid
✓ Connection to Waypoint server was successful
✓ Project "example-nodejs" and all apps are registered with the server.
✓ Plugins loaded and configured successfully
✓ Authentication requirements appear satisfied.

Project initialized!

You may now call 'waypoint up' to deploy your project or
commands such as 'waypoint build' to perform steps individually.

Three main steps to notice here are build, deploy and release that have to be created and configured in order to install the application onto the cluster. You can also do customizations like provide the kubeconfig file etc. by checking the plugins documentation.

To run all these steps, there is a simple command:

waypoint up

» Building...
✓ Initializing Docker client...
✓ Building image...
 │  ---> f520a8e63f23
 │ Step 2/4 : COPY . ./
 │  ---> 645a76511ab0
 │ Step 3/4 : ENV PORT 8080
 │  ---> Running in 96cb40982097
 │  ---> 7f4e3ebf149d
 │ Step 4/4 : CMD python name.py
 │  ---> Running in cbe6b9b11306
 │  ---> 17cc424bc5cd
 │ Successfully built 17cc424bc5cd
✓ Injecting Waypoint Entrypoint...
✓ Tagging Docker image: waypoint.local/saiyam-waypoint:latest => saiyam911/cd-demo:1
✓ Pushing Docker image...
 │ 7e453511681f: Layer already exists
 │ b544d7bb9107: Layer already exists
 │ baf481fca4b7: Layer already exists
 │ 3d3e92e98337: Layer already exists
 │ 8967306e673e: Layer already exists
 │ 9794a3b3ed45: Layer already exists
 │ 5f77a51ade6a: Layer already exists
 │ e40d297cf5f8: Layer already exists
 │ 1: digest: sha256:70f7663523f3aedf044561e079fcb27726132f40227c2da0319c475ecb20cc
 │ 5b size: 6178

» Deploying...
✓ Kubernetes client connected to https://91.211.154.49:6443 with namespace default
✓ Creating deployment...
✓ Deployment successfully rolled out!

» Releasing...
✓ Kubernetes client connected to https://91.211.154.49:6443 with namespace default
✓ Creating service...
✓ Service is ready!

The deploy was successful! A Waypoint deployment URL is shown below. This
can be used internally to check your deployment and is not meant for external
traffic. You can manage this hostname using "waypoint hostname."

 Release URL: http://172.31.3.114:31769
Deployment URL: https://freely-obliging-pigeon--v1.waypoint.run

Boom!! The application is deployed to the Kubernetes cluster

kubectl get pods
NAME                                                         READY   STATUS    RESTARTS   AGE
saiyam-waypoint-01emsbpmqebmamd0dvfacebf2e-849b964bc-rcw2n   1/1     Running   0          69s

kubectl get svc
NAME              TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)          AGE
kubernetes        ClusterIP   192.168.128.1     <none>        443/TCP          172m
saiyam-waypoint   NodePort    192.168.174.167   <none>        8080:31769/TCP   35s

Access the Service via the URL generated - https://freely-obliging-pigeon--v1.waypoint.run Your Alt Text Your Alt Text

In just a matter of minutes Application got deployed to the cluster, make some changes in the templates/name.html file and again do waypoint up and you would be able to see a new revision created. Let's check the waypoint UI to see more info and this is also helpful for debugging purposes -> https://91.211.154.49:9702

Your Alt Text

Click authenticate and generate temporary token: Your Alt Text

waypoint token new
bM152PWkXxfoy4vA51JFhR7LrQefsoZp5gRUr4j25i5Rrf8n3p9gceJg6WxDzXFpjqzY5Qda95b8T3zeBDaf2a38R3rZttABkeyDa

Your Alt Text

Your Alt Text

You can see all the build logs, deployments and releases in one place and can look out for issues. Also you can see logs and do exec from the waypoint cli.

waypoint logs 
2020-10-16T19:25:34.914Z ZQNCTH:  * Serving Flask app "name" (lazy loading)
2020-10-16T19:25:34.914Z ZQNCTH:  * Environment: production
2020-10-16T19:25:34.914Z ZQNCTH:    WARNING: This is a development server. Do not use it in a
production deployment.
2020-10-16T19:25:34.914Z ZQNCTH:    Use a production WSGI server instead.
2020-10-16T19:25:34.914Z ZQNCTH:  * Debug mode: off
2020-10-16T19:25:34.918Z ZQNCTH:  * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
2020-10-16T19:25:36.342Z ZQNCTH: 192.168.4.1 - - [16/Oct/2020 19:25:36] "GET /
HTTP/1.1" 200 -
2020-10-16T19:25:42.042Z ZQNCTH: 192.168.4.1 - - [16/Oct/2020 19:25:42] "GET /
HTTP/1.1" 200 -
2020-10-16T19:25:46.335Z ZQNCTH: 192.168.4.1 - - [16/Oct/2020 19:25:46] "GET /
HTTP/1.1" 200 -
waypoint exec ls
Connected to deployment v1
Dockerfile  README.md        main.py  prestart.sh  uwsgi.ini
LICENSE     cloudbuild.yaml  name.py  templates    waypoint.hcl

Overall, a few minutes from code to Kubernetes cluster make a developer's life easy. The product looks promising and will grow over time with community adoption/feedback.

Feel free to try Waypoint out on Civo - join our beta tester community and get a discount of $70 per month from your usage - that's enough to run a 3-node Kubernetes cluster!