Introduction - HashiCorp Waypoint
Yesterday (15 October 2020) HashiCorp launched an open source tool named Waypoint. Let us try to understand the problem statement and see how Waypoint aims to fix those.
HashiCorp Co-Founder and CTO Mitchell Hashimoto very rightly said during the 2020 HashiConf keynote that traditional software developer lifecycle includes different phases: Code, Test, Build, Deploy, Release, Operate and Measure. Out of this list, commonly-accepted tools for Code, Test, Operate and Measure exist, but the three areas of Build, Deploy and Relese have some challenges that Waypoint aims to solve.
Before we get in to the guide,I've also put together a video walkthrough which is well worth checking out...
On with the guide...
There are build systems that help you build and then deploy to specific platforms, but due to wide variety of platforms out there for application deployment, you will have to use and learn different build/deploy methods specific to the different platforms. The same goes for Release management. Now, in order to have a common workflow with ease of use across all platforms, Hashicorp has introduced Waypoint, with the aim of "providing a consistent workflow to build, deploy and release an application to any platform".
From the docs - "Waypoint is a tool that enables developers to describe how to get their applications from development to production in a single file and deploy using a single command: waypoint up".
Waypoint comes with logs and exec tools that helps you check if there are any issues with the proposed deployment. It is highly pluggable and extensible: in the image below below you can see some of the existing plugins for Docker and Kubernetes that are there by default - and the community can create and contribute more plugins.
So, based on the plugins and with minimal configuration, Waypoint lets you build, deploy and release applications across platforms, saving you the hassle of having to write more lines of configuration files in different languages for different platforms. Here are some of the resources to get you started - Docs Github
In this post I will show you how you can install and use Waypoint to build, deploy and release a sample application to a Civo Kubernetes cluster. If you have not yet signed up, you can do so here - try Civo for 1 month with $250 free credit.
Demo
Deploying a cluster
First, let's create a Civo cluster. It's easiest using the Civo CLI tool, but you can also use the Kubernetes web UI in your Civo account to create it.
Download civo cli from here
Configure to use by providing your API key, which you will find here.
Create cluster using the below command:
civo k3s create --wait --save
The cluster polished-tree (6c8d2b30-496a-46d0-91bf-22124fc14f21) has been created in 2 min 53 sec
Installing Waypoint onto our machine
You can download and install Waypoint on any platform. from your local machine to a virtual machine in the cloud. This example shows Waypoint installation on a CentOS7 box running on Civo.
Installing git, docker, and kubectl. Make sure you have downloaded the cluster configuration in the previous step to ~/.kube/config (if you used the command-line tool above, it will be saved in the correct place).
yum install git -y
yum install docker -y
systemctl start docker
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
mv kubectl /usr/local/bin/
#Check if cluster is runnning
kubectl get nodes
NAME STATUS ROLES AGE VERSION
kube-node-1c23 Ready <none> 110s v1.18.6+k3s1
kube-node-3e04 Ready <none> 110s v1.18.6+k3s1
kube-master-5fdc Ready master 3m52s v1.18.6+k3s1
Install and configure Waypoint for a sample repository:
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo yum -y install waypoint
git clone https://github.com/saiyam1814/waypoint-demo.git
cd waypoint-demo
waypoint install --platform=kubernetes -accept-tos
service/waypoint created
statefulset.apps/waypoint-server created
Waypoint server successfully installed and configured!
The CLI has been configured to connect to the server automatically. This
connection information is saved in the CLI context named "install-1602830444".
Use the "waypoint context" CLI to manage CLI contexts.
The server has been configured to advertise the following address for
entrypoint communications. This must be a reachable address for all your
deployments. If this is incorrect, manually set it using the CLI command
"waypoint server config-set".
Advertise Address: 91.211.154.49:9701
HTTP UI Address: 91.211.154.49:9702
Check that Waypoint is Running
kubectl get pods
NAME READY STATUS RESTARTS AGE
svclb-waypoint-qltwk 2/2 Running 0 93s
svclb-waypoint-cgvdb 2/2 Running 0 93s
svclb-waypoint-d2m4v 2/2 Running 0 93s
waypoint-server-0 1/1 Running 0 93s
Deploying our application
Now that we have Wavpoint installed, let's initiate if for the application that is to be installed.
waypoint init
Initial Waypoint configuration created!
No Waypoint configuration was found in this directory.
A sample configuration has been created in the file "waypoint.hcl". This
file is heavily commented to help you get started.
Once you've setup your initial configuration, run "waypoint init" again to
validate the configuration and initialize your project.
This creates a waypoint.hcl file with basic structure that needs to be configured as per the application. Now for our current example project, there is already a waypoint.hcl present in waypoint-demodirectory
project = "saiyam-waypoint"
app "saiyam-waypoint" {
labels = {
"service" = "saiyam-waypoint",
"env" = "dev"
}
build {
use "docker" {}
registry {
use "docker" {
image = "saiyam911/cd-demo"
tag = "1"
}
}
}
deploy {
use "kubernetes" {
probe_path = "/"
service_port = 8080
}
}
release {
use "kubernetes" {
node_port = 31769
port = 8080
}
}
}
Let's run waypoint init
waypoint init
✓ Configuration file appears valid
✓ Connection to Waypoint server was successful
✓ Project "example-nodejs" and all apps are registered with the server.
✓ Plugins loaded and configured successfully
✓ Authentication requirements appear satisfied.
Project initialized!
You may now call 'waypoint up' to deploy your project or
commands such as 'waypoint build' to perform steps individually.
Three main steps to notice here are build, deploy and release that have to be created and configured in order to install the application onto the cluster. You can also do customizations like provide the kubeconfig file etc. by checking the plugins documentation.
To run all these steps, there is a simple command:
waypoint up
» Building...
✓ Initializing Docker client...
✓ Building image...
│ ---> f520a8e63f23
│ Step 2/4 : COPY . ./
│ ---> 645a76511ab0
│ Step 3/4 : ENV PORT 8080
│ ---> Running in 96cb40982097
│ ---> 7f4e3ebf149d
│ Step 4/4 : CMD python name.py
│ ---> Running in cbe6b9b11306
│ ---> 17cc424bc5cd
│ Successfully built 17cc424bc5cd
✓ Injecting Waypoint Entrypoint...
✓ Tagging Docker image: waypoint.local/saiyam-waypoint:latest => saiyam911/cd-demo:1
✓ Pushing Docker image...
│ 7e453511681f: Layer already exists
│ b544d7bb9107: Layer already exists
│ baf481fca4b7: Layer already exists
│ 3d3e92e98337: Layer already exists
│ 8967306e673e: Layer already exists
│ 9794a3b3ed45: Layer already exists
│ 5f77a51ade6a: Layer already exists
│ e40d297cf5f8: Layer already exists
│ 1: digest: sha256:70f7663523f3aedf044561e079fcb27726132f40227c2da0319c475ecb20cc
│ 5b size: 6178
» Deploying...
✓ Kubernetes client connected to https://91.211.154.49:6443 with namespace default
✓ Creating deployment...
✓ Deployment successfully rolled out!
» Releasing...
✓ Kubernetes client connected to https://91.211.154.49:6443 with namespace default
✓ Creating service...
✓ Service is ready!
The deploy was successful! A Waypoint deployment URL is shown below. This
can be used internally to check your deployment and is not meant for external
traffic. You can manage this hostname using "waypoint hostname."
Release URL: http://172.31.3.114:31769
Deployment URL: https://freely-obliging-pigeon--v1.waypoint.run
Boom!! The application is deployed to the Kubernetes cluster
kubectl get pods
NAME READY STATUS RESTARTS AGE
saiyam-waypoint-01emsbpmqebmamd0dvfacebf2e-849b964bc-rcw2n 1/1 Running 0 69s
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 192.168.128.1 <none> 443/TCP 172m
saiyam-waypoint NodePort 192.168.174.167 <none> 8080:31769/TCP 35s
Access the Service via the URL generated - https://freely-obliging-pigeon--v1.waypoint.run
In just a matter of minutes Application got deployed to the cluster, make some changes in the templates/name.html file and again do waypoint up and you would be able to see a new revision created.
Let's check the waypoint UI to see more info and this is also helpful for debugging purposes -> https://91.211.154.49:9702
Click authenticate and generate temporary token:
waypoint token new
bM152PWkXxfoy4vA51JFhR7LrQefsoZp5gRUr4j25i5Rrf8n3p9gceJg6WxDzXFpjqzY5Qda95b8T3zeBDaf2a38R3rZttABkeyDa
You can see all the build logs, deployments and releases in one place and can look out for issues. Also you can see logs and do exec from the waypoint cli.
waypoint logs
2020-10-16T19:25:34.914Z ZQNCTH: * Serving Flask app "name" (lazy loading)
2020-10-16T19:25:34.914Z ZQNCTH: * Environment: production
2020-10-16T19:25:34.914Z ZQNCTH: WARNING: This is a development server. Do not use it in a
production deployment.
2020-10-16T19:25:34.914Z ZQNCTH: Use a production WSGI server instead.
2020-10-16T19:25:34.914Z ZQNCTH: * Debug mode: off
2020-10-16T19:25:34.918Z ZQNCTH: * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
2020-10-16T19:25:36.342Z ZQNCTH: 192.168.4.1 - - [16/Oct/2020 19:25:36] "GET /
HTTP/1.1" 200 -
2020-10-16T19:25:42.042Z ZQNCTH: 192.168.4.1 - - [16/Oct/2020 19:25:42] "GET /
HTTP/1.1" 200 -
2020-10-16T19:25:46.335Z ZQNCTH: 192.168.4.1 - - [16/Oct/2020 19:25:46] "GET /
HTTP/1.1" 200 -
waypoint exec ls
Connected to deployment v1
Dockerfile README.md main.py prestart.sh uwsgi.ini
LICENSE cloudbuild.yaml name.py templates waypoint.hcl
Overall, a few minutes from code to Kubernetes cluster make a developer's life easy. The product looks promising and will grow over time with community adoption/feedback.
Feel free to try Waypoint out on Civo
