In this blog post, we will look at Skaffold & how it can be used to
improve the developer workflow for Kubernetes applications.
What is Skaffold?
Skaffold is a Google container tool, which
handles the workflow for building, pushing and deploying the
Kubernetes applications.
How to install Skaffold:
Skaffold comes as a client-side only single binary cli tool. Its
installation is as simple as downloading the binary & copying it to
/usr/local/bin
. Detailed Skaffold Installation steps.
We will look at how we can use Skaffold to build & deploy
Botkube.
We can generate simple skaffold.yaml
configuration file with skaffold
init
command in the project’s root directory.
skaffold init
command will take us through a wizard to generate a
simple skaffold.yaml
file, as below :
$ skaffold init
apiVersion: skaffold/v2alpha3
kind: Config
metadata:
name: botkube
build:
artifacts:
- image: infracloudio/botkube
context: build
deploy:
kubectl:
manifests:
- deploy-all-in-one-tls.yaml
- deploy-all-in-one.yaml
Do you want to write this configuration to skaffold.yaml? [y/n]: y
Configuration skaffold.yaml was written
You can now run [skaffold build] to build the artifacts
or [skaffold run] to build and deploy
or [skaffold dev] to enter development mode, with auto-redeploy
But, we need to make some more changes to the generated skaffold.yaml
file to make the build & deploy work with Botkube’s file structure. The
changes we need to make for build the step work are:
>>>
artifacts:
- image: infracloudio/botkube
context: build
>>>
<<<
artifacts:
- image: infracloudio/botkube
context: .
docker:
dockerfile: build/Dockerfile
<<<
Image tagging policies in Skaffold:
We can set a Docker image tagging policy with below configuration in
build section:
<<<
build:
local:
push: false # Set true to a push built images to remote Docker repository
tagPolicy:
gitCommit: {}
<<<
Skaffold supports below tag policies:
- Tag by
git commit
- Tag by
current date & time
- Tag by
environment variables based template
- Tag by
digest of the Docker image
The updated skaffold.yaml
file will look like this:
apiVersion: skaffold/v2alpha3
kind: Config
metadata:
name: botkube
build:
local:
push: false # Setting false will not push the image to remote repository
tagPolicy:
gitCommit: {}
artifacts:
- image: botkubetest
context: .
docker:
dockerfile: build/Dockerfile
deploy:
kubectl:
manifests:
- deploy-all-in-one.yaml
Skaffold workflow pipeline can be run in below modes:
-
skaffold run
– to build & deploy once
-
skaffold dev
– to trigger the watch loop build & deploy
workflow with cleanup on exit
-
skaffold debug
– to run a pipeline in debug mode
Now, we can run skaffold dev
command to build the Docker image &
deploy to the Kubernetes cluster (uses current Kubernetes context). We
can also add kubeContext
parameter in deploy section of
skaffold.yaml
to explicitly specify Kubernetes context.
NOTE : For Minikube, we need to run eval $(minikube docker-env)
&
set imagePullPolicy
to Never
, so that Minikube can get access to the
locally built Docker images, else we will get ImagePullBackOff
error.
The skaffold dev
output looks like below & Botkube will be deployed to
Minikube:
$ skaffold dev
Listing files to watch...
- botkubetest
Generating tags...
- botkubetest -> botkubetest:v0.10.0-6-gf90361d-dirty
Checking cache...
- botkubetest: Not found. Building
Found [minikube] context, using local docker daemon.
Building [botkubetest]...
Sending build context to Docker daemon 9.087MB
Step 1/17 : FROM golang:1.12-alpine3.10 AS BUILD-ENV
---> 6d6865cf688e
...
...
...
...
Step 17/17 : ENTRYPOINT /go/bin/botkube
---> Using cache
---> a610c9924625
Successfully built a610c9924625
Successfully tagged botkubetest:v0.10.0-6-gf90361d-dirty
Tags used in deployment:
- botkubetest -> botkubetest:a610c9924625c825bc7775ab959921adcf075f83c64474ef1899174d884821e2
local images can't be referenced by digest. They are tagged and referenced by a unique ID instead
Starting deploy...
- namespace/botkube created
- configmap/botkube-configmap created
- secret/botkube-communication-secret created
- serviceaccount/botkube-sa created
- clusterrole.rbac.authorization.k8s.io/botkube-clusterrole created
- clusterrolebinding.rbac.authorization.k8s.io/botkube-clusterrolebinding created
- deployment.apps/botkube created
Watching for changes...
Supported Builders & Deployers in Skaffold:
For deploying the applications to a remote Kubernetes cluster, we need
to either push locally built images to remote Docker repository & pull
image in remote cluster or use in-cluster builders like kaniko.
We can to set push: true
in skaffold.yaml
file in order to push the
built image to remote Docker repository. Skaffold uses local Docker
login credentials to push to the remote Docker repository. Also, we need
to create a custom image pull secret in the cluster & provide it in the
Kubenetes deployment specification to be able to pull the images in the
cluster.
Skaffold currently supports following Builders:
- Dockerfile
- locally with Docker
- in-cluster with Kaniko
- on cloud with Google Cloud Build
- Jib Maven and Gradle
- locally
- on cloud with Google Cloud Build
- Bazel locally
- Cloud Native Buildpacks
- locally with Docker
- on cloud with Google Cloud Build
- Custom script
- locally
- in-cluster
source https://skaffold.dev/docs/
Skaffold currently supports following Deployers:
- Kubernetes Command-Line Interface (kubectl)
- Helm
- Kustomize
In order to use Helm as Deployer, we need to update the skaffold.yaml
file as below:
apiVersion: skaffold/v2alpha3
kind: Config
metadata:
name: botkube
build:
local:
push: false # Setting false will not push the image to remote repository
tagPolicy:
gitCommit: {}
artifacts:
- image: botkubetest
context: .
docker:
dockerfile: build/Dockerfile
deploy:
statusCheckDeadlineSeconds: 600 # Period Skaffold waits for deployment to stabilise
kubeContext: minikube # Context to deploy the application to
helm:
flags:
upgrade:
[--timeout=600] # Helm upgrade timeout
install:
[--timeout=600] # Helm install timeout
releases:
- name: botkube
chartPath: helm/botkube/
skipBuildDependencies: true # Skip Helm dependencies build
namespace: botkube
version: v0.10.0
setValues:
communications.webhook.enabled: true
communications.webhook.url: <WEBHOOK_URL>
config.settings.clustername: <CLUSTER_NAME>
#image.repository: infracloudio/botkube
#image.tag: v0.10.0
imageStrategy:
helm: {}
As per Skaffold docs, when we run skaffold dev
, following steps are
carried out:
1) Collects and watches your source code for changes
2) Syncs files directly to pods if user marks them as syncable
3) Builds artifacts from the source code
4) Tests the built artifacts using container-structure-tests
5) Tags the artifacts
6) Pushes the artifacts
7) Deploys the artifacts
8) Monitors the deployed artifacts
9) Cleans up deployed artifacts on exit (Ctrl+C)
source: https://skaffold.dev/docs/
Source
Profiles in Skaffold:
Skaffold supports multiple profiles to allow deployment to different
environment/clusters. Profiles can be specified in skaffold.yaml
file
as follows:
apiVersion: skaffold/v2alpha3
kind: Config
profiles:
- name: minikube
activation:
- kubeContext: minikube
- env: ENV=local_dev
build:
...
# Build configuration for minikube profile
...
deploy:
...
# Deploy configuration for minikube profile
...
- name: eks-cluster
activation:
- kubeContext: <EKS_CLUSTER_CONTEXT_NAME>
- env: ENV=integration
build:
...
# Build configuration for eks-cluster profile
...
deploy:
...
# Deploy configuration for eks-cluster profile
...
Below command can be used to run Skaffold with different profiles:
skaffold [run|dev] -p <profile_name> -f <skaffold_config_yaml_file>
We can use Skaffold in CI/CD pipeline for building, deploying separately
or both as below:
Build:
We can just build the project with below command.
skaffold build [-p <profile>] > build_result.json
build_result.json
file mainly contains image tags that are generated,
among other things.
Deploy:
This is useful for deploying pre-built images with Skaffold.
skaffold deploy [-p <profile>] --build-artifacts=build_result.json
build_result.json
can also consist of just the image tags like below.
{
"builds": [
{
"imageName": "REPO_NAME/IMAGE_NAME",
"tag": "REPO_NAME/IMAGE_NAME:IMAGE_TAG"
},
{
"imageName": "REPO_NAME/IMAGE_NAME",
"tag": "REPO_NAME/IMAGE_NAME:IMAGE_TAG"
}
]
}
Delete Resources:
This will delete all the cluster resources created by Skaffold.
skaffold delete [options]
Logging and Port-forwarding :
Logging:
Log Tailing is enabled by default for dev
and debug
mode & disabled
by default for run
mode. It can be enabled with the --tail
flag.
skaffold run --tail
Details about logging.
Port-forwarding:
Port forwarding is disabled by default, can be enabled by using
--port-forward
. Port forwarding is supported only in dev
& debug
modes.
There are two types of port forwarding:
- Automatic Port Forwarding
- User Defined Port Forwarding
Details about port forwarding.
Highlighting Features of Skaffold:
Skaffold comes as a client-side only single binary cli tool. As there is
no on-cluster component, hence no overhead or maintenance work on
cluster.
Skaffold has a pluggable architecture, so we have a choice of using
suitable tools at each stage in pipeline.
Source: https://skaffold.dev/docs/design/
Skaffold provides built-in support for the following tools, at each
stage in pipeline.
Source: https://skaffold.dev/docs/design/
It is very easy to share projects (code & configuration) with just two
steps git clone
and skaffold run
.
Skaffold automatically manages logging and port-forwarding for the
deployments in dev
& debug
mode.
Single skaffold.yaml
file can be used to deploy to multiple
environments using below mechanisms:
- Skaffold profile
- User level configuration
- Environment variables
- Cli flags
Skaffold is very easy to use tool to automate the end-to-end developer
workflow for Kubernetes applications. The pipelines created with
skaffold are fast & efficient.
It has potential to be used as the CI/CD tool as well. It supports
multiple tools at each stage of the workflow, It has a lot of features
compared to other such tools, making it even better choice.
If you would like to hear more about Improving Kubernetes Developer
Workflow with Kind & Skaffold, take a look at this
talk
from one of my InfraCloud colleagues, Vishal Biyani, from PDC Tech
Carnival 2020.
Looking for help with building your DevOps strategy or want to outsource DevOps to the experts? learn why so many startups & enterprises consider us as one of the best DevOps consulting & services companies.
References: