Introduction
Most of the times the CI pipeline’s automated test phase requires a
database to be provisioned so that the business application can be
tested in its entirety.
In this guide we will provision a managed Postgres database on the
Google cloud platform using Crossplane and GitLab’s Auto DevOps pipeline
as part of CI.
A little bit about GitLab’s Auto DevOps, GitLab Managed Apps and
Crossplane in the subsequent sections.
GitLab Auto DevOps
GitLab provides a feature called Auto DevOps which automatically creates
CI/CD pipelines for your source code repository. It is an opinionated
pipeline with reasonable defaults. The pipeline can be customized by way
of a yaml configuration. Auto DevOps facilitates standardisation and
consistency of the CI/CD pipelines.
GitLab Managed
Apps
GitLab provides a feature of a One-Click Install of an application onto
a Kubernetes cluster connected to a repository. Applications such as
Prometheus, Cert-Manager or the newly added Crossplane control plane can
be installed via the Managed Apps dashboard.
Crossplane
Crossplane is an open source multi cloud control plane. It allows you to
provision cloud services the kubectl way by exposing them as Kubernetes
resources by the use of CRDs and controllers. Crossplane connects to the
cloud provider using service account credentials and orchestrates
creation of services via the cloud provider APIs. This helps developers
and administrators alike in provisioning and configuration of these
services using Kubernetes resources hence, allowing a uniform interface
for those working on the platform. It follows the model similar to a
StorageClass and a PersistentVolumeClaim where an administrator could
define the class of services available to be consumed and the developers
request for a claim.
Reference: Crossplane Architecture
This guide will demonstrate the following.
- Create a sourcecode repository for a sample application in GitLab.
- Connect a Kubernetes cluster to that respository.
- Install Crossplane via the GitLab Managed Apps interface.
- Run the Auto DevOps pipeline for the repository which will provision
a managed Postgres instance of the cloud provider (CloudSQL in this
case)
- Verify the application is able to connect to the Postgres instance.
Create a repository and a GitLab Managed cluster.
Login to gitlab.com and create a new project. We
will use the following repository as a sample application in this guide.
https://github.com/mahendrabagul/go-postgres-example
Once the project is created, you can use the instructions given on
landing page to push your application source code to the newly created
repository.
The next step is to setup a cluster for the project. In order to do
this, Navigate to the “Operations/Kubernetes” in the left hand
menu. GitLab allows creation of a cluster or adding an existing cluster
to the repository. In this guide we will create a new GKE cluster via
the GitLab interface.
Click on the “Add Kubernetes Cluster”. You will be landed on “Add a
Kubernetes cluster integration”. Select “Google GKE” from “Create new
Cluster tab”. You will then be asked to sign to you google account.
Select your account/Log in to your account. (In my case, I already have
a project in Google Cloud account. You can refer this link to create a new project on Google Cloud.
You will then be presented with a form with a bunch of fields to be
filled.
Enter the details for the cluster configuration details. Select the
GitLab-managed cluster option since this will basically provision a
service account with the right RBAC permissions and which is used by the
GitLab managed apps
Once the cluster is provisioned on GCP the GitLab UI redirects to the
configuration of the connected cluster.
GitLab uses Helm (2.x) to install the Managed Applications on the
connected Kubernetes cluster. Hence, the pre-requisite of installing
Crossplane is to first install Helm as the managed app.
Install the Ingress controller and Crossplane with Google cloud Platform
as the choice of Stack via the Managed Apps interface.
Crossplane needs to be configured with the cloud provider to allow
provisioning of managed services. Follow this guide to configure
Crossplane with the cloud provider.
Important Note:
You should have the following resources created on your Kubernetes
cluster after configuring Crossplane.
- A Connection and a GlobalAddress kinds in order to allow connection
from the workload on the GKE cluster with CloudSQL.
- Resource Classes. These are classes of service for CloudSQL that can
be defined by an administrator.
- Necessary RBAC permissions if the GKE cluster is an existing cluster
added to the repo.
Auto DevOps Configuration Options
The Auto DevOps pipeline can be configured in multiple ways to utilize
Crossplane for provisioning Postgres. The above steps to configure
Crossplane will create 2 CloudSQLInstance resources, one of them will be
referenced as the class of service via the Auto DevOps configuration.
- The Environment
variables,
AUTO_DEVOPS_POSTGRES_MANAGED and AUTO_DEVOPS_POSTGRES_MANAGED_CLASS_SELECTOR provision
PostgreSQL using Crossplane.
- If the Auto DevOps chart is bundled as part of the source code
repository, the
values.yaml needs the following changes.
-
postgres.managed set to true which will select a default
resource class.
-
postgres.managed set
to true with postgres.managedClassSelector providing the
resource class to choose based on labels. For
eg: postgres.managedClassSelector.matchLabels.gitlab-ad-demo="true" will
select the CloudSQLInstance class which satisfies the claim
based on the labels.
The Auto DevOps pipeline would get triggered upon a commit or executed
manually. The pipeline provisions a postgresqlinstance which is a
resource defined by the Crossplane CRDs. This essentially is a claim for
creating a Postgres Instance referencing a CloudSQLInstanceClass based
on labels or a default class.
Verify the creation of a postgresqlinstance
A quick look on the google cloud SQL dashboard should show the newly
provisioned postgres instance.
The sample application creates a table and inserts a few records in the
database. Logs from the example
application
Looking for help with building your CI/CD pipelines? learn why so many startups and enterprises consider as their CI/CD consulting & services provider.
Further References
- https://crossplane.io/docs/master/
- https://docs.gitlab.com/ee/topics/autodevops/
- https://www.youtube.com/watch?v=Zv6llSEUDYk\&list=PLFGfElNsQthbiSy32TsYZyVgMk8Rv54kN\&index=5\&t=0s
