Building a Kubernetes CI CD Pipeline

This blog coincides with the Rancher Labs Online Meetup August 2018 covering Continuous Integration/Continuous Delivery (CI/CD). This topic is becoming increasingly important in a world where services are getting small, with frequent updates. It allows for companies to completely automate the building, testing and deployment of code to ensure that it is done in a consistent, repeatable way.

There are several different CI/CD tools available and there are a bunch of them that will integrate with Kubernetes natively.

This blog is going to cover Kubernetes CI CD using the hosted GitLab.com solution but the Kubernetes integrations that will be covered are generic and should work with any CI/CD provider that interface directly to Kubernetes using a service account.

Prerequisites

1. Rancher 2.0 cluster for deploying workloads to.
2. gitlab.com login.

GitLab.com setup

For the purposes of this blog we are going to use one of the templates that GitLab provides, first step is to login to gitlab.com via https://gitlab.com/users/sign_in

Create a Project

The first step is to create a project:

• Click New project.

• Choose the Create from template tab.

• Click under

• Set Project name.

• Click Create Project.

• Wait for the project to finish importing.

Add your Kubernetes endpoint to your project

Under choose

Click Kubernetes cluster, then choose Add existing cluster

All of the above fields will need to be filled in, the following sections will detail how to fill this in.

API URL

The API URL is the URL that GitLab will use to talk to the Kubernetes API in your cluster that is going to be used to deploy your workloads. Depending on where your Kubernetes cluster is running you will need to ensure that the port is open to allow for communication from gitlab.com to the <address>:<port> of the Kubernetes cluster.

In order to retrieve the API URL we are going to run a script on the Rancher Server that controls your Kubernetes cluster, this will generate a kubeconfig file that contains the information we need to configure the Kubernetes settings with GitLab.

• Log onto the server that is running your Rancher Server container.

• Download the content of get_kubeconfig_custom_cluster_rancher2.sh from https://gist.github.com/superseb/f6cd637a7ad556124132ca39961789a4

• Create a file on the server and paste the contents into it.

• Make the file executable chmod +x <filename>.

• Run the script as ./<filename> <name_of_cluster_to _deploy_to>

• This will generate a kubeconfig file in the local directory.

• Run cat kubeconfig | grep server:

• The https value is what needs to be populated into the API URL field in the Add existing cluster

CA Certificate

The CA certificate is required as these are often custom certificates that aren’t in the certificate store of the GitLab server, this will allow the connection to be secured.

From the folder containing the kubeconfig generated in the API URL instructions:

• Run cat kubeconfig | grep certificate-authority-data.

This give you a base64 encoded certificate string, the field in GitLab requires it in PEM format.

• Save the contents of the encoded string to a file i.e. cert.base64

• Run base64 -d cert.base64 > cert.pem.

• This will then give you the certificate in pem format, which you can then copy and paste into the CA Certificate field in GitLab

Token

For the gitlab.com instance to be able to talk to the cluster we are going to create a service account that it will use. We are also going to create a namespace for GitLab to deploy the apps to.

To simplify this I’ve put all of this into a file, which can be viewed at http://x.co/rm082018

For this to create the necessary prerequisites you will need to run the following command:

kubectl apply -f http://x.co/rm082018 (optionally add –kubeconfig <kubeconfig> if you want to use a cluster other than the default one specified in your .kube/config file)

This will create a service account and create a token for it, which is the token that we need to specify in the GitLab Kubernetes configuration pane.

To obtain the secret execute

kubectl describe secrets/gitlab-secret -n gitlab-managed-apps | grep token:

Copy the token and paste that into the GitLab configuration

Project Namespace

If you have followed this blog so far and have used applied my Kubernetes manifest file then you will need to set the Project Namespace as gitlab-managed-apps. If you have updated the manifest then you will need to set this to reflect the namespace that you have set.

Rancher Server 2.0 setup

As part of the GitLab template projects they deploy a PostgresSQL pod. This means that you will need to have a dynamic storage provisioner, if you don’t have one of these set up then go into the catalog on the cluster that you are going to deploy to and Launch the Library NFS provisioner. This isn’t recommended for production use but it will get the auto devops function working.

Enabling Auto devops

In the GitLab UI, go into the Settings – CI/CD and expand Auto DevOps

Click the Enable Auto Devops radio button

In the Domain section it requires you to specify the DNS name that will be used to reach the service that is going to be deployed. The DNS name should point to the ingress on the cluster that the service will be deployed to. For testing you can just use <host-ip>.nip.io which will resolve to the host ip that is specified.

This will not give resilience and will only allow http not https but again this is enough to show it working, if you want to use https then you would need to add a wildcard cert the ingress controller.

Click Save changes and this should automatically launch you a pipeline and start a job running.

You can go into CI/CD – Pipelines to see the progress.

At the end of the production phase you should see the http address that you can access your application on.

Hopefully this has allowed you to successfully deploy a nice demonstration CI/CD pipeline. As I stated at the start the Kubernetes piece should work for the majority of CI/CD Kubernetes integrations.

Further Training

For further training, you can watch a demo on setting up a Kubernetes CI CD pipeline on on our online meetup video recording.