Announcing RKE, a Lightweight Kubernetes Installer

Today, we are announcing a new open-source project called the Rancher Kubernetes Engine (RKE), our new Kubernetes installer. RKE is extremely simple, lightning fast, and works everywhere.

Why a new Kubernetes installer?

In the last two years, Rancher has become one of the most popular ways to stand up and manage Kubernetes clusters. Users love Rancher as a Kubernetes installer because it is very easy to use. Rancher fully automates etcd, the Kubernetes master, and worker node operations. Rancher 1.x, however, also implements container networking. Therefore, a failure of the Rancher management plane could disrupt the operation of the Kubernetes cluster. Users who want to stand up Kubernetes clusters today have many choices of installers. Two of the most popular installers we have encountered are kops and Kubespray:

1. Kops is perhaps the most widely used Kubernetes installer. It is in fact much more than an installer. Kops prepares all required cloud resources, installs Kubernetes, and then wires up cloud monitoring services to ensure the continuing operation of the Kubernetes cluster. Kops is closely integrated with the underlying cloud infrastructure. Kops works the best on AWS. Support for other infrastructure platforms like GCE and vSphere is a work in progress.
2. Kubespray is a popular standalone Kubernetes installer written in Ansible. It can install a Kubernetes cluster on any servers. Even though Kubespray has some degree of integration with various cloud APIs, it is fundamentally cloud independent and can, therefore, work with any cloud, virtualization clusters, or bare-metal servers. Kubespray has grown to be a sophisticated project with participation from a large community of developers.

Kubeadm is another Kubernetes setup tool that comes with upstream Kubernetes. Kubeadm, however, does not yet support capabilities like HA clusters. Even though pieces of kubeadm code are used in projects like kops and Kubespray, kubeadm is not ready as a production-grade Kubernetes installer. Rancher 2.0 is designed to work with any Kubernetes clusters. We encourage users to leverage cloud-hosted Kubernetes services like GKE and AKS. For users who want to set up their own clusters, we considered incorporating either kops or Kubespray into our product lineup. Kops does not suit our needs because it does not work with all cloud providers. Kubespray is in fact very close to what we want. We especially like how Kubespray can install Kubernetes anywhere. In the end, we decided not to use Kubespray and instead build our own lightweight installer for two reasons:

1. We can have a simpler system by starting from scratch and take advantage of many advances in Kubernetes itself.
2. We can have a faster installer by going with a container-based approach, just like how we installed Kubernetes in Rancher 1.6.

How RKE Works

RKE is a standalone executable that reads from a cluster configuration file and brings up, brings down, or upgrades a Kubernetes cluster. Here is a sample configuration file:

---
auth:
  strategy: x509

network:
  plugin: flannel

ssh_key_path: /home/user/.ssh/id_rsa

nodes:
  - address: server1
    user: ubuntu
    role: [controlplane, etcd]
  - address: server2
    user: ubuntu
    role: [worker]

services:
  etcd:
    image: quay.io/coreos/etcd:latest
  kube-api:
    image: rancher/k8s:v1.8.3-rancher2
    service_cluster_ip_range: 10.233.0.0/18
    extra_args:
      v: 4
  kube-controller:
    image: rancher/k8s:v1.8.3-rancher2
    cluster_cidr: 10.233.64.0/18
    service_cluster_ip_range: 10.233.0.0/18
  scheduler:
    image: rancher/k8s:v1.8.3-rancher2
  kubelet:
    image: rancher/k8s:v1.8.3-rancher2
    cluster_domain: cluster.local
    cluster_dns_server: 10.233.0.3
    infra_container_image: gcr.io/google_containers/pause-amd64:3.0
  kubeproxy:
    image: rancher/k8s:v1.8.3-rancher2

addons: |-
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-nginx
      namespace: default
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 80

We start the file by specifying authentication strategy, network model, and local SSH key path. The main body of the cluster configuration file consists of the following three parts:

1. The nodes section describes all the servers that make up the Kubernetes cluster. Each node assumes one or more of the three roles: controlplane, etcd, and worker. You can add or remove nodes in a Kubernetes cluster by changing the nodes section and rerunning the RKE command.
2. The services section describes all the system services that run on the Kubernetes cluster. RKE packages all system services as containers.
3. The add-ons section describes the user-level programs that run on the Kubernetes cluster. An RKE user, therefore, can specify the Kubernetes cluster configuration and application configuration in the same file.

RKE is not a long-running service that can monitor and operate the Kubernetes cluster. RKE is designed to work in conjunction with a full-fledged container management system like Rancher 2.0 or with a stand-alone monitoring system like AWS CloudWatch, Datadog, or Sysdig. You can then construct your own scripts to monitor the health of RKE clusters.

RKE as an Embedded Kubernetes Installer

People who build distributed applications have to deal with backend databases, data access layers, clustering, and scaling. Instead of using a traditional application server, developers are beginning to use Kubernetes as a distributed application platform:

• They use etcd as the backend database.
• They use Kubernetes Custom Resource Definition (CRD) to as the data access layer, and they use kubectl to perform basic CRUD operations on their data model.
• They package their applications as containers and use Kubernetes for clustering and scaling.

Applications built this way are shipped to customers as Kubernetes YAML files. Customers can run these applications easily if they already have Kubernetes clusters running or have access to a cloud-hosted Kubernetes service like GKE or AKS. But what happens to the customers who want to install the applications on virtualized or bare-metal servers? An application developer can address this need by bundling RKE into the application as an embedded Kubernetes installer. The application install can start by invoking RKE and create a Kubernetes cluster for the customer. We are seeing a tremendous amount of interest to embed a lightweight installer like RKE into distributed applications.

Next Steps

You can download RKE from GitHub. I encourage you to read the blog post by Hussein Galal, who wrote a significant portion of RKE code, for a more in-depth introduction to RKE. Join us tomorrow for an Online Meetup where we’ll give a demo of RKE. Please sign up today.