Kubernetes Authentication in Rancher and RBAC

Update: Craig Jellick updated this topic of Kubernetes RBAC for Rancher 2.0 here.

As of version 1.6.3, Rancher introduced the Role-Based Access Control (RBAC) feature in Kubernetes. This feature allows admins to configure different policies that allow or deny access for users and service accounts to Kubernetes API resources. To better understand how the RBAC feature works, this post will shed light on how authentication works with the Kubernetes API, and how the RBAC authorization module works with authenticated users.

Authentication with Kubernetes in Rancher

Rancher uses Webhook Token Authentication strategy to authenticate user’s bearer tokens. First, a user authenticates with Rancher and navigates to the Kubernetes > CLI tab to get the kube configuration file, which contains the bearer token. Then, kubectl uses this token, along with the web hook authentication remote service, to authenticate users with the Kubernetes API: Diagram of Authentication with Kubernetes in
Rancher When the user tries to authenticate with the Kubernetes API using the bearer token, the authentication webhook communicates with the Rancher Kubernetes auth service and sends an authentication review object containing the token. Then, the Rancher Kubernetes auth service sends a review status that specifies whether or not the user is authenticated. The review status contains information about the user, including the name, uid, and groups. The authorization module in the Kubernetes API uses this information later to determine the access level for this user. Here’s an example of an authentication request that Kubernetes sends to the Rancher Kubernetes authentication service. Authentication Request:

          {
          "kind": "TokenReview",
          "apiVersion": "authentication.k8s.io/v1beta1",
          "Metad ata": {
               "creationTimestamp": null
               },
          "spec": {
          "token": "xxxxxxxxxxxxxxxxx"
                  },
          "status":{
                  "user": {}
                  }
          }

This Rancher Kubernetes auth service decides whether or not this user is authenticated, and sends a response to Kubernetes. Authentication Response:

          {
          "apiVersion": "authentication.k8s.io/v1beta1",
          "kind": "TokenReview",
          "status": {
               "authenticated": true,
               "user": {
                    "groups": [
                        ….
                        "system:masters"
                        ],
                    "username": "galal-hussein"
                    }
               }
          }

As you can see, because the environment owner sends this request, the user is defined in the system:masters group, which already has access to all resources in the Kubernetes cluster:

          kind: ClusterRoleBinding
          metadata:
           labels:
            kubernetes.io/bootstrapping: rbac-defaults
           name: cluster-admin
          roleRef:
            apiGroup: rbac.authorization.k8s.io
            kind: ClusterRole
            name: cluster-admin
           subjects:
           - apiGroup: rbac.authorization.k8s.io
            kind: Group
            name: system:masters

The ClusterRole “cluster-admin” resource allows access to all Kubernetes resources in all API groups:

          apiVersion: rbac.authorization.k8s.io/v1beta1
          kind: ClusterRole
          metadata:
           labels:
             kubernetes.io/bootstrapping: rbac-defaults
           name: cluster-admin
          rules:
          - apiGroups:
           - '*'
           resources:
           - '*'
           verbs:
           - '*'
          - nonResourceURLs:
           - '*'
           verbs:
           - '*'

The RBAC Authorization Module

After the request to the Kubernetes API is successfully authenticated, the request must be authorized. The request to the API contains information about the username of the requester, the requested action, and the object the action affects. The RBAC authorization module defines four top-level objects that control the authorization decisions for authenticated users:

[Roles]
[ClusterRoles]
[RoleBinding]
[ClusterRoleBinding]

Roles and ClusterRoles both identify sets of permissions for Kubernetes API resources. The only difference between them is that Roles can be defined within the namespaces whereas ClusterRoles are cluster wide. RoleBindings and ClusterRoleBinding assign the defined roles to users, groups, or service accounts. Permissions also can be granted within a namespace using RoleBinding, or cluster wide using ClusterRoleBinding. Examples will be discussed in the following section.

Enable RBAC with Kubernetes in Rancher

To enable RBAC feature with fresh installation of Kubernetes in Rancher, you can either edit the default environment or create a new environment template. In the Kubernetes environment options, you can enable RBAC or, if you’ve already launched the Kubernetes infrastructure service, you can click Up to Date to update the configuration options for Kubernetes.

RBAC Examples

These examples assume that you already started Kubernetes with the RBAC feature enabled, as explained in the previous section. It also assumes that you enabled GitHub authentication with Rancher. As the owner of the Kubernetes environment, you have full access over the Kubernetes API, as we saw before, because the cluster-admin role is assigned by default to the environment owner. on-admin users will not have access to any API resources by default. Assuming that you added some GitHub users and groups as members of the Kubernetes environment, and then tried to access the Kubernetes API with them, you would get the following message:

          > kubectl get services
          Error from server (Forbidden): the server does not allow access
          to the requested resource (get services)

To enable access for a GitHub organization across all Kubernetes clusters, create the following ClusterRole:

           kind: ClusterRole
           apiVersion: rbac.authorization.k8s.io/v1beta1
           metadata:
            name: default-reader
           rules:
            - apiGroups: [""]
             resources:
              - services
             verbs: ["get", "watch", "list"]
            - nonResourceURLs: ["*"]
             verbs: ["get", "watch", "list"]

This role defines listing and getting access to the services resource. At this point, the cluster role is not associated with any user or group, so the following step is creating the ClusterRoleBinding:

           kind: ClusterRoleBinding
           apiVersion: rbac.authorization.k8s.io/v1beta1
           metadata:
            name: default-reader-role-binding
           subjects:
            - kind: Group
             name: "github_org:"
           roleRef:
            kind: ClusterRole
            name: default-reader
            apiGroup: rbac.authorization.k8s.io

The role binding specifies a GitHub organization group: “github_org:“. You will notice that when applying the role bindings to groups, Rancher authentication has special syntax for each authentication type. For more details, please refer to the Rancher documentation. After creating the role binding, you should be able to list only the services for any user that belongs to this GitHub organization:

           > kubectl get pods
           Error from the server (Forbidden): the server does not allow access
           to the requested resource (get pods)

           > kubectl get service
           NAME        CLUSTER-IP  EXTERNAL-IP  PORT(S)  AGE
           kubernetes  10.43.0.1         443/TCP   42m