understanding certificate-based "users" of k8s

1 k8s does NOT have a user database

Unlike many other systems, k8s by default does not maintain its own database to save users. 

That being said, authentication plugins exist to authenticate users with LDAP/OID and save user config as k8s resources into etcd database. But that's not the topic covered in this blog.

In this blog, we only talk about k8s certificate-based users.

2 k8s uses x509 certificates as users ID

k8s identifies and verifies its users by certificates. Only certificates signed by the k8s cluster's own certificate authority are authenticated as valid users.

So here are the key points.

• every user certificate must be signed by k8s itself
• every user certificate has an expiration date
• information within the certificate can be used to describe the user
• CN: is taken as the username
• O: is taken as group(s)

3 Example

In this example, we will create a user and give it permission to read pods in the default namespace.

3.1 Locate the cluster's CA

Different k8s implementations may save their CA configuration in different places, this example is taken from OpenShift OKD 3.

On any control plane node, the k8s cluster CA's private key and certificate files are as below.

/etc/origin/master/ca.key

/etc/origin/master/ca.crt

Later on, we will use this key pair to sign a user CSR.

3.2 create a CSR

# generate user private key

$ openssl genrsa -out bob.key 4096

# generate usr CSR

$ openssl req -new -key bob.key -out bob.csr -subj '/CN=bob/O=finance'

Note, the user name is "bob", and he belongs to group "finance".

3.3 sign user's CSR

# sign usr CSR, generate user certificate

$ openssl x509 -req -in bob.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out bob.crt -days 365

Now, we got the user's certificate "bob.crt" which represents user BOB from now on.

3.4 generate kubeconfig file

The command line tool "kubectl" uses kubeconfig file to store users' certificates.

$ cat bob.kubeconfig

# vi: set ft=yaml sts=2 ts=2 sw=2:

apiVersion: v1

kind: Config

clusters:

    - name: okd3

      cluster:

        server: https://xxxxxxx:8443

contexts:

    - name: context1

      context:

          cluster: okd3

          namespace: default

          user: bob

current-context: context1

users:

    - name: bob

      user:

          client-certificate: bob.crt

          client-key: bob.key

Now, we try to connect the k8s API server using this kubeconfig file.

$ kubectl --kubeconfig bob.kubeconfig get pods -n default

Error from server (Forbidden): pods is forbidden: User "bob" cannot list pods in the namespace "default": no RBAC policy matched

As the output shows, user "bob" was recognized and authenticated but he currently doesn't have the permission yet to list pods in namespace "default".

3.5 create a role and rolebinding to grant permissions

Both role and rolebinding are namespaced resources.

# create a role that has permissions to read/list pods

$ kubectl create role pod-reader --verb=get,list --resource=pods -n default

role.rbac.authorization.k8s.io/pod-reader created

# bind role to user group "finance" to which bob belongs

$ kubectl create rolebinding pod-reader --role=pod-reader --group=finance -n default

rolebinding.rbac.authorization.k8s.io/pod-reader created

Now, we run the failed pods listing command again.

$ kubectl --kubeconfig bob.kubeconfig get pods -n default

NAME                          READY   STATUS    RESTARTS   AGE

docker-registry-20-8r2n5      1/1     Running   0          70d

ipfailover-2-6bfq2            1/1     Running   0          88d

ipfailover-2-76mwn            1/1     Running   0          78d

ipfailover-2-p6ss6            1/1     Running   0          88d

logging-eventrouter-1-kf47j   1/1     Running   1          91d

openshift-redirect-2-cxn84    1/1     Running   0          91d

registry-console-3-mf5jb      1/1     Running   0          91d

router-21-5sshb               1/1     Running   0          88d

router-21-hw8s5               1/1     Running   0          88d

router-21-l99mt               1/1     Running   0          78d

Now bob is able to list pods in the "default" namespace. If you want to give bob more permissions, just modify the role or add more roles and role bindings.

3.6 More about kubeconfig file

kubeconfig file is the main config file for kubectl and many other tools. Please be reminded that this file is saved on the client-side, not a k8s managed resource.

Regarding the user section within this file, the users' private key and certificate can not only be referenced by file names as shown in 3.4, but also be saved directly in the config file itself.

users:

    - name: bob

      user:

          client-certificate-data: LS0................

          client-key-data: LS0...............

The data there is base64 encoded, they can be got by:

$ cat bob.crt | base64 -w0

$ cat bob.key | base64 -w0