Setup Instructions¶
Install sample K8s cluster (skip if you already have the cluster)
Install K3s
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--flannel-backend=none --disable traefik' sh -s - --write-kubeconfig-mode 644
Make K3's cluster config the default
cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: kubearmor-ub20
region: us-east-2
nodeGroups:
- name: ng-1
amiFamily: "Ubuntu2004"
privateNetworking: true
desiredCapacity: 2
# taint nodes so that application pods are
# not scheduled until Cilium is deployed.
taints:
- key: "node.cilium.io/agent-not-ready"
value: "true"
effect: "NoSchedule"
ssh:
allow: true
preBootstrapCommands:
- "sudo apt install linux-headers-$(uname -r)"
eksctl create cluster -f sample-ubuntu-18.04-cluster.yaml
aws eks --region us-east-1 update-kubeconfig --name kubearmor-ub20
1. Install accuknox cli tool, daemonsets and services¶
Install accuknox cli tool¶
curl -sfL https://raw.githubusercontent.com/accuknox/accuknox-cli/main/install.sh | sudo sh -s -- -b /usr/local/bin
Install Daemonsets and services¶
accuknox install
For just kubearmor and discovery-engine installation (skipping cilium) use:
accuknox install --disable cilium
Output from kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system local-path-provisioner-5bd75fdd7f-9r8qh 1/1 Running 0 133d
kube-system coredns-96cc4f57d-xw7j6 1/1 Running 0 133d
kube-system metrics-server-ff9dbcb6c-94hfk 1/1 Running 0 133d
kube-system cilium-operator-66bff48676-667pk 1/1 Running 0 4h4m
kube-system cilium-hv77h 1/1 Running 0 4h4m
kube-system hubble-relay-8548d8946c-tmcrk 1/1 Running 0 4h3m
kube-system kubearmor-host-policy-manager-766447b4d7-crdmn 2/2 Running 0 4h3m
kube-system kubearmor-policy-manager-54ffc4dc56-s7ljw 2/2 Running 0 4h3m
kube-system kubearmor-lt46z 1/1 Running 0 4h3m
kube-system kubearmor-relay-645667c695-s6j72 1/1 Running 0 4h3m
explorer knoxautopolicy-6fbf6f6c76-cc4t6 1/1 Running 0 3h18m
We have following installed:
- Kubearmor application protection engine
- Cilium CNI
- Discovery engine
- Hubble Relay and KubeArmor Relay
2. Install sample k8s application¶
Install the following app or you can try your own k8s app.
kubectl apply -f https://raw.githubusercontent.com/kubearmor/KubeArmor/main/examples/wordpress-mysql/wordpress-mysql-deployment.yaml
The use-cases described in subsequent step uses this sample application.
3. Demo Scenario & Use-cases¶
Use-case 1: Audit access to sensitive data paths
MySQL keeps all its database tables as part of /var/lib/mysql
folder path. Audit access to this folder path recursively (sub-folders inclusive).
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: ksp-mysql-audit-dir
namespace: wordpress-mysql
spec:
severity: 5
selector:
matchLabels:
app: mysql
file:
matchDirectories:
- dir: /var/lib/mysql/
recursive: true
action:
Audit
kubectl exec -it mysql-6c6cb995bb-x9zqx -n wordpress-mysql -- bash
kubectl apply -f ksp-mysql-audit-dir.yaml
karmor log
Use-case 2: Block access to files containing sensitive data
Wordpress pod contains a file wp-config.php
that has sensitive auth credentials. Use-case is to block access to this file from unknown processes.
kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql -- bash
cat /var/www/html/wp-config.php
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: ksp-wordpress-block-config
namespace: wordpress-mysql
spec:
severity: 10
selector:
matchLabels:
app: wordpress
file:
matchPaths:
- path: /var/www/html/wp-config.php
fromSource:
- path: /bin/cat
action:
Block
kubectl apply -f ksp-wordpress-block-config.yaml
kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql –- cat /var/www/html/wp-config.php
Use-case 3: Block access to k8s service account token
Pod is the basic execution unit in k8s. One problem with this approach is that all the processes within that pod have unrestricted access to the pod's volume mounts. One such volume mount is service account token. Thus, accessing service account token using a injected binary is a common attack pattern in k8s. This use-case explains how you can protect access to the service account token through known processes only.
kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql -– cat /run/secrets/kubernetes.io/serviceaccount/token
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: ksp-wordpress-block-sa
namespace: wordpress-mysql
spec:
severity: 7
selector:
matchLabels:
app: wordpress
file:
matchDirectories:
- dir: /run/secrets/kubernetes.io/serviceaccount/
recursive: true
action:
Block
kubectl apply -f ksp-wordpress-block-sa.yaml
kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql -– cat /run/secrets/kubernetes.io/serviceaccount/token
Use-case 4: Block execution of unwanted processes
Container images might get shipped with binaries that are not supposed to be executed in production environments. For e.g., wordpress contains apt
, apt-get
binaries that are used for dynamic package management. These should never be used in the production environment since it will create drift (change) in the container contents i.e, introduce new files/binaries that might increase the attack surface. The policy given below blocks execution of such processes.
kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql –- bash
apt
apt-get update
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: ksp-wordpress-block-process
namespace: wordpress-mysql
spec:
severity: 3
selector:
matchLabels:
app: wordpress
process:
matchPaths:
- path: /usr/bin/apt
- path: /usr/bin/apt-get
action:
Block
kubectl apply -f ksp-wordpress-block-process.yaml
kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql –- bash
apt
apt-get
4. Get Auto Discovered Policies¶
In the above cases in step 4, we had explicit deny based policies. KubeArmor also supports Allow-based policies i.e., allow only specific actions and audit/deny everything else. Also the allow-policies are auto-discovered by examining the workloads at runtime.
To retrieve the auto discovered policies you can use:
accuknox discover -n wordpres-mysql -l "app=wordpress" -f yaml
wordpress-mysql
namespace having label app=wordpress
.
Sample Output
❯ accuknox discover -n wordpress-mysql -l "app=wordpress" -f yaml
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-3960684242
namespace: wordpress-mysql
spec:
action: Allow
file:
matchPaths:
- fromSource:
- path: /usr/sbin/apache2
path: /dev/urandom
- fromSource:
- path: /usr/local/bin/php
path: /etc/hosts
network:
matchProtocols:
- fromSource:
- path: /usr/local/bin/php
protocol: tcp
- fromSource:
- path: /usr/local/bin/php
protocol: udp
process:
matchPaths:
- path: /usr/sbin/apache2
- path: /usr/local/bin/php
selector:
matchLabels:
app: wordpress
severity: 1
5. Uninstall¶
accuknox uninstall