Skip to content

Setup Instructions

Alt

Install sample K8s cluster (skip if you already have the cluster)

Install K3s

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--flannel-backend=none --disable traefik' sh -s - --write-kubeconfig-mode 644

Make K3's cluster config the default

cp /etc/rancher/k3s/k3s.yaml ~/.kube/config

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: kubearmor-ub20
  region: us-east-2

nodeGroups:
  - name: ng-1
    amiFamily: "Ubuntu2004"
    privateNetworking: true
    desiredCapacity: 2
    # taint nodes so that application pods are
    # not scheduled until Cilium is deployed.
    taints:
     - key: "node.cilium.io/agent-not-ready"
       value: "true"
       effect: "NoSchedule"
    ssh:
      allow: true
    preBootstrapCommands:
      - "sudo apt install linux-headers-$(uname -r)"

eksctl create cluster -f  sample-ubuntu-18.04-cluster.yaml 

aws eks --region us-east-1 update-kubeconfig --name kubearmor-ub20

Ref: KubeArmor support matrix

1. Install accuknox cli tool, daemonsets and services

Install accuknox cli tool

curl -sfL https://raw.githubusercontent.com/accuknox/accuknox-cli/main/install.sh | sudo sh -s -- -b /usr/local/bin

Install Daemonsets and services

accuknox install

For just kubearmor and discovery-engine installation (skipping cilium) use: 

accuknox install --disable cilium

Output from kubectl get pods -A
NAMESPACE     NAME                                             READY   STATUS    RESTARTS       AGE
kube-system   local-path-provisioner-5bd75fdd7f-9r8qh          1/1     Running   0              133d
kube-system   coredns-96cc4f57d-xw7j6                          1/1     Running   0              133d
kube-system   metrics-server-ff9dbcb6c-94hfk                   1/1     Running   0              133d
kube-system   cilium-operator-66bff48676-667pk                 1/1     Running   0              4h4m
kube-system   cilium-hv77h                                     1/1     Running   0              4h4m
kube-system   hubble-relay-8548d8946c-tmcrk                    1/1     Running   0              4h3m
kube-system   kubearmor-host-policy-manager-766447b4d7-crdmn   2/2     Running   0              4h3m
kube-system   kubearmor-policy-manager-54ffc4dc56-s7ljw        2/2     Running   0              4h3m
kube-system   kubearmor-lt46z                                  1/1     Running   0              4h3m
kube-system   kubearmor-relay-645667c695-s6j72                 1/1     Running   0              4h3m
explorer      knoxautopolicy-6fbf6f6c76-cc4t6                  1/1     Running   0              3h18m

We have following installed:

  • Kubearmor application protection engine
  • Cilium CNI
  • Discovery engine
  • Hubble Relay and KubeArmor Relay

2. Install sample k8s application

Install the following app or you can try your own k8s app.

kubectl apply -f https://raw.githubusercontent.com/kubearmor/KubeArmor/main/examples/wordpress-mysql/wordpress-mysql-deployment.yaml

The use-cases described in subsequent step uses this sample application.

3. Demo Scenario & Use-cases

Alt

Use-case 1: Audit access to sensitive data paths

MySQL keeps all its database tables as part of /var/lib/mysql folder path. Audit access to this folder path recursively (sub-folders inclusive).

apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: ksp-mysql-audit-dir
  namespace: wordpress-mysql
spec:
  severity: 5
  selector:
    matchLabels:
      app: mysql
  file:
    matchDirectories:
    - dir: /var/lib/mysql/
      recursive: true
  action:
    Audit

kubectl exec -it mysql-6c6cb995bb-x9zqx -n wordpress-mysql -- bash

Alt

kubectl apply -f ksp-mysql-audit-dir.yaml

karmor log

Alt

Use-case 2: Block access to files containing sensitive data

Wordpress pod contains a file wp-config.php that has sensitive auth credentials. Use-case is to block access to this file from unknown processes.

kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql -- bash

cat /var/www/html/wp-config.php

Alt

apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: ksp-wordpress-block-config
  namespace: wordpress-mysql
spec:
  severity: 10
  selector:
    matchLabels:
      app: wordpress
  file:
    matchPaths:
    - path: /var/www/html/wp-config.php
      fromSource:
      - path: /bin/cat
  action:
    Block

kubectl apply -f ksp-wordpress-block-config.yaml

kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql –- cat /var/www/html/wp-config.php

Alt

Use-case 3: Block access to k8s service account token

Pod is the basic execution unit in k8s. One problem with this approach is that all the processes within that pod have unrestricted access to the pod's volume mounts. One such volume mount is service account token. Thus, accessing service account token using a injected binary is a common attack pattern in k8s. This use-case explains how you can protect access to the service account token through known processes only.

kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql -– cat /run/secrets/kubernetes.io/serviceaccount/token

Alt

apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: ksp-wordpress-block-sa
  namespace: wordpress-mysql
spec:
  severity: 7
  selector:
    matchLabels:
      app: wordpress
  file:
    matchDirectories:
    - dir: /run/secrets/kubernetes.io/serviceaccount/
      recursive: true
  action:
    Block

kubectl apply -f ksp-wordpress-block-sa.yaml

kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql -– cat /run/secrets/kubernetes.io/serviceaccount/token

Alt

Use-case 4: Block execution of unwanted processes

Container images might get shipped with binaries that are not supposed to be executed in production environments. For e.g., wordpress contains apt, apt-get binaries that are used for dynamic package management. These should never be used in the production environment since it will create drift (change) in the container contents i.e, introduce new files/binaries that might increase the attack surface. The policy given below blocks execution of such processes.

kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql –- bash

apt

apt-get update

Alt

apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: ksp-wordpress-block-process
  namespace: wordpress-mysql
spec:
  severity: 3
  selector:
    matchLabels:
      app: wordpress
  process:
    matchPaths:
    - path: /usr/bin/apt
    - path: /usr/bin/apt-get
  action:
    Block

kubectl apply -f ksp-wordpress-block-process.yaml

kubectl exec -it wordpress-5679855487-58rfz -n wordpress-mysql –- bash

apt

apt-get

Alt

4. Get Auto Discovered Policies

In the above cases in step 4, we had explicit deny based policies. KubeArmor also supports Allow-based policies i.e., allow only specific actions and audit/deny everything else. Also the allow-policies are auto-discovered by examining the workloads at runtime.

To retrieve the auto discovered policies you can use: 

accuknox discover -n wordpres-mysql -l "app=wordpress" -f yaml
This discovers the policies for a workload in wordpress-mysql namespace having label app=wordpress.

Sample Output
❯ accuknox discover -n wordpress-mysql -l "app=wordpress" -f yaml
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: autopol-system-3960684242
  namespace: wordpress-mysql
spec:
  action: Allow
  file:
    matchPaths:
    - fromSource:
      - path: /usr/sbin/apache2
      path: /dev/urandom
    - fromSource:
      - path: /usr/local/bin/php
      path: /etc/hosts
  network:
    matchProtocols:
    - fromSource:
      - path: /usr/local/bin/php
      protocol: tcp
    - fromSource:
      - path: /usr/local/bin/php
      protocol: udp
  process:
    matchPaths:
    - path: /usr/sbin/apache2
    - path: /usr/local/bin/php
  selector:
    matchLabels:
      app: wordpress
  severity: 1

5. Uninstall

accuknox uninstall

References

  1. Elastic and Splunk Integration Guide
  2. KubeArmor support matrix
  3. Integrating Kubearmor with Prometheus and Grafana
Back to top