Skip to content

KubeArmor and Cilium on Debian 10 (Buster)

Overview

This user journey guides you to install and verify the compatibility of Kuberarmor and Cilium on Debian 10 (Buster) with 4.19 Kernel Version by applying policies on kubernetes workloads.

Step 1: Install etcd in control plane VM

Install etcd using below command:

sudo apt-get install etcd

Alt

Once etcd is installed, configure the following values in /etc/default/etcd as shown below.

ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
ETCD_ADVERTISE_CLIENT_URLS=http://0.0.0.0:2379

Restart etcd & check the etcd status:

sudo service etcd restart
sudo service etcd status

Alt

Step 2: Install KVM-Service in control plane

Download the latest deb Package

wget https://github.com/kubearmor/kvm-service/releases/download/0.1/kvmservice_0.1_linux-amd64.deb

Alt

dpkg -i kvmservice_0.1_linux-amd64.deb 

Alt

systemctl status kvmservice 

Alt

Step 3: Install Karmor in control plane

curl -sfL https://raw.githubusercontent.com/kubearmor/kubearmor-client/main/install.sh | sudo sh -s -- -b /usr/local/bin

Alt

Step 4: Onboard VMs using Karmor

cat kvmpolicy1.yaml
apiVersion: security.kubearmor.com/v1
kind: KubeArmorVirtualMachine
metadata:
  name: testvm1
  labels:
    name: vm1
    vm: true

Run this command to add the VM:

karmor vm add kvmpolicy1.yaml

Alt

To see the onboarded VM’s

karmor vm list

Alt

Step 5: Generate installation scripts for configured worker VMs

Generate VM installation scripts for the configured VM by running the following command:

karmor vm --kvms getscript -v testvm1

Alt

Step 6: Execute the installation script in VMs

Note: Docker needs to install before running the script.

Install pre-requisites:

Repositories: /etc/apt/sources.list should include the non-free repository and look something like this:

vi /etc/apt/sources.list

Add the following:

deb http://deb.debian.org/debian sid main contrib non-free
deb-src http://deb.debian.org/debian sid main contrib non-free

Alt

Install build dependencies:

apt-get update

According to

sudo apt-get install arping bison clang-format cmake dh-python \
    dpkg-dev pkg-kde-tools ethtool flex inetutils-ping iperf \
    libbpf-dev libclang-dev libclang-cpp-dev libedit-dev libelf-dev \
    libfl-dev libzip-dev linux-libc-dev llvm-dev libluajit-5.1-dev \
    luajit python3-netaddr python3-pyroute2 python3-distutils python3

Install & compile BCC:

git clone https://github.com/iovisor/bcc.git
mkdir bcc/build; cd bcc/build
cmake ..
make
sudo make install

Install linux-headers:

sudo apt install linux-headers-$(uname -r)

If you getting the following error,

Alt

Follow this steps to slove the error:

apt install gcc-8

Alt

sudo apt install linux-headers-$(uname -r)

Alt

Comment the following line on the script and save it:

vi testvm1.sh
#sudo docker run --name kubearmor $DOCKER_OPTS $KUBEARMOR_IMAGE $KUBEARMOR_OPTS

Alt

Execute the installation script:

Copy the generated installation scripts to appropriate VMs using scp or rsync method and execute the scripts to run Cilium.

The script downloads Cilium Docker images and run them as containers in each VM. Cilium running in each VM connects to the KVM-Service control plane to register themselves and receive information about other VMs in the cluster, labels, IPs and configured security policies.

Execute the script on worker VM by running the following command:

./testvm1.sh

Alt

Note: Make sure the kvm-service is running on control plane VM & To onboard more worker VM repeat Step 4, Step 5 & Step 6.

You can verify by running following command:

sudo docker ps

Alt

Step 7: Install Kubearmor on worker VMs

Download the latest release of KubeArmor

wget https://github.com/kubearmor/KubeArmor/releases/download/v0.3.1/kubearmor_0.3.1_linux-amd64.deb

Alt

dpkg -i kubearmor_0.3.1_linux-amd64.deb 

Note: While installing if you get the following error,

Alt

Run the following command to fix the error:

apt --fix-broken install   
dpkg -i kubearmor_0.3.1_linux-amd64.deb 

Alt

Start & check the status of Kubearmor:

sudo systemctl start kubearmor
sudo systemctl enable kubearmor
sudo systemctl status kubearmor

Alt

Step 8: Apply and Verify Kubearmor system policy

cat khp-example-vmname.yaml
apiVersion: security.kubearmor.com/v1
kind: KubeArmorHostPolicy
metadata:
  name: hsp-kubearmor-dev-proc-path-block
spec:
  process:
    matchPaths:
    - path: /usr/bin/sleep # try sleep 1
  action:
    Block

Run this command to apply the policy:

karmor vm policy add khp-example-vmname.yaml

Alt

Step 9: Policy Violation

sleep 10

Alt

Verifying Policy Violation Logs:

karmor log

Alt

Step 10: Apply and Verify Cilium network policy

1. Allow connectivity with the control plane ( and port 2379)

cat vm-allow-control-plane.yaml
kind: CiliumNetworkPolicy
metadata:
  name: "vm-allow-control-plane"
spec:
  description: "Policy to allow traffic to kv-store"
  nodeSelector:
    matchLabels:
      name: vm1
  egress:
  - toCIDR:
    - 10.138.0.5/32
    toPorts:
    - ports:
      - port: "2379"
        protocol: TCP

2. For SSH connectivity allow port 22 and 169.254.169.254 port 80

cat vm-allow-ssh.yaml
kind: CiliumNetworkPolicy
metadata:
  name: "vm-allow-ssh"
spec:
  description: "Policy to allow SSH"
  nodeSelector:
    matchLabels:
      name: vm1
  egress:
  - toPorts:
    - ports:
      - port: "22"
        protocol: TCP
  - toCIDR:
    - 169.254.169.254/32
    toPorts:
    - ports:
      - port: "80"
        protocol: TCP

3. This policy block the DNS access in VM

cat vm-dns-visibility.yaml
kind: CiliumNetworkPolicy
metadata:
  name: "vm-dns-visibility"
spec:
  description: "Policy to enable DNS visibility"
  nodeSelector:
    matchLabels:
      name: vm1
  egress:
  - toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
        - matchPattern: "*"

4. This policy allow access of “www.google.co.in” alone in VM

cat vm-allow-www-google-co-in.yaml
kind: CiliumNetworkPolicy
metadata:
  name: "vm-allow-www.google.co.in"
spec:
  description: "Policy to allow traffic to www.google.co.in"
  nodeSelector:
    matchLabels:
      name: vm1
  egress:
  - toFQDNs:
    - matchName: www.google.co.in
    toPorts:
    - ports:
      - port: "80"
        protocol: TCP
      - port: "443"
        protocol: TCP

Run this command to apply the policy:

karmor vm --kvms policy add vm-allow-control-plane.yaml
karmor vm --kvms policy add vm-allow-ssh.yaml 
karmor vm --kvms policy add vm-dns-visibility.yaml 
karmor vm --kvms policy add vm-allow-www-google-co-in.yaml 

Alt

Step 11: Policy Violation on worker node

curl http://www.google.co.in/

Alt

curl https://go.dev/

Alt

Verifying Policy Violation Logs:

docker exec -it cilium hubble observe -f -t policy-verdict

Alt

Back to top