Skip to content

Cilium Policy Audit

Policy auditing is a feature added to Cilium by Accuknox which enables users to apply network policies in the cluster in the audit mode. Audit mode helps users to understand and observe the effects of a network policy before it gets enforced.

L3/L4 Policy Audit

In L3/L4 Cilium network policy specification, users can use auditMode: true option in combination with ingressDeny or egressDeny clauses and apply the network policy in audit mode. When a deny policy is applied in audit mode, the network packets matching such policies will be allowed to flow through the network, but cilium-monitor logs will provide the following information about the flows.

  • PolicyName - Name of the policy with which the flow matches.
  • auditMode - Is it a audit mode policy (true/false)?
  • action - What happens if the policy is enforced (allow/deny)?

Users can build tools/scripts which utilize the cilium-monitor logs to observe/visualize the effects of the network policies before enforcing it.

Note: Audit mode does not have any effect when used with ingress or egress clauses in L3/L4 network policies. The behavior of such policies will be as same as a policy which does not have auditMode: true option.

Demo

The following steps showcases how to apply network policies in audit mode using Cilium.

1. Setup

Refer to our Cilium installation guide and install Cilium in your kubernetes cluster.

2. Deploy Demo Application

kubectl create -f https://raw.githubusercontent.com/accuknox/tools/main/other/deathstar-demo/cilium-deathstar-demo-app.yaml
The demo application has the following topology.

Alt

3. Apply Network Policy

a. Allow All

Let's apply a policy which enables any pod in the cluster to access deathstar's HTTP APIs. 

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "rule1"
spec:
  description: "Allow any pod to access deathstar's HTTP server"
  endpointSelector:
    matchLabels:
      org: empire
      class: deathstar
  ingress:
  - toPorts:
    - ports:
      - port: "80"
        protocol: TCP
To apply the above policy, execute the following command. 
kubectl apply -f https://raw.githubusercontent.com/accuknox/tools/main/other/cilium-policy-audit-stats/cilium-cnp-policy-audit-rule1.yaml

b. Deny xwing

Let's apply another policy which denies access to xwing pod. 

apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "rule2"
spec:
  auditMode: true
  description: "Deny xwing to access deathstar"
  endpointSelector:
    matchLabels:
      org: empire
      class: deathstar
  ingressDeny:
  - fromEndpoints:
    - matchLabels:
        class: xwing
    toPorts:
    - ports:
      - port: "80"
        protocol: TCP
In the usual case, this policy will be enforced. Since we have configured the above policy with auditMode: true, this policy will not be enforced, but it will be just audited.

To apply this audit mode policy, execute the following command. 

kubectl apply -f https://raw.githubusercontent.com/accuknox/tools/main/other/cilium-policy-audit-stats/cilium-cnp-policy-audit-rule2.yaml

4. Validate the Policies

a. Identity the Cilium Agent Pod

In order to validate the audit mode policy, first we have to identity the cilium pod which is running in the same node as the deathstar pod. 

$ kubectl get pods -l class=deathstar -o wide
NAME                        READY   STATUS    RESTARTS      AGE     IP           NODE         NOMINATED NODE   READINESS GATES
deathstar                   1/1     Running   1 (17h ago)   2d13h   10.0.1.131   k8s-master   <none>           <none>

$ kubectl -n kube-system get pods -l k8s-app=cilium -o wide
NAME           READY   STATUS    RESTARTS      AGE   IP              NODE          NOMINATED NODE   READINESS GATES
cilium-lrg5j   1/1     Running   1 (17h ago)   35h   10.157.68.207   k8s-worker1   <none>           <none>
cilium-m89bs   1/1     Running   2 (17h ago)   35h   10.157.68.25    k8s-master    <none>           <none>

From the above output, we can identity that, cilium-m89bs is running in the same node as the deathstar pod.

b. Run the Cilium Monitor

Run the cilium-monitor process inside the identity cilium pod. 

kubectl -n kube-system exec cilium-m89bs -- cilium monitor -t policy-verdict

c. Access Deathstar's APIs

Open another shell and execute the following command. 

$ kubectl exec tiefighter -- curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing
Ship landed
$ kubectl exec xwing -- curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing
Ship landed

d. Cilium Policy Verdict

Even though both the requests are successful, if you see the output of the cilium-monitor, you will see a similar output. 

Policy verdict log: flow 0x0, PolicyName rule1, local EP ID 262, remote ID 3884, proto 6, ingress, action ALLOW auditMode:false, match all, 10.0.0.53:55238 -> 10.0.1.34:80 tcp SYN

Policy verdict log: flow 0x0, PolicyName rule2, local EP ID 262, remote ID 30448, proto 6, ingress, action DENY auditMode:true, match all, 10.0.0.54:54928 -> 10.0.1.34:80 tcp SYN

The output indicates the following,

  1. tiefighter --> deathstar: Policy verdict is ALLOW because of rule1. This is expected because in rule1 we have configured to allow any pod to access deathstar's HTTP APIs.

  2. xwing --> deathstar: Policy verdict is DENY because of ingressDeny clause in rule2. But the request was allowed because the policy rule2 is set to auditMode: true

Users can utilize the above information from cilium-monitor logs and build tools on top of that to observe and visualize the effects of ingressDeny or egressDeny policies before enforcing it.

Back to top