Cilium Policy Audit¶
Policy auditing is a feature added to Cilium by Accuknox which enables users to apply network policies in the cluster in the audit mode. Audit mode helps users to understand and observe the effects of a network policy before it gets enforced.
L3/L4 Policy Audit¶
In L3/L4 Cilium network policy specification, users can use auditMode: true
option in combination with ingressDeny
or egressDeny
clauses and apply the network policy in audit mode. When a deny policy is applied in audit mode, the network packets matching such policies will be allowed to flow through the network, but cilium-monitor
logs will provide the following information about the flows.
PolicyName
- Name of the policy with which the flow matches.auditMode
- Is it a audit mode policy (true/false)?action
- What happens if the policy is enforced (allow/deny)?
Users can build tools/scripts which utilize the cilium-monitor
logs to observe/visualize the effects of the network policies before enforcing it.
Note: Audit mode does not have any effect when used with ingress
or egress
clauses in L3/L4 network policies. The behavior of such policies will be as same as a policy which does not have auditMode: true
option.
Demo¶
The following steps showcases how to apply network policies in audit mode using Cilium.
1. Setup¶
Refer to our Cilium installation guide and install Cilium in your kubernetes cluster.
2. Deploy Demo Application¶
kubectl create -f https://raw.githubusercontent.com/accuknox/tools/main/other/deathstar-demo/cilium-deathstar-demo-app.yaml
3. Apply Network Policy¶
a. Allow All¶
Let's apply a policy which enables any pod in the cluster to access deathstar
's HTTP APIs.
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
name: "rule1"
spec:
description: "Allow any pod to access deathstar's HTTP server"
endpointSelector:
matchLabels:
org: empire
class: deathstar
ingress:
- toPorts:
- ports:
- port: "80"
protocol: TCP
kubectl apply -f https://raw.githubusercontent.com/accuknox/tools/main/other/cilium-policy-audit-stats/cilium-cnp-policy-audit-rule1.yaml
b. Deny xwing¶
Let's apply another policy which denies access to xwing
pod.
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
name: "rule2"
spec:
auditMode: true
description: "Deny xwing to access deathstar"
endpointSelector:
matchLabels:
org: empire
class: deathstar
ingressDeny:
- fromEndpoints:
- matchLabels:
class: xwing
toPorts:
- ports:
- port: "80"
protocol: TCP
auditMode: true
, this policy will not be enforced, but it will be just audited.
To apply this audit mode policy, execute the following command.
kubectl apply -f https://raw.githubusercontent.com/accuknox/tools/main/other/cilium-policy-audit-stats/cilium-cnp-policy-audit-rule2.yaml
4. Validate the Policies¶
a. Identity the Cilium Agent Pod¶
In order to validate the audit mode policy, first we have to identity the cilium
pod which is running in the same node as the deathstar
pod.
$ kubectl get pods -l class=deathstar -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
deathstar 1/1 Running 1 (17h ago) 2d13h 10.0.1.131 k8s-master <none> <none>
$ kubectl -n kube-system get pods -l k8s-app=cilium -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
cilium-lrg5j 1/1 Running 1 (17h ago) 35h 10.157.68.207 k8s-worker1 <none> <none>
cilium-m89bs 1/1 Running 2 (17h ago) 35h 10.157.68.25 k8s-master <none> <none>
From the above output, we can identity that, cilium-m89bs
is running in the same node as the deathstar
pod.
b. Run the Cilium Monitor¶
Run the cilium-monitor
process inside the identity cilium
pod.
kubectl -n kube-system exec cilium-m89bs -- cilium monitor -t policy-verdict
c. Access Deathstar's APIs¶
Open another shell and execute the following command.
$ kubectl exec tiefighter -- curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing
Ship landed
$ kubectl exec xwing -- curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing
Ship landed
d. Cilium Policy Verdict¶
Even though both the requests are successful, if you see the output of the cilium-monitor
, you will see a similar output.
Policy verdict log: flow 0x0, PolicyName rule1, local EP ID 262, remote ID 3884, proto 6, ingress, action ALLOW auditMode:false, match all, 10.0.0.53:55238 -> 10.0.1.34:80 tcp SYN
Policy verdict log: flow 0x0, PolicyName rule2, local EP ID 262, remote ID 30448, proto 6, ingress, action DENY auditMode:true, match all, 10.0.0.54:54928 -> 10.0.1.34:80 tcp SYN
The output indicates the following,
-
tiefighter --> deathstar
: Policy verdict isALLOW
because ofrule1
. This is expected because inrule1
we have configured to allow any pod to access deathstar's HTTP APIs. -
xwing --> deathstar
: Policy verdict isDENY
because ofingressDeny
clause inrule2
. But the request was allowed because the policyrule2
is set toauditMode: true
Users can utilize the above information from cilium-monitor
logs and build tools on top of that to observe and visualize the effects of ingressDeny
or egressDeny
policies before enforcing it.