Cilium Policy Audit¶
Policy auditing is a feature added to Cilium by Accuknox which enables users to apply network policies in the cluster in the audit mode. Audit mode helps users to understand and observe the effects of a network policy before it gets enforced.
L3/L4 Policy Audit¶
In L3/L4 Cilium network policy specification, users can use
auditMode: true option in combination with
egressDeny clauses and apply the network policy in audit mode. When a deny policy is applied in audit mode, the network packets matching such policies will be allowed to flow through the network, but
cilium-monitor logs will provide the following information about the flows.
PolicyName- Name of the policy with which the flow matches.
auditMode- Is it a audit mode policy (true/false)?
action- What happens if the policy is enforced (allow/deny)?
Users can build tools/scripts which utilize the
cilium-monitor logs to observe/visualize the effects of the network policies before enforcing it.
Note: Audit mode does not have any effect when used with
egress clauses in L3/L4 network policies. The behavior of such policies will be as same as a policy which does not have
auditMode: true option.
The following steps showcases how to apply network policies in audit mode using Cilium.
Refer to our Cilium installation guide and install Cilium in your kubernetes cluster.
2. Deploy Demo Application¶
kubectl create -f https://raw.githubusercontent.com/accuknox/tools/main/other/deathstar-demo/cilium-deathstar-demo-app.yaml
3. Apply Network Policy¶
a. Allow All¶
Let's apply a policy which enables any pod in the cluster to access
deathstar's HTTP APIs.
apiVersion: "cilium.io/v2" kind: CiliumNetworkPolicy metadata: name: "rule1" spec: description: "Allow any pod to access deathstar's HTTP server" endpointSelector: matchLabels: org: empire class: deathstar ingress: - toPorts: - ports: - port: "80" protocol: TCP
kubectl apply -f https://raw.githubusercontent.com/accuknox/tools/main/other/cilium-policy-audit-stats/cilium-cnp-policy-audit-rule1.yaml
b. Deny xwing¶
Let's apply another policy which denies access to
apiVersion: "cilium.io/v2" kind: CiliumNetworkPolicy metadata: name: "rule2" spec: auditMode: true description: "Deny xwing to access deathstar" endpointSelector: matchLabels: org: empire class: deathstar ingressDeny: - fromEndpoints: - matchLabels: class: xwing toPorts: - ports: - port: "80" protocol: TCP
auditMode: true, this policy will not be enforced, but it will be just audited.
To apply this audit mode policy, execute the following command.
kubectl apply -f https://raw.githubusercontent.com/accuknox/tools/main/other/cilium-policy-audit-stats/cilium-cnp-policy-audit-rule2.yaml
4. Validate the Policies¶
a. Identity the Cilium Agent Pod¶
In order to validate the audit mode policy, first we have to identity the
cilium pod which is running in the same node as the
$ kubectl get pods -l class=deathstar -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES deathstar 1/1 Running 1 (17h ago) 2d13h 10.0.1.131 k8s-master <none> <none> $ kubectl -n kube-system get pods -l k8s-app=cilium -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES cilium-lrg5j 1/1 Running 1 (17h ago) 35h 10.157.68.207 k8s-worker1 <none> <none> cilium-m89bs 1/1 Running 2 (17h ago) 35h 10.157.68.25 k8s-master <none> <none>
From the above output, we can identity that,
cilium-m89bs is running in the same node as the
b. Run the Cilium Monitor¶
cilium-monitor process inside the identity
kubectl -n kube-system exec cilium-m89bs -- cilium monitor -t policy-verdict
c. Access Deathstar's APIs¶
Open another shell and execute the following command.
$ kubectl exec tiefighter -- curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing Ship landed $ kubectl exec xwing -- curl -s -XPOST deathstar.default.svc.cluster.local/v1/request-landing Ship landed
d. Cilium Policy Verdict¶
Even though both the requests are successful, if you see the output of the
cilium-monitor, you will see a similar output.
Policy verdict log: flow 0x0, PolicyName rule1, local EP ID 262, remote ID 3884, proto 6, ingress, action ALLOW auditMode:false, match all, 10.0.0.53:55238 -> 10.0.1.34:80 tcp SYN Policy verdict log: flow 0x0, PolicyName rule2, local EP ID 262, remote ID 30448, proto 6, ingress, action DENY auditMode:true, match all, 10.0.0.54:54928 -> 10.0.1.34:80 tcp SYN
The output indicates the following,
tiefighter --> deathstar: Policy verdict is
rule1. This is expected because in
rule1we have configured to allow any pod to access deathstar's HTTP APIs.
xwing --> deathstar: Policy verdict is
rule2. But the request was allowed because the policy
rule2is set to
Users can utilize the above information from
cilium-monitor logs and build tools on top of that to observe and visualize the effects of
egressDeny policies before enforcing it.