Setup Instructions¶

This install instructions allow you to setup sample k8s cluster with:

Cilium CNI
Kubearmor Application Protection Engine
Auto Policy Discovery Engine
Command line tools

Alt Text

1. Install sample k8s cluster¶

Local k3s clusterEKS cluster

Install k3s

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--flannel-backend=none --disable traefik' sh -s - --write-kubeconfig-mode 644

Make k3s cluster config the default

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

You can add this to your .bashrc. (Note that any other k8s cluster will not be reachable using kubectl once you override KUBECONFIG).

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: kubearmor-ub20
  region: us-east-2

nodeGroups:
  - name: ng-1
    amiFamily: "Ubuntu2004"
    privateNetworking: true
    desiredCapacity: 2
    # taint nodes so that application pods are
    # not scheduled until Cilium is deployed.
    taints:
     - key: "node.cilium.io/agent-not-ready"
       value: "true"
       effect: "NoSchedule"
    ssh:
      allow: true
    preBootstrapCommands:
      - "sudo apt install linux-headers-$(uname -r)"

2. Install Daemonsets and Services¶

curl -s https://raw.githubusercontent.com/accuknox/tools/main/install.sh | bash

This will install all the components.

Output from kubectl get pods -A

NAMESPACE     NAME                                             READY   STATUS      RESTARTS   AGE
kube-system   helm-install-traefik-crd-gwlpt                   0/1     Completed   0          3h17m
kube-system   helm-install-traefik-lzkqg                       0/1     Completed   1          3h17m
kube-system   svclb-traefik-47bc4                              2/2     Running     2          3h9m
kube-system   metrics-server-86cbb8457f-cw9jd                  1/1     Running     1          3h9m
kube-system   local-path-provisioner-7c7846d5f8-kxdxj          1/1     Running     1          3h3m
kube-system   coredns-7448499f4d-qk6pv                         1/1     Running     0          15m
kube-system   traefik-5ffb8d6846-w8clc                         1/1     Running     1          3h3m
kube-system   cilium-operator-6bbdb895b5-ff752                 1/1     Running     0          12m
kube-system   hubble-relay-84999fcb48-8d5ss                    1/1     Running     0          11m
kube-system   cilium-wkgzn                                     1/1     Running     0          11m
explorer      mysql-0                                          1/1     Running     0          10m
kube-system   kubearmor-67jtk                                  1/1     Running     0          8m34s
kube-system   kubearmor-policy-manager-986bd8dbc-4s79d         2/2     Running     0          8m34s
kube-system   kubearmor-host-policy-manager-5bcccfc4f5-gkbck   2/2     Running     0          8m34s
kube-system   kubearmor-relay-645667c695-brzpg                 1/1     Running     0          8m34s
explorer      knoxautopolicy-6bf6c98dbb-pfwt9                  1/1     Running     0          8m20s

We have following installed:

kubearmor protection engine
cilium CNI
Auto policy discovery engine
MySQL database to keep discovered policies
Hubble Relay and KubeArmor Relay

3. Install Sample k8s application¶

Install anyone of the following app or you can try your own k8s app.

Wordpress-Mysql AppOnline Boutique: Google Microservice Demo App

kubectl apply -f https://raw.githubusercontent.com/kubearmor/KubeArmor/main/examples/wordpress-mysql/wordpress-mysql-deployment.yaml

Application Reference

kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/microservices-demo/master/release/kubernetes-manifests.yaml

4. Get Auto Discovered Policies¶

curl -s https://raw.githubusercontent.com/accuknox/tools/main/get_discovered_yamls.sh | bash

Sample Output

❯ curl -s https://raw.githubusercontent.com/accuknox/tools/main/get_discovered_yamls.sh | bash
Downloading discovered policies from pod=knoxautopolicy-6bf6c98dbb-pfwt9
Got 38 cilium policies for namespace=default in file cilium_policies_default.yaml
Got 4 cilium policies for namespace=explorer in file cilium_policies_explorer.yaml
Got 13 cilium policies for namespace=kube-system in file cilium_policies_kube-system.yaml
Got 38 knox policies for namespace=default in file knox_net_policies_default.yaml
Got 4 knox policies for namespace=explorer in file knox_net_policies_explorer.yaml
Got 13 knox policies for namespace=kube-system in file knox_net_policies_kube-system.yaml
Got 146 kubearmor policies in file kubearmor_policies.yaml

5. Metrics and Logs¶

The On-Prem Feeder provides the feasibility of pushing the agent logs to Elastic Host using beats and feeder agent.
The On-Prem Feeder agent also has the capability of pushing metrics into On Prem Prometheus.
Prometheus collects and stores its metrics as time series data i.e., metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.
Elasticsearch is a search and analytics engine. It is an open source, full-text search and analysis engine, based on the Apache Lucene search engine.
Logstash is a log aggregator that collects data from various input sources, executes different transformations and enhancements and then ships the data to various supported output destinations.
Kibana is a visualization layer that works on top of Elasticsearch, providing users with the ability to analyze and visualize the data. And last but not least — Beats are lightweight agents that are installed on edge hosts to collect different types of data for forwarding into the stack.

The below section explains installation of Feeder Agent , Elastic(Optional) and Prometheus(Optional)

5.1. Installation of Feeder Agent¶

As we are passing the elastic and kibana resource in the values.yaml of the feeder service , we can toggle the elastic/kibana installation along with feeder-service as below.

helm repo add accuknox-onprem-agents https://USERNAME:[email protected]/repository/accuknox-onprem-agents
helm repo update
helm search repo accuknox-onprem-agents

kubectl create ns accuknox-feeder-service
helm upgrade --install accuknox-eck-operator accuknox-onprem-agents/eck-operator (only if ELK is required)
helm upgrade --install accuknox-feeder-service accuknox-onprem-agents/feeder-service -n accuknox-feeder-service

5.2. Installation of Elastic¶

Please enable the elastic resource as true to install Elastic along with feeder.

helm upgrade --install --set elasticsearch.enabled=true --set kibana.enabled=true accuknox-feeder-service accuknox-onprem-agents/feeder-service -n accuknox-feeder-service

Note: If there is ELK set up already running on the cluster, the CRD apply may fail.

The Elastic master and data pods should be in up and running state on the same namespace.
Additionally the same can be enabled using below command by updating values.yaml
```
elasticsearch:
  enabled: true
```

5.3. Installation of Kibana¶

Please enable the Kibana resource as true to install Kibana along with feeder.

helm upgrade --install --set elasticsearch.enabled=true  --set kibana.enabled=true accuknox-feeder-service accuknox-onprem-agents/feeder-service -n accuknox-feeder-service

The Kibana pods should be in up and running state on the same namespace.
Additionally the same can be enabled using below command by updating values.yaml
```
kibana:
  enabled: true
```

5.4. View Metrics¶

Feeder as a SERVER

Please toggle the below variable for to push metrics directly to an endpoint.

GRPC_SERVER_ENABLED
  value: true
 ```
- Once the feeder agent starts running, the metrics should start flowing up.
- Please use `localhost:8000/metrics` endpoint to check metrics flow.

<b> Feeder as a CLIENT</b>

- Please toggle the below variable for to push metrics to GRPC Server in SAAS Platform.
```yaml
GRPC_CLIENT_ENABLED
  value: true
GRPC_SERVER_URL
  value: "localhost"
GRPC_SERVER_PORT
  value: 8000

Once the feeder agent starts running, the metrics will be pushed to prometheus in SAAS and can be viewed in ACCUKNOX platform UI. Note: All of the above can be updated runtime as in Step 5.5.3

5.4.1 Installation of Prometheus (Required only when Feeder acts as Server)¶

Please refer the page for Installation of Prometheus.

5.4.2 Prometheus Configuration:(Required only when Feeder acts as Server)¶

Please add the below configuration in prometheus (on Prem) to see the agent metrics in Prometheus

  job_name: <feeder>-chart
  honor_timestamps: true
  scrape_interval: 30s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  follow_redirects: true
  static_configs:
  - targets:
    - <localhost>:8000

5.5. View Logs in Elastic¶

5.5.1. Beats Setup¶

The Beats agent will be spinned along with Filebeat running along as a sidecar.
The filebeat configuration file in the package can be updated to specific Elastic instances, and logs can be viewed in Kibana.
The logs are forwarded to Elastic when the below env variable is enabled.
```
- name: ELASTIC_FEEDER_ENABLED
    value: true
```

5.5.2. Elastic Configuration Parameters:¶

We will create a ConfigMap named filebeat-configmap with the content of filebeat.yml file.

kind: ConfigMap
metadata:
  name: filebeat-configmap
data:
  filebeat.yml: |
    filebeat.inputs:
    - type: log

      # Change to true to enable this input configuration.
      enabled: true

      # Paths that should be crawled and fetched. Glob based paths.
      paths:
      - /var/log/*.log
    output.elasticsearch:
      hosts: ${ELASTICSEARCH_HOST}
      username: ${ELASTICSEARCH_USERNAME}
      password: ${ELASTICSEARCH_PASSWORD}
      ssl.verification_mode: none

The below Configuration parameters can be updated for elastic configuration.

(If Default params needs to be modified)

 - name: ELASTICSEARCH_HOST
        value: https://<svc-name>
 - name: ELASTICSEARCH_PORT
        value: "<svc-port>"
 - name: ELASTICSEARCH_USERNAME
        value: "elastic"
 - name: ELASTICSEARCH_PASSWORD
        value: "<elastic-password>"

To get elastic password

kubectl get secret elasticsearch-es-elastic-user -o go-template='{{.data.elastic | base64decode}}' -n namespace

5.5.3. Updating Elastic Search Host (Runtime):(If required to switch different Elastic host)¶

kubectl set env deploy/feeder-service -n accuknox-feeder-service  ELASTICSEARCH_HOST="https://elasticsearch-es-http.test-feed.svc.cluster.local:9200" ELASTICSEARCH_USERNAME=elastic ELASTICSEARCH_PASSWORD=xxxxxxxxxx

- Note: Likewise other configuration parameters can be updated in Runtime.

5.5.4. Validate Log Path:¶

To view logs and to check filebeat (status) please use the below command

  kubectl exec -it -n accuknox-feeder-service pod/<podname> -c filebeat-sidecar -- /bin/bash
  filebeat -e

To Update the Log path configured, please modify the below log input path under file beat inputs.

filebeat.inputs:
        - type: container
        paths:
        - /log_output/cilium.log

5.6. View Logs in Splunk¶

The logs are forwarded to Splunk when the below env variable is enabled.
```
- name: SPLUNK_FEEDER_ENABLED
    value: true
```

The below Configuration parameters can be updated for Splunk configuration.

(If Default params needs to be modified)

 - name: SPLUNK_FEEDER_URL
        value: https://<splunk-host>
 - name: SPLUNK_FEEDER_TOKEN
        value: "Token configured on HEC in Splunk App"
 - name: SPLUNK_FEEDER_SOURCE_TYPE
        value: "Source Type configured on HEC in Splunk App"
 - name: SPLUNK_FEEDER_SOURCE
        value: "Splunk Source configured on HEC in Splunk App"
 - name: SPLUNK_FEEDER_INDEX
     value: "Splunk Index configured on HEC in Splunk App"

5.6.1. Enabling/Disabling Splunk (Runtime):¶

kubectl set env deploy/feeder -n feeder-service SPLUNK_FEEDER_ENABLED="true"

- By enabling the flag to true (as above), the logs will be pushed to Splunk.Conversely disabling it to "false" will stop pushing logs. - Note: Likewise other configuration parameters can be updated in Runtime.

6. Uninstall¶

curl -s https://raw.githubusercontent.com/accuknox/tools/main/uninstall.sh | bash