How to forward metrics?

On Prem Feeder¶

The On-Prem Feeder provides the feasibility of pushing the agent logs to Elastic Host using beats and feeder agent.
The feeder agent also has the capability of pushing metrics into On Prem Prometheus.
Prometheus collects and stores its metrics as time series data i.e., metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.
Elasticsearch is a search and analytics engine. It is an open source, full-text search and analysis engine, based on the Apache Lucene search engine.
Logstash is a log aggregator that collects data from various input sources, executes different transformations and enhancements and then ships the data to various supported output destinations.
Kibana is a visualization layer that works on top of Elasticsearch, providing users with the ability to analyze and visualize the data. And last but not least — Beats are lightweight agents that are installed on edge hosts to collect different types of data for forwarding into the stack.

The below section explains installation of Elastic and Prometheus (If required)

1. Installation of Elastic¶

Please refer the page for Installation of Elastic.

2. Installation of Prometheus¶

Please refer the page for Installation of Prometheus.

3. Installation of Feeder Service (On Prem)¶

As we are passing the elastic and kibana resource in the values.yaml of the feeder service , we can toggle the elastic/kibana installation along with feeder-service as below.
```
helm install feeder-service -n feeder-service
```

4. Metrics¶

Once the feeder agent starts running, the metrics should start flowing up.
Please use localhost:9092/metrics endpoint to check metrics flow.

Prometheus Configuration:¶

Please add the below configuration in prometheus (above installed or on Prem) to see the agent metrics

  job_name: <feeder>-chart
  honor_timestamps: true
  scrape_interval: 30s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  follow_redirects: true
  static_configs:
  - targets:
    - <localhost>:9092

5. Metrics Labelling¶

The Cilium metrics can be seen under the below label

   cilium_<metricname> (Eg, cilium_http_requests_total)

The Kubearmor metrics can be seen under the below label

    kubearmor_<metricname> (Eg, kubearmor_action_requests_total)

The Vae metrics can be seen under the below label

  vae_<metricname> (Eg, vae_Proc_Count_API_requests_total)