How to forward metrics?
On Prem Feeder¶
- The On-Prem Feeder provides the feasibility of pushing the agent logs to Elastic Host using beats and feeder agent.
- The feeder agent also has the capability of pushing metrics into On Prem Prometheus.
- Prometheus collects and stores its metrics as time series data i.e., metrics information is stored with the timestamp at which it was recorded, alongside optional
key-valuepairs called labels.
- Elasticsearch is a search and analytics engine. It is an open source, full-text search and analysis engine, based on the Apache Lucene search engine.
- Logstash is a log aggregator that collects data from various input sources, executes different transformations and enhancements and then ships the data to various supported output destinations.
- Kibana is a visualization layer that works on top of Elasticsearch, providing users with the ability to analyze and visualize the data. And last but not least — Beats are lightweight agents that are installed on edge hosts to collect different types of data for forwarding into the stack.
The below section explains installation of Elastic and Prometheus (If required)
1. Installation of Elastic¶
Please refer the page for Installation of Elastic.
2. Installation of Prometheus¶
Please refer the page for Installation of Prometheus.
3. Installation of Feeder Service (On Prem)¶
- As we are passing the elastic and kibana resource in the values.yaml of the feeder service , we can toggle the elastic/kibana installation along with feeder-service as below.
helm install feeder-service -n feeder-service
- Once the feeder agent starts running, the metrics should start flowing up.
- Please use
localhost:9092/metricsendpoint to check metrics flow.
- Please add the below configuration in prometheus (above installed
oron Prem) to see the agent metrics
job_name: <feeder>-chart honor_timestamps: true scrape_interval: 30s scrape_timeout: 10s metrics_path: /metrics scheme: http follow_redirects: true static_configs: - targets: - <localhost>:9092
5. Metrics Labelling¶
- The Cilium metrics can be seen under the below label
cilium_<metricname> (Eg, cilium_http_requests_total)
- The Kubearmor metrics can be seen under the below label
kubearmor_<metricname> (Eg, kubearmor_action_requests_total)
- The Vae metrics can be seen under the below label
vae_<metricname> (Eg, vae_Proc_Count_API_requests_total)