How to create a ServiceMonitor in Microk8s: Observability Tutorial
Hello,
In this quick tutorial we will explore how to create a ServiceMonitor in microk8s when you have the observability addon enabled.
When you enable the observability addon with:
microk8s enable observabilityIt will create the obserability namespace and install the Kubernetes Prometheus Stack.
k get pods -n observabilityNAME READY STATUS RESTARTS AGEkube-prom-stack-prometheus-node-exporter-htzw9 1/1 tempo-0 2/2 Running 4 (16h ago) 27halertmanager-kube-prom-stack-kube-prome-alertmanager-0 2/2 kube-prom-stack-kube-prome-operator-cbf896985-fb5fc 1/1 kube-prom-stack-kube-state-metrics-57c8c84df6-hflgj 1/1 kube-prom-stack-grafana-74d8979894-sjv77 3/3 prometheus-kube-prom-stack-kube-prome-prometheus-0 2/2 loki-0 1/1 loki-promtail-sg8bv 1/1 Running You will get a Grafana configured with a Prometheus data source and other data sources as well.
To monitor a custom service, let’s say a prometheus metrics exporter for Apache Kafka you will need to create a ServiceMonitor. For the service monitor to be discovered you’ll need to add some labels to it.
To find out the labels you need to add describe the prometheus operator:
k describe prometheus/kube-prom-stack-kube-prome-prometheus -n observability[...] Pod Monitor Namespace Selector: Pod Monitor Selector: Match Labels: Release: kube-prom-stack Service Account Name: kube-prom-stack-kube-prome-prometheus Service Monitor Namespace Selector: Service Monitor Selector: Match Labels: Release: kube-prom-stackThen create the ServiceMonitor with the corresponding labels: kube-prom-stack.
apiVersion: monitoring.coreos.com/v1kind: ServiceMonitormetadata: name: prometheus-exporter-kafka-monitor labels: team: observability release: kube-prom-stackspec: selector: matchLabels: app: prometheus-kafka-exporter app.kubernetes.io/managed-by: Helm chart: prometheus-kafka-exporter-1.8.0 heritage: Helm release: prometheus-exporter endpoints: - port: exporter-portThanks for reading!
