History from Prometheus: CPU and memory over time

Pods, Deployments, StatefulSets and DaemonSets have a Metrics tab after Overview. It charts CPU and memory over time, from the Prometheus in your cluster. The pod's request and limit show as reference lines.

It needs no paid plan. The Utilization card on Overview has a History link that opens the tab.

• Pick a range: 15m, 1h, 6h, 24h, 7d or 30d.
• Drag across a chart to zoom into a time window. Click the × on the window chip to go back.
• Per pod shows the top 8 pods by average use over the whole range. For a Pod, the switch is Per container.
• The tab refreshes every 30 seconds on short ranges and every 5 minutes on long ranges, while the tab is open.

Glauco reads container_cpu_usage_seconds_total and container_memory_working_set_bytes.

Glauco asks Prometheus through the Kubernetes API server, using the service proxy and your own kubeconfig credentials. Nothing goes to Koode. Glauco does not open a port-forward or any local port.

In Auto mode, Glauco looks for a service in this order:

1. The Prometheus Operator or kube-prometheus-stack: services labelled operated-prometheus=true, then self-monitor=true.
2. The Prometheus Helm chart: app.kubernetes.io/name=prometheus,app.kubernetes.io/component=server, or the older app=prometheus,component=server.
3. Lens metrics: lens-metrics/prometheus.
4. A service named like prometheus with port 9090.

Glauco prefers the port named web or http, and otherwise uses 9090. It checks each candidate with a query for cAdvisor data. It remembers the result for 10 minutes per cluster.

Open Settings and go to Prometheus. The settings are per cluster.

Auto

Glauco finds Prometheus, as above.

Off

Hides the Metrics tab.

Manual

You give the Namespace, Service, Port, Scheme (http or https) and Path prefix. The prefix can be /prometheus, or /select/0/prometheus for a VictoriaMetrics cluster.

Click Test to check the address. There is also a default range.

Your account needs get on services/proxy in the namespace of Prometheus. Check it in a terminal:

kubectl auth can-i get services/proxy -n monitoring

If the answer is no, ask your cluster admin for a minimal Role like this one:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: prometheus-proxy
  namespace: monitoring
rules:
  - apiGroups: [""]
    resources: ["services/proxy"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: prometheus-proxy
  namespace: monitoring
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: prometheus-proxy
subjects:
  - kind: User
    name: you@example.com
    apiGroup: rbac.authorization.k8s.io

Change the namespace to the one where Prometheus runs. Change the subject to your user or group.

The chart can start later than the range you pick, and Glauco shows Data from and a date. Prometheus keeps a limited history. kube-prometheus-stack keeps 10 days by default, so a long range starts later.

• Glauco shows

  No Prometheus found

  Glauco found no Prometheus in Auto mode. Install kube-prometheus-stack, or set the address in Settings, as shown above.

• Glauco shows

  Needs get services/proxy in monitoring

  Your account lacks the permission, and the message names the namespace. See The permission it needs.

• Glauco shows

  The API server could not reach Prometheus

  The API server cannot reach the Prometheus pod. Two common causes:

  • A private GKE cluster. Add a firewall rule from the control plane to the Prometheus pod port.
  • EKS with an overlay CNI. The control plane cannot reach pod IPs.

  Fix the network, or use Manual with another address that the API server can reach.

• Glauco shows

  Prometheus did not answer in time

  Try a shorter range.

• Glauco shows

  This is not a Prometheus answer

  The address answers, but not as Prometheus. Check the address and the Path prefix.

• Glauco shows

  No CPU data

  Prometheus does not scrape the cAdvisor metrics of the kubelet. Set up that scrape in Prometheus.

For the live bars without history, see CPU and memory bars and Metrics unavailable: install metrics-server.