Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kube-prometheus-stack] Prometheus targets are empty when using serviceDiscoveryRole: EndpointSlice #4864

Open
deutschj opened this issue Sep 18, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@deutschj
Copy link
Contributor

Describe the bug a clear and concise description of what the bug is.

I'd like to use the serviceDiscoveryRole: EndpointSlice configuration in the kube-prometheus-stack, and therefore manually created EndpointSlice objects, identical to the Endpoints objects that are created by the kube-prometheus-stack when filling out the endpoints: section for a component, e.g. kubeControllerManager.
The EndpointSlice objects have the correct endpoint IP addresses and ports in them. However, they don't seem to be recognized by Prometheus, as the corresponding target in the Prometheus UI is empty and no metrics for the target are displayed.

Support for the EndpointSlice role was introduced to Prometheus Operator in v0.76.0: prometheus-operator/prometheus-operator#3862

I wanted to ask whether the discovery of EndpointSlice objects using the kube-prometheus-stack is already supported, and would be very glad if someone could help me with configuring this the right way.

What's your helm version?

v3.10.1

What's your kubectl version?

v1.26.0

Which chart?

kube-prometheus-stack

What's the chart version?

v62.3.1

What happened?

Set serviceDiscoveryRole: EndpointSlice and created EndpointSlice objects for kubeControllerManager and kubeScheduler with the correct IP addresses and ports. However, the EndpointSlice objects are not recognized by the Prometheus ServiceMonitors created when installing the kube-prometheus-stack, and metrics for these targets are missing (targets section in the Prometheus UI is empty as well).

What you expected to happen?

EndpointSlice objects should show up as targets in the Prometheus UI and metrics should be successfully scraped from them.

How to reproduce it?

No response

Enter the changed values of values.yaml?

prometheus:
prometheusSpec:
serviceDiscoveryRole: EndpointSlice

Enter the command that you execute and failing/misfunctioning.

helm install kube-prom-stack -f values.yaml -n monitoring --version 62.3.1 prometheus-community/kube-prometheus-stack

Anything else we need to know?

No response

@deutschj deutschj added the bug Something isn't working label Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant