Metrics Aggregator

[DirectXMan12]

In some cases, it would be nice to be able to have a single endpoint which was able to aggregate multiple different providers of the custom metrics API into a single implementation of the custom metrics API.

This could be useful if one is manually sharding metrics collection in a large cluster, or has multiple different types of metrics collection in a cluster.

[DirectXMan12]

cc @luxas @smarterclayton

[luxas]

+1 we talked about this.

Will you create an example here?

[vangarp]

From reading at other places (like kubernetes-sigs/prometheus-adapter#45) seems like this is not possible at all right now. @luxas you mentioned you have an example on how one can do this?

[markusthoemmes]

@DirectXMan12 Could you line out your thoughts on how to achieve this a little more? We have this exact requirement for our Knative setup.

[DirectXMan12]

There'd be a few different ways to do this. The custom metrics API is set up such that it's possible to use discovery to find which metrics a given server provides, and thus dispatching on that information should be fairly easy. You'd then need a CRD representing a given source (probably a service name,namespace tuple with some metadata attached). The CRD could be expanded to explicitly choose certain metrics if desired.

At that point, you'd have to make a decision on how to handle duplicate metric names across servers. You could have registration allow claiming particular namespaces, or implicitly claim the namespace the service is in.

I'm curious as to what the usecase in Knative is, though.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[weikinhuang]

Hi, can we reopen this issue. I would like to use DirectXMan12/k8s-prometheus-adapter for pods not managed by knative, and have the knative autoscaler handle knative metrics.

Right now when you install knative serving, it installs it's own custom.metrics.k8s.io registration, making prometheus-adapter not able to be installed at the same time as it.

[k8s-ci-robot]

@weikinhuang: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[markusthoemmes]

/reopen
/remove-lifecycle rotten

[k8s-ci-robot]

@markusthoemmes: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen
/remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[SleepyBrett]

We were thinking about a proxy metrics adapter. So y9ou would wrap your metrics query in some syntax .. say prometheus(<prometheusquery>) and the write the metrics proxy to parse that and send it along to the proper endpoint based on the provider key.

[bradleyhession1]

Have any attempts been made to create a solution to this yet? We have a requirement for using both StackDriver + Prometheus together with HPA's

[tomkerkhove]

And KEDA (https://github.com/kedacore/keda) has customers who are using other metric providers as well.

[k8s-ci-robot]

@Alsheh: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.