Setup server-side filter of the pods

[ahg-g]

What would you like to be added:

Currently the pod controller filters the pods at the epp side, the filter is primarily the pool selector and ready status. At scale this could be expensive. To address this, we should setup the watch to perform server side filter.

In controller-runtime, we can do this using the ByObject cache option: https://github.com/kubernetes-sigs/controller-runtime/blob/6f611112490242f9e8d5b0ccdc0f3bb8c86c85ef/pkg/cache/cache.go#L240

The tricky part is that the filter depends on the pool, which may change during runtime, and so we need to think of a way to update the filter on the pod informer/cache.

Why is this needed:
To improve the scalability of the epp.

[hzxuzhonghu]

This is easy to use native informer. But the controller-runtime seems only allow static filter, it may need to implement a wrapper interface

[ahg-g]

Does informer allow updating the server-side selector after it was started? My understanding is that it requires a recreate also.

[hzxuzhonghu]

Does informer allow updating the server-side selector after it was started?

No, but we can update filter on client-side easily. Istio pkg/kube/kclient/client.go has already implemented client-side filter

[nirrozenbaum]

seems like we've done the maximum possible filtering (as long as we use controller-runtime).
pods are filtered by pool ns in server side and by pool selector labels on client side.

I think this issue can be closed.