Move filter and scorer plugins registration to a separate file #729

mayabar · 2025-04-23T08:23:20Z

fix issue [#718]

linux-foundation-easycla · 2025-04-23T08:23:25Z

The committers listed above are authorized under a signed CLA.

✅ login: mayabar / name: Maya Barnea (66c4cc2, e6b29fc, b4c6630, 7f3b120)

k8s-ci-robot · 2025-04-23T08:23:29Z

Welcome @mayabar!

It looks like this is your first PR to kubernetes-sigs/gateway-api-inference-extension 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api-inference-extension has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2025-04-23T08:23:30Z

Hi @mayabar. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

netlify · 2025-04-23T08:23:37Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`66c4cc2`
🔍 Latest deploy log	https://app.netlify.com/sites/gateway-api-inference-extension/deploys/680a3b8d164e9d0008bf0dd8
😎 Deploy Preview	https://deploy-preview-729--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

mayabar · 2025-04-23T09:11:00Z

/assign @nirrozenbaum

nirrozenbaum · 2025-04-23T09:27:54Z

/ok-to-test

kfswain · 2025-04-23T17:34:19Z

Looks like a noop PR & heremetic tests pass so the filters should be WAI, won't ask for benchmarking on this PR.
/lgtm
/approve

k8s-ci-robot · 2025-04-23T17:34:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kfswain, mayabar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kfswain]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pkg/epp/scheduling/default_config.go

pkg/epp/scheduling/scheduler.go

nirrozenbaum · 2025-04-24T13:08:24Z

pkg/epp/scheduling/scheduler_test.go

 	for _, test := range tests {
 		t.Run(test.name, func(t *testing.T) {
-			scheduler := NewScheduler(&fakeDataStore{pods: test.input})
+			scheduler := NewSchedulerWithConfig(&fakeDataStore{pods: test.input}, schedConfig)


seems like schedulerConfig here is the default, you can init with NewScheduler :)

nirrozenbaum · 2025-04-24T13:09:08Z

overall looks awesome!
left just a few very minor comments

Signed-off-by: Maya Barnea <[email protected]>

nirrozenbaum · 2025-04-24T15:47:34Z

/lgtm

…netes-sigs#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <[email protected]> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <[email protected]> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <[email protected]> --------- Signed-off-by: Maya Barnea <[email protected]>

* Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * EPP Architecture proposal (#683) * initial changes * Adding to proposal to give a quick barebones definition to refactor * feedback changes * more feedback addressing * removed unused Fake struct (#723) Signed-off-by: Nir Rozenbaum <[email protected]> * epp: return correct response for trailers (#726) This looks like a copy paste error. * Refactor scheduler to run plugins (#677) * Refactor scheduler to run plugins * Add scheduler plugin latency metric * Address comments * Address comments * Complete the InferencePool documentation (#673) * Initial guide for inference pool * Add extensionReference to the InferencePool spec * Fix list formatting * Remove unused labels * Autogenerate the spec * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Rename llm-pool names in rollout example * Add use cases for replacing an inference pool * Rewording the background section * Create replacing-inference-pool.md * Replace instructions with a link for how to replace an inference pool * Update replacing-inference-pool.md * Update mkdocs.yml * Update replacing-inference-pool.md * Update inferencemodel_types.go * Update inferencepool.md * Update site-src/guides/replacing-inference-pool.md Co-authored-by: Rob Scott <[email protected]> --------- Co-authored-by: Rob Scott <[email protected]> * reduce log level in metrics logger not to trash the log (#708) * reduce log level in metrics logger not to trash the log Signed-off-by: Nir Rozenbaum <[email protected]> * rename flush metrics to refresh metrics Signed-off-by: Nir Rozenbaum <[email protected]> * revert log level Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * few updates in datastore (#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <[email protected]> * PoolSet documentation Signed-off-by: Nir Rozenbaum <[email protected]> * error phrasing Signed-off-by: Nir Rozenbaum <[email protected]> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <[email protected]> * linter Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * scheduler refactoring (#730) Signed-off-by: Nir Rozenbaum <[email protected]> * filter irrelevant pod in pod_reconciler (#696) * EPP: Update GetRandomPod() to return nil if no pods exist (#731) Signed-off-by: Daneyon Hansen <[email protected]> * Move filter and scorer plugins registration to a separate file (#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <[email protected]> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <[email protected]> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <[email protected]> --------- Signed-off-by: Maya Barnea <[email protected]> * Update issue templates (#738) * Update issue templates * Updates artifacts for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <[email protected]> * Updates bbr chart for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <[email protected]> * Updates artifacts for v0.3.0 release Signed-off-by: Kellen Swain <[email protected]> * Adding blank issue template so that all issues start with label --------- Signed-off-by: Kellen Swain <[email protected]> * Add unit test coverage for pod APIs under datastore/pkg * few updates in datastore (#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <[email protected]> * PoolSet documentation Signed-off-by: Nir Rozenbaum <[email protected]> * error phrasing Signed-off-by: Nir Rozenbaum <[email protected]> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <[email protected]> * linter Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * few updates in datastore (#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <[email protected]> * PoolSet documentation Signed-off-by: Nir Rozenbaum <[email protected]> * error phrasing Signed-off-by: Nir Rozenbaum <[email protected]> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <[email protected]> * linter Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * Add unit test coverage for pod APIs under datastore/pkg --------- Signed-off-by: Nir Rozenbaum <[email protected]> Signed-off-by: Daneyon Hansen <[email protected]> Signed-off-by: Maya Barnea <[email protected]> Signed-off-by: Kellen Swain <[email protected]> Co-authored-by: Kellen Swain <[email protected]> Co-authored-by: Nir Rozenbaum <[email protected]> Co-authored-by: John Howard <[email protected]> Co-authored-by: Cong Liu <[email protected]> Co-authored-by: Nicole Xin <[email protected]> Co-authored-by: Rob Scott <[email protected]> Co-authored-by: nayihz <[email protected]> Co-authored-by: Daneyon Hansen <[email protected]> Co-authored-by: Maya Barnea <[email protected]>

* main: fixing errors in new template & disabling the default blank template (kubernetes-sigs#742) Updating proposal directories to match their PR number (kubernetes-sigs#741) added a target dedicated for running unit-test only (kubernetes-sigs#739) Add unit tests for pod APIs under pkg/datastore (kubernetes-sigs#712) docs: add concepts and definitions to README.md (kubernetes-sigs#734) Update issue templates (kubernetes-sigs#738) Move filter and scorer plugins registration to a separate file (kubernetes-sigs#729) EPP: Update GetRandomPod() to return nil if no pods exist (kubernetes-sigs#731) filter irrelevant pod in pod_reconciler (kubernetes-sigs#696) scheduler refactoring (kubernetes-sigs#730) # Conflicts: # pkg/epp/scheduling/plugins/picker.go # pkg/epp/scheduling/plugins/picker/random_picker.go # pkg/epp/scheduling/plugins/pickers/random.go # pkg/epp/scheduling/scheduler.go # pkg/epp/scheduling/scheduler_test.go

…netes-sigs#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <[email protected]> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <[email protected]> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <[email protected]> --------- Signed-off-by: Maya Barnea <[email protected]>

* Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * Add unit test coverage for pod APIs under datastore/pkg * EPP Architecture proposal (kubernetes-sigs#683) * initial changes * Adding to proposal to give a quick barebones definition to refactor * feedback changes * more feedback addressing * removed unused Fake struct (kubernetes-sigs#723) Signed-off-by: Nir Rozenbaum <[email protected]> * epp: return correct response for trailers (kubernetes-sigs#726) This looks like a copy paste error. * Refactor scheduler to run plugins (kubernetes-sigs#677) * Refactor scheduler to run plugins * Add scheduler plugin latency metric * Address comments * Address comments * Complete the InferencePool documentation (kubernetes-sigs#673) * Initial guide for inference pool * Add extensionReference to the InferencePool spec * Fix list formatting * Remove unused labels * Autogenerate the spec * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Update site-src/api-types/inferencepool.md Co-authored-by: Rob Scott <[email protected]> * Rename llm-pool names in rollout example * Add use cases for replacing an inference pool * Rewording the background section * Create replacing-inference-pool.md * Replace instructions with a link for how to replace an inference pool * Update replacing-inference-pool.md * Update mkdocs.yml * Update replacing-inference-pool.md * Update inferencemodel_types.go * Update inferencepool.md * Update site-src/guides/replacing-inference-pool.md Co-authored-by: Rob Scott <[email protected]> --------- Co-authored-by: Rob Scott <[email protected]> * reduce log level in metrics logger not to trash the log (kubernetes-sigs#708) * reduce log level in metrics logger not to trash the log Signed-off-by: Nir Rozenbaum <[email protected]> * rename flush metrics to refresh metrics Signed-off-by: Nir Rozenbaum <[email protected]> * revert log level Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * few updates in datastore (kubernetes-sigs#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <[email protected]> * PoolSet documentation Signed-off-by: Nir Rozenbaum <[email protected]> * error phrasing Signed-off-by: Nir Rozenbaum <[email protected]> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <[email protected]> * linter Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * scheduler refactoring (kubernetes-sigs#730) Signed-off-by: Nir Rozenbaum <[email protected]> * filter irrelevant pod in pod_reconciler (kubernetes-sigs#696) * EPP: Update GetRandomPod() to return nil if no pods exist (kubernetes-sigs#731) Signed-off-by: Daneyon Hansen <[email protected]> * Move filter and scorer plugins registration to a separate file (kubernetes-sigs#729) * Move filters and scorers registration to filter/scorer specific files * Default scheduler config contains empty list of scorers Signed-off-by: Maya Barnea <[email protected]> * Default plugin is not a scorer any more Signed-off-by: Maya Barnea <[email protected]> * fix scheduler test + lint comments Signed-off-by: Maya Barnea <[email protected]> --------- Signed-off-by: Maya Barnea <[email protected]> * Update issue templates (kubernetes-sigs#738) * Update issue templates * Updates artifacts for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <[email protected]> * Updates bbr chart for v0.3.0-rc.1 release Signed-off-by: Kellen Swain <[email protected]> * Updates artifacts for v0.3.0 release Signed-off-by: Kellen Swain <[email protected]> * Adding blank issue template so that all issues start with label --------- Signed-off-by: Kellen Swain <[email protected]> * Add unit test coverage for pod APIs under datastore/pkg * few updates in datastore (kubernetes-sigs#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <[email protected]> * PoolSet documentation Signed-off-by: Nir Rozenbaum <[email protected]> * error phrasing Signed-off-by: Nir Rozenbaum <[email protected]> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <[email protected]> * linter Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * few updates in datastore (kubernetes-sigs#713) * few updates in datastore Signed-off-by: Nir Rozenbaum <[email protected]> * PoolSet documentation Signed-off-by: Nir Rozenbaum <[email protected]> * error phrasing Signed-off-by: Nir Rozenbaum <[email protected]> * removed unused pool arg from PodUpdateOrAddIfNotExist Signed-off-by: Nir Rozenbaum <[email protected]> * linter Signed-off-by: Nir Rozenbaum <[email protected]> --------- Signed-off-by: Nir Rozenbaum <[email protected]> * Add unit test coverage for pod APIs under datastore/pkg --------- Signed-off-by: Nir Rozenbaum <[email protected]> Signed-off-by: Daneyon Hansen <[email protected]> Signed-off-by: Maya Barnea <[email protected]> Signed-off-by: Kellen Swain <[email protected]> Co-authored-by: Kellen Swain <[email protected]> Co-authored-by: Nir Rozenbaum <[email protected]> Co-authored-by: John Howard <[email protected]> Co-authored-by: Cong Liu <[email protected]> Co-authored-by: Nicole Xin <[email protected]> Co-authored-by: Rob Scott <[email protected]> Co-authored-by: nayihz <[email protected]> Co-authored-by: Daneyon Hansen <[email protected]> Co-authored-by: Maya Barnea <[email protected]>