Skip to content

Add handler to run execution in separate goroutine #120902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 16, 2024
Merged

Add handler to run execution in separate goroutine #120902

merged 1 commit into from
Feb 16, 2024

Conversation

linxiulei
Copy link
Contributor

@linxiulei linxiulei commented Sep 26, 2023 •
edited
Loading

This handler allows running execution prior to actual serving in a separate goroutine when serving requests. Doing so benefits cases in serving long running requests because it allows freeing memory used by the separate goroutine and keeps the serving routines slim.

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #120901

Special notes for your reviewer:

Observed 150 MiB memory reduced in stack memory in the test that had 5000 active watch requests. The heap memory stayed roughly the same.

Does this PR introduce a user-facing change?

An optimization is implemented to reduce stack memory usage for watch requests.  It is can be disabled with the feature gate: APIServingWithRoutine=false

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Sep 26, 2023
@k8s-ci-robot k8s-ci-robot added area/apiserver sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Sep 26, 2023
@aojea
Copy link
Member

aojea commented Sep 26, 2023

/cc

@k8s-ci-robot k8s-ci-robot requested a review from aojea September 26, 2023 19:34
@alexzielenski
Copy link
Member

Looks promising! Thank you!
/triage accepted

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Sep 26, 2023
@linxiulei
Copy link
Contributor Author

/retest

@linxiulei linxiulei force-pushed the watch_stack branch 2 times, most recently from 57983bc to 0d6cfcc Compare September 27, 2023 12:37
@linxiulei
Copy link
Contributor Author

/cc @wojtek-t @liggitt

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 27, 2023
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 28, 2023
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Dec 22, 2023
@linxiulei
Copy link
Contributor Author

Added unit tests, feature gate and comments. Updated release notes to mention feature gate.

@linxiulei
Copy link
Contributor Author

/retest

@linxiulei
Copy link
Contributor Author

Gentle ping @wojtek-t

wojtek-t
wojtek-t reviewed Feb 13, 2024
View reviewed changes
Copy link
Member

@wojtek-t wojtek-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to look deeper into tests, but I think it's going the right direction.

@linxiulei linxiulei force-pushed the watch_stack branch 3 times, most recently from 877ae3a to 3004566 Compare February 13, 2024 14:18
@linxiulei linxiulei force-pushed the watch_stack branch 2 times, most recently from 908b627 to ce80932 Compare February 13, 2024 16:58
wojtek-t
wojtek-t reviewed Feb 16, 2024
View reviewed changes
Copy link
Member

@wojtek-t wojtek-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last comment - other than that LGTM.

staging/src/k8s.io/apiserver/pkg/server/config.go
@@ -1038,6 +1038,11 @@ func DefaultBuildHandlerChain(apiHandler http.Handler, c *Config) http.Handler {
handler = genericapifilters.WithTracing(handler, c.TracerProvider)
}
handler = genericapifilters.WithLatencyTrackers(handler)
// WithRoutine will execute future handlers in a separate goroutine and serving
// handler in current goroutine to minimize the stack memory usage.
if utilfeature.DefaultFeatureGate.Enabled(genericfeatures.APIServingWithRoutine) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add a comment that this has to be put after WithPanicRecovery (as it is now), but to ensure that someone will not change it later.

Also please squash the commits.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comment and squashed.

@linxiulei
Add handler to run watch serving in separate goroutine
7b2698a 
This handler allows running execution prior to actual serving in a separate
goroutine when serving requests. Doing so benefits cases in serving long running
requests because it allows freeing memory used by the separate goroutine
and keeps the serving routines slim.

Signed-off-by: Eric Lin <[email protected]>
@wojtek-t
Copy link
Member

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 16, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 17dba7f46e6a1facbf1cf08c28dbafadcad68362

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: linxiulei, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 16, 2024
@k8s-ci-robot k8s-ci-robot merged commit 6c04679 into kubernetes:master Feb 16, 2024
@k8s-ci-robot k8s-ci-robot added this to the v1.30 milestone Feb 16, 2024
@mborsz mborsz mentioned this pull request Jun 21, 2024
Closed
@linxiulei linxiulei mentioned this pull request Jun 27, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@liggitt liggitt liggitt left review comments

@aojea aojea aojea left review comments

@wojtek-t wojtek-t wojtek-t left review comments

@hzxuzhonghu hzxuzhonghu Awaiting requested review from hzxuzhonghu

@sttts sttts Awaiting requested review from sttts

Assignees

@wojtek-t wojtek-t

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.30
Development

Successfully merging this pull request may close these issues.

Serving watch requests takes large memory space in goroutine's stack
6 participants
@linxiulei @aojea @alexzielenski @wojtek-t @k8s-ci-robot @liggitt