Skip to content

[WIP] proposal - add readypods field to InferencePoolStatus #714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[WIP] proposal - add readypods field to InferencePoolStatus #714

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions api/v1alpha2/inferencepool_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ import (
// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:storageversion
// +kubebuilder:printcolumn:name="Inference Pool",type=string,JSONPath=`.metadata.name`
// +kubebuilder:printcolumn:name="ReadyPods",type=date,JSONPath=`.status.readyPods`
// +genclient
type InferencePool struct {
metav1.TypeMeta `json:",inline"`
Expand Down Expand Up @@ -157,6 +159,10 @@ type InferencePoolStatus struct {
//
// +kubebuilder:validation:MaxItems=32
Parents []PoolStatus `json:"parent,omitempty"`
//ReadyPods is the number of pods that are associated with the InferencePool and are ready
// to serve inference requests. A pod is considered ready if its Ready condition is set to
// "True" and it is not marked for deletion.
ReadyPods int `json:"readyPods,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the goal for this field? Historically in Kubernetes we've tried to ensure a separation of concerns here. Ideally the portion of a Gateway implementation that handles endpoint readiness should be something that can be disconnected from higher level routing config.

More broadly, I'd be worried that trying to keep track of this in status would dramatically increase the number of updates to InferencePool status and the overall load on the controller. Instead of needing to reconcile when an InferencePool changed, it would need to reconcile anytime the readiness or number of connected endpoints changed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to ^. I prefer continuing to use the ResolvedRefs=False condition to indicate the following:

  1. spec.selector returns 0 Pods.
  2. ExtensionReference can be resolved.

@robscott @kfswain @ahg-g is my understanding of ResolvedRefs consistent with yours?

}

// PoolStatus defines the observed state of InferencePool from a Gateway.
Expand Down
11 changes: 10 additions & 1 deletion client-go/applyconfiguration/api/v1alpha2/inferencepoolstatus.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 14 additions & 1 deletion config/crd/bases/inference.networking.x-k8s.io_inferencepools.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,14 @@ spec:
singular: inferencepool
scope: Namespaced
versions:
- name: v1alpha2
- additionalPrinterColumns:
- jsonPath: .metadata.name
name: Inference Pool
type: string
- jsonPath: .status.readyPods
name: ReadyPods
type: date
name: v1alpha2
schema:
openAPIV3Schema:
description: InferencePool is the Schema for the InferencePools API.
Expand Down Expand Up @@ -271,6 +278,12 @@ spec:
type: object
maxItems: 32
type: array
readyPods:
description: |-
ReadyPods is the number of pods that are associated with the InferencePool and are ready
to serve inference requests. A pod is considered ready if its Ready condition is set to
"True" and it is not marked for deletion.
type: integer
type: object
type: object
served: true
Expand Down