Skip to content

feat: Support node taints per nodepool and control plane #909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 25, 2024
Merged

feat: Support node taints per nodepool and control plane #909

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
af7637a
feat: Support node taints per nodepool and control plane
jimmidyson Sep 17, 2024
3dd874d
fixup! docs: Add default taints doc
jimmidyson Sep 24, 2024
9af2aac
fixup! feat: Support removing all taints
jimmidyson Sep 24, 2024
78740a2
fixup! test: Add test for taints utility
jimmidyson Sep 24, 2024
67bae1e
fixup! docs: Update for apply config
jimmidyson Sep 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_awsclusterconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,34 @@ spec:
default: m5.xlarge
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
encryptionAtRest:
description: |-
Expand Down
29 changes: 29 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_awsworkernodeconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,35 @@ spec:
description: The AWS instance type to use for the cluster Machines.
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should
be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
type: object
served: true
Expand Down
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_dockerclusterconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,34 @@ spec:
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
docker:
type: object
Expand Down
29 changes: 29 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_dockernodeconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,35 @@ spec:
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should
be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
type: object
served: true
Expand Down
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_nutanixclusterconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,34 @@ spec:
required:
- machineDetails
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
encryptionAtRest:
description: |-
Expand Down
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_nutanixnodeconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,34 @@ spec:
required:
- machineDetails
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
type: object
served: true
Expand Down
52 changes: 52 additions & 0 deletions api/v1alpha1/nodeconfig_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ func (s AWSWorkerNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolin
type AWSWorkerNodeConfigSpec struct {
// +kubebuilder:validation:Optional
AWS *AWSWorkerNodeSpec `json:"aws,omitempty"`

GenericNodeSpec `json:",inline"`
}

// AWSControlPlaneConfigSpec defines the desired state of AWSNodeConfig.
Expand All @@ -58,6 +60,8 @@ type AWSWorkerNodeConfigSpec struct {
type AWSControlPlaneNodeConfigSpec struct {
// +kubebuilder:validation:Optional
AWS *AWSControlPlaneNodeSpec `json:"aws,omitempty"`

GenericNodeSpec `json:",inline"`
}

// +kubebuilder:object:root=true
Expand All @@ -79,6 +83,8 @@ func (s DockerNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolint:g
type DockerNodeConfigSpec struct {
// +kubebuilder:validation:Optional
Docker *DockerNodeSpec `json:"docker,omitempty"`

GenericNodeSpec `json:",inline"`
}

// +kubebuilder:object:root=true
Expand All @@ -100,8 +106,54 @@ func (s NutanixNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolint:
type NutanixNodeConfigSpec struct {
// +kubebuilder:validation:Optional
Nutanix *NutanixNodeSpec `json:"nutanix,omitempty"`

GenericNodeSpec `json:",inline"`
}

type GenericNodeSpec struct {
// Taints specifies the taints the Node API object should be registered with.
// +kubebuilder:validation:Optional
Taints []Taint `json:"taints,omitempty"`
}

// The node this Taint is attached to has the "effect" on
// any pod that does not tolerate the Taint.
type Taint struct {
// The taint key to be applied to a node.
// +kubebuilder:validation:Required
Key string `json:"key"`

// The taint value corresponding to the taint key.
// +kubebuilder:validation:Optional
Value string `json:"value,omitempty"`

// The effect of the taint on pods that do not tolerate the taint.
// Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
// +kubebuilder:validation:Required
// +kubebuilder:default=NoSchedule
// +kubebuilder:validation:Enum:=NoSchedule;PreferNoSchedule;NoExecute
Effect TaintEffect `json:"effect"`
}

type TaintEffect string

const (
// Do not allow new pods to schedule onto the node unless they tolerate the taint,
// but allow all pods submitted to Kubelet without going through the scheduler
// to start, and allow all already-running pods to continue running.
// Enforced by the scheduler.
TaintEffectNoSchedule TaintEffect = "NoSchedule"

// Like TaintEffectNoSchedule, but the scheduler tries not to schedule
// new pods onto the node, rather than prohibiting new pods from scheduling
// onto the node entirely. Enforced by the scheduler.
TaintEffectPreferNoSchedule TaintEffect = "PreferNoSchedule"

// Evict any already-running pods that do not tolerate the taint.
// Currently enforced by NodeController.
TaintEffectNoExecute TaintEffect = "NoExecute"
)

//nolint:gochecknoinits // Idiomatic to use init functions to register APIs with scheme.
func init() {
SchemeBuilder.Register(&AWSWorkerNodeConfig{}, &DockerNodeConfig{}, &NutanixNodeConfig{})
Expand Down
43 changes: 42 additions & 1 deletion api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading