Skip to content

Commit a85f32f

Browse files
jimmidysonjimmidyson
authored
feat: Support node taints per nodepool and control plane (#909)
**What problem does this PR solve?**: **Which issue(s) this PR fixes**: Fixes https://jira.nutanix.com/browse/NCN-102652 **How Has This Been Tested?**: <!-- Please describe the tests that you ran to verify your changes. Provide output from the tests and any manual steps needed to replicate the tests. --> **Special notes for your reviewer**: <!-- Use this to provide any additional information to the reviewers. This may include: - Best way to review the PR. - Where the author wants the most review attention on. - etc. -->
1 parent 3106b2f commit a85f32f

22 files changed

+945
-29
lines changed

‎api/v1alpha1/crds/caren.nutanix.com_awsclusterconfigs.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,34 @@ spec:
343343
default: m5.xlarge
344344
type: string
345345
type: object
346+
taints:
347+
description: Taints specifies the taints the Node API object should be registered with.
348+
items:
349+
description: |-
350+
The node this Taint is attached to has the "effect" on
351+
any pod that does not tolerate the Taint.
352+
properties:
353+
effect:
354+
default: NoSchedule
355+
description: |-
356+
The effect of the taint on pods that do not tolerate the taint.
357+
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
358+
enum:
359+
- NoSchedule
360+
- PreferNoSchedule
361+
- NoExecute
362+
type: string
363+
key:
364+
description: The taint key to be applied to a node.
365+
type: string
366+
value:
367+
description: The taint value corresponding to the taint key.
368+
type: string
369+
required:
370+
- effect
371+
- key
372+
type: object
373+
type: array
346374
type: object
347375
encryptionAtRest:
348376
description: |-

‎api/v1alpha1/crds/caren.nutanix.com_awsworkernodeconfigs.yaml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,35 @@ spec:
9090
description: The AWS instance type to use for the cluster Machines.
9191
type: string
9292
type: object
93+
taints:
94+
description: Taints specifies the taints the Node API object should
95+
be registered with.
96+
items:
97+
description: |-
98+
The node this Taint is attached to has the "effect" on
99+
any pod that does not tolerate the Taint.
100+
properties:
101+
effect:
102+
default: NoSchedule
103+
description: |-
104+
The effect of the taint on pods that do not tolerate the taint.
105+
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
106+
enum:
107+
- NoSchedule
108+
- PreferNoSchedule
109+
- NoExecute
110+
type: string
111+
key:
112+
description: The taint key to be applied to a node.
113+
type: string
114+
value:
115+
description: The taint value corresponding to the taint key.
116+
type: string
117+
required:
118+
- effect
119+
- key
120+
type: object
121+
type: array
93122
type: object
94123
type: object
95124
served: true

‎api/v1alpha1/crds/caren.nutanix.com_dockerclusterconfigs.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,34 @@ spec:
260260
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
261261
type: string
262262
type: object
263+
taints:
264+
description: Taints specifies the taints the Node API object should be registered with.
265+
items:
266+
description: |-
267+
The node this Taint is attached to has the "effect" on
268+
any pod that does not tolerate the Taint.
269+
properties:
270+
effect:
271+
default: NoSchedule
272+
description: |-
273+
The effect of the taint on pods that do not tolerate the taint.
274+
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
275+
enum:
276+
- NoSchedule
277+
- PreferNoSchedule
278+
- NoExecute
279+
type: string
280+
key:
281+
description: The taint key to be applied to a node.
282+
type: string
283+
value:
284+
description: The taint value corresponding to the taint key.
285+
type: string
286+
required:
287+
- effect
288+
- key
289+
type: object
290+
type: array
263291
type: object
264292
docker:
265293
type: object

‎api/v1alpha1/crds/caren.nutanix.com_dockernodeconfigs.yaml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,35 @@ spec:
4848
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
4949
type: string
5050
type: object
51+
taints:
52+
description: Taints specifies the taints the Node API object should
53+
be registered with.
54+
items:
55+
description: |-
56+
The node this Taint is attached to has the "effect" on
57+
any pod that does not tolerate the Taint.
58+
properties:
59+
effect:
60+
default: NoSchedule
61+
description: |-
62+
The effect of the taint on pods that do not tolerate the taint.
63+
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
64+
enum:
65+
- NoSchedule
66+
- PreferNoSchedule
67+
- NoExecute
68+
type: string
69+
key:
70+
description: The taint key to be applied to a node.
71+
type: string
72+
value:
73+
description: The taint value corresponding to the taint key.
74+
type: string
75+
required:
76+
- effect
77+
- key
78+
type: object
79+
type: array
5180
type: object
5281
type: object
5382
served: true

‎api/v1alpha1/crds/caren.nutanix.com_nutanixclusterconfigs.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -413,6 +413,34 @@ spec:
413413
required:
414414
- machineDetails
415415
type: object
416+
taints:
417+
description: Taints specifies the taints the Node API object should be registered with.
418+
items:
419+
description: |-
420+
The node this Taint is attached to has the "effect" on
421+
any pod that does not tolerate the Taint.
422+
properties:
423+
effect:
424+
default: NoSchedule
425+
description: |-
426+
The effect of the taint on pods that do not tolerate the taint.
427+
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
428+
enum:
429+
- NoSchedule
430+
- PreferNoSchedule
431+
- NoExecute
432+
type: string
433+
key:
434+
description: The taint key to be applied to a node.
435+
type: string
436+
value:
437+
description: The taint value corresponding to the taint key.
438+
type: string
439+
required:
440+
- effect
441+
- key
442+
type: object
443+
type: array
416444
type: object
417445
encryptionAtRest:
418446
description: |-

‎api/v1alpha1/crds/caren.nutanix.com_nutanixnodeconfigs.yaml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,34 @@ spec:
201201
required:
202202
- machineDetails
203203
type: object
204+
taints:
205+
description: Taints specifies the taints the Node API object should be registered with.
206+
items:
207+
description: |-
208+
The node this Taint is attached to has the "effect" on
209+
any pod that does not tolerate the Taint.
210+
properties:
211+
effect:
212+
default: NoSchedule
213+
description: |-
214+
The effect of the taint on pods that do not tolerate the taint.
215+
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
216+
enum:
217+
- NoSchedule
218+
- PreferNoSchedule
219+
- NoExecute
220+
type: string
221+
key:
222+
description: The taint key to be applied to a node.
223+
type: string
224+
value:
225+
description: The taint value corresponding to the taint key.
226+
type: string
227+
required:
228+
- effect
229+
- key
230+
type: object
231+
type: array
204232
type: object
205233
type: object
206234
served: true

‎api/v1alpha1/nodeconfig_types.go

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ func (s AWSWorkerNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolin
5050
type AWSWorkerNodeConfigSpec struct {
5151
// +kubebuilder:validation:Optional
5252
AWS *AWSWorkerNodeSpec `json:"aws,omitempty"`
53+
54+
GenericNodeSpec `json:",inline"`
5355
}
5456

5557
// AWSControlPlaneConfigSpec defines the desired state of AWSNodeConfig.
@@ -58,6 +60,8 @@ type AWSWorkerNodeConfigSpec struct {
5860
type AWSControlPlaneNodeConfigSpec struct {
5961
// +kubebuilder:validation:Optional
6062
AWS *AWSControlPlaneNodeSpec `json:"aws,omitempty"`
63+
64+
GenericNodeSpec `json:",inline"`
6165
}
6266

6367
// +kubebuilder:object:root=true
@@ -79,6 +83,8 @@ func (s DockerNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolint:g
7983
type DockerNodeConfigSpec struct {
8084
// +kubebuilder:validation:Optional
8185
Docker *DockerNodeSpec `json:"docker,omitempty"`
86+
87+
GenericNodeSpec `json:",inline"`
8288
}
8389

8490
// +kubebuilder:object:root=true
@@ -100,8 +106,54 @@ func (s NutanixNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolint:
100106
type NutanixNodeConfigSpec struct {
101107
// +kubebuilder:validation:Optional
102108
Nutanix *NutanixNodeSpec `json:"nutanix,omitempty"`
109+
110+
GenericNodeSpec `json:",inline"`
111+
}
112+
113+
type GenericNodeSpec struct {
114+
// Taints specifies the taints the Node API object should be registered with.
115+
// +kubebuilder:validation:Optional
116+
Taints []Taint `json:"taints,omitempty"`
117+
}
118+
119+
// The node this Taint is attached to has the "effect" on
120+
// any pod that does not tolerate the Taint.
121+
type Taint struct {
122+
// The taint key to be applied to a node.
123+
// +kubebuilder:validation:Required
124+
Key string `json:"key"`
125+
126+
// The taint value corresponding to the taint key.
127+
// +kubebuilder:validation:Optional
128+
Value string `json:"value,omitempty"`
129+
130+
// The effect of the taint on pods that do not tolerate the taint.
131+
// Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
132+
// +kubebuilder:validation:Required
133+
// +kubebuilder:default=NoSchedule
134+
// +kubebuilder:validation:Enum:=NoSchedule;PreferNoSchedule;NoExecute
135+
Effect TaintEffect `json:"effect"`
103136
}
104137

138+
type TaintEffect string
139+
140+
const (
141+
// Do not allow new pods to schedule onto the node unless they tolerate the taint,
142+
// but allow all pods submitted to Kubelet without going through the scheduler
143+
// to start, and allow all already-running pods to continue running.
144+
// Enforced by the scheduler.
145+
TaintEffectNoSchedule TaintEffect = "NoSchedule"
146+
147+
// Like TaintEffectNoSchedule, but the scheduler tries not to schedule
148+
// new pods onto the node, rather than prohibiting new pods from scheduling
149+
// onto the node entirely. Enforced by the scheduler.
150+
TaintEffectPreferNoSchedule TaintEffect = "PreferNoSchedule"
151+
152+
// Evict any already-running pods that do not tolerate the taint.
153+
// Currently enforced by NodeController.
154+
TaintEffectNoExecute TaintEffect = "NoExecute"
155+
)
156+
105157
//nolint:gochecknoinits // Idiomatic to use init functions to register APIs with scheme.
106158
func init() {
107159
SchemeBuilder.Register(&AWSWorkerNodeConfig{}, &DockerNodeConfig{}, &NutanixNodeConfig{})

‎api/v1alpha1/zz_generated.deepcopy.go

Lines changed: 39 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/content/addons/serviceloadbalancer.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
+++
22
title = "Service LoadBalancer"
3+
icon = "fa-solid fa-arrows-turn-to-dots"
34
+++
45

56
When an application running in a cluster needs to be exposed outside of the cluster, one option is

0 commit comments

Comments
 (0)