Skip to content

Commit a209459

Browse files
dkoshkindkoshkin
authored
feat: COSI controller Addon (#1008)
**What problem does this PR solve?**: This PR adds the COSI controller Addon, without any providers. The chart comes from https://github.com/mesosphere/charts/tree/master/stable/cosi The addon does not take in any configuration so the API to enable it is just: ```yaml apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: name: <NAME> spec: topology: variables: - name: clusterConfig value: addons: cosi: {} ``` Once we add providers, the API can be similar to what it is with CSI with a `providers:` field. **Which issue(s) this PR fixes**: Fixes # **How Has This Been Tested?**: <!-- Please describe the tests that you ran to verify your changes. Provide output from the tests and any manual steps needed to replicate the tests. --> New unit tests. Also deployed it on a Nutanix Cluster and the Nutanix COSI driver from nutanix-cloud-native/cosi-driver-nutanix#375 Create the credentials Secret ``` apiVersion: v1 kind: Secret metadata: annotations: labels: name: objectstorage-provisioner namespace: cosi-driver-nutanix stringData: ACCESS_KEY: "<>" ACCOUNT_NAME: "<>" ENDPOINT: "http://10.1.1.1:80" PC_SECRET: "<>:9440:<>:<>" SECRET_KEY: "<>" type: Opaque ``` ``` NUTANIX_COSI_DIR=<> helm install cosi-driver -n cosi-driver-nutanix --create-namespace --set=secret.enabled=false --set=cosiController.enabled=false --skip-crds$NUTANIX_COSI_DIR/charts/ kubectl apply -f $NUTANIX_COSI_DIR/project/examples/bucketclass.yaml kubectl apply -f $NUTANIX_COSI_DIR/project/examples/bucketclaim.yaml kubectl apply -f $NUTANIX_COSI_DIR/project/examples/bucketaccessclass.yaml kubectl apply -f $NUTANIX_COSI_DIR/project/examples/bucketaccess.yaml # From nutanix-cloud-native/cosi-driver-nutanix#374 kubectl apply -f https://raw.githubusercontent.com/nutanix-cloud-native/cosi-driver-nutanix/944ce9ba66bc34e59bd94519a88da8bf9fae3d0c/project/examples/awscliapppod.yaml ``` The Secret was correctly generated and the test Pod wrote and read to a new bucket. ``` $ kubectl logs awscli Defaulted container "awscli" out of: awscli, write-aws-credentials (init), write-test-file (init) + aws s3 ls 2025-01-07 20:43:31 sample-bucketclass8ce19295-f704-45bb-a533-15591c55a747 ++ cat /tmp/test-directory/file.txt + readonly BUCKET_NAME=sample-bucketclass8ce19295-f704-45bb-a533-15591c55a747 + BUCKET_NAME=sample-bucketclass8ce19295-f704-45bb-a533-15591c55a747 ++ date +%Y%m%d_%H%M%S + readonly FILE_NAME=20250107_205233.txt + FILE_NAME=20250107_205233.txt + aws s3 cp /tmp/test-directory/file.txt s3://sample-bucketclass8ce19295-f704-45bb-a533-15591c55a747/20250107_205233.txt upload: ../tmp/test-directory/file.txt to s3://sample-bucketclass8ce19295-f704-45bb-a533-15591c55a747/20250107_205233.txt + aws s3 cp s3://sample-bucketclass8ce19295-f704-45bb-a533-15591c55a747/20250107_205233.txt - sample-bucketclass8ce19295-f704-45bb-a533-15591c55a747 ``` **Special notes for your reviewer**: <!-- Use this to provide any additional information to the reviewers. This may include: - Best way to review the PR. - Where the author wants the most review attention on. - etc. -->
1 parent d5d6cfd commit a209459

35 files changed

+531
-3
lines changed

‎api/v1alpha1/addon_types.go

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,13 +70,19 @@ type DockerAddons struct {
7070

7171
// +kubebuilder:validation:Optional
7272
CSI *DockerCSI `json:"csi,omitempty"`
73+
74+
// +kubebuilder:validation:Optional
75+
COSI *DockerCOSI `json:"cosi,omitempty"`
7376
}
7477

7578
type NutanixAddons struct {
7679
GenericAddons `json:",inline"`
7780

7881
// +kubebuilder:validation:Optional
7982
CSI *NutanixCSI `json:"csi,omitempty"`
83+
84+
// +kubebuilder:validation:Optional
85+
COSI *NutanixCOSI `json:"cosi,omitempty"`
8086
}
8187

8288
type GenericAddons struct {
@@ -96,8 +102,6 @@ type GenericAddons struct {
96102
ServiceLoadBalancer *ServiceLoadBalancer `json:"serviceLoadBalancer,omitempty"`
97103
}
98104

99-
// +kubebuilder:validation:Optional
100-
// +kubebuilder:validation:Enum=ClusterResourceSet;HelmAddon
101105
type AddonStrategy string
102106

103107
// CNI required for providing CNI configuration.
@@ -109,13 +113,15 @@ type CNI struct {
109113

110114
// Addon strategy used to deploy the CNI provider to the workload cluster.
111115
// +kubebuilder:default=HelmAddon
116+
// +kubebuilder:validation:Enum=ClusterResourceSet;HelmAddon
112117
Strategy *AddonStrategy `json:"strategy,omitempty"`
113118
}
114119

115120
// NFD tells us to enable or disable the node feature discovery addon.
116121
type NFD struct {
117122
// Addon strategy used to deploy Node Feature Discovery (NFD) to the workload cluster.
118123
// +kubebuilder:default=HelmAddon
124+
// +kubebuilder:validation:Enum=ClusterResourceSet;HelmAddon
119125
Strategy *AddonStrategy `json:"strategy,omitempty"`
120126
}
121127

@@ -124,6 +130,7 @@ type ClusterAutoscaler struct {
124130
// Addon strategy used to deploy cluster-autoscaler to the management cluster
125131
// targeting the workload cluster.
126132
// +kubebuilder:default=HelmAddon
133+
// +kubebuilder:validation:Enum=ClusterResourceSet;HelmAddon
127134
Strategy *AddonStrategy `json:"strategy,omitempty"`
128135
}
129136

@@ -136,9 +143,17 @@ type GenericCSI struct {
136143
SnapshotController *SnapshotController `json:"snapshotController,omitempty"`
137144
}
138145

146+
type GenericCOSI struct {
147+
// Addon strategy used to deploy the COSI controller to the workload cluster.
148+
// +kubebuilder:default=HelmAddon
149+
// +kubebuilder:validation:Enum=HelmAddon
150+
Strategy *AddonStrategy `json:"strategy,omitempty"`
151+
}
152+
139153
type SnapshotController struct {
140154
// Addon strategy used to deploy the snapshot controller to the workload cluster.
141155
// +kubebuilder:default=HelmAddon
156+
// +kubebuilder:validation:Enum=ClusterResourceSet;HelmAddon
142157
Strategy *AddonStrategy `json:"strategy,omitempty"`
143158
}
144159

@@ -197,6 +212,7 @@ type CSIProvider struct {
197212

198213
// Addon strategy used to deploy the CSI provider to the workload cluster.
199214
// +kubebuilder:default=HelmAddon
215+
// +kubebuilder:validation:Enum=ClusterResourceSet;HelmAddon
200216
Strategy *AddonStrategy `json:"strategy,omitempty"`
201217

202218
// The reference to any secret used by the CSI Provider.
@@ -231,6 +247,14 @@ type CSICredentials struct {
231247
SecretRef LocalObjectReference `json:"secretRef"`
232248
}
233249

250+
type DockerCOSI struct {
251+
GenericCOSI `json:",inline"`
252+
}
253+
254+
type NutanixCOSI struct {
255+
GenericCOSI `json:",inline"`
256+
}
257+
234258
// CCM tells us to enable or disable the cloud provider interface.
235259
type CCM struct {
236260
// A reference to the Secret for credential information for the target Prism Central instance
@@ -239,6 +263,7 @@ type CCM struct {
239263

240264
// Addon strategy used to deploy the CCM to the workload cluster.
241265
// +kubebuilder:default=HelmAddon
266+
// +kubebuilder:validation:Enum=ClusterResourceSet;HelmAddon
242267
Strategy *AddonStrategy `json:"strategy,omitempty"`
243268
}
244269

‎api/v1alpha1/constants.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ const (
2222
CNIVariableName = "cni"
2323
// NFDVariableName is the NFD external patch variable name.
2424
NFDVariableName = "nfd"
25-
25+
// COSIVariableName is the COSI external patch variable name.
26+
COSIVariableName = "cosi"
2627
// ClusterAutoscalerVariableName is the cluster-autoscaler external patch variable name.
2728
ClusterAutoscalerVariableName = "clusterAutoscaler"
2829
// ServiceLoadBalancerVariableName is the Service LoadBalancer config patch variable name.

‎api/v1alpha1/crds/caren.nutanix.com_dockerclusterconfigs.yaml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,15 @@ spec:
104104
required:
105105
- provider
106106
type: object
107+
cosi:
108+
properties:
109+
strategy:
110+
default: HelmAddon
111+
description: Addon strategy used to deploy the COSI controller to the workload cluster.
112+
enum:
113+
- HelmAddon
114+
type: string
115+
type: object
107116
csi:
108117
properties:
109118
defaultStorage:

‎api/v1alpha1/crds/caren.nutanix.com_nutanixclusterconfigs.yaml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,15 @@ spec:
104104
required:
105105
- provider
106106
type: object
107+
cosi:
108+
properties:
109+
strategy:
110+
default: HelmAddon
111+
description: Addon strategy used to deploy the COSI controller to the workload cluster.
112+
enum:
113+
- HelmAddon
114+
type: string
115+
type: object
107116
csi:
108117
properties:
109118
defaultStorage:

‎api/v1alpha1/zz_generated.deepcopy.go

Lines changed: 62 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎api/variables/aggregate_types.go

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,16 @@ type Addons struct {
5353
carenv1.GenericAddons `json:",inline"`
5454

5555
CSI *CSI `json:"csi,omitempty"`
56+
57+
COSI *COSI `json:"cosi,omitempty"`
5658
}
5759

5860
type CSI struct {
5961
carenv1.GenericCSI `json:",inline"`
6062

6163
Providers map[string]carenv1.CSIProvider `json:"providers"`
6264
}
65+
66+
type COSI struct {
67+
carenv1.GenericCOSI `json:",inline"`
68+
}

‎charts/cluster-api-runtime-extensions-nutanix/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,8 @@ A Helm chart for cluster-api-runtime-extensions-nutanix
7474
| hooks.cni.cilium.crsStrategy.defaultCiliumConfigMap.name | string | `"cilium"` | |
7575
| hooks.cni.cilium.helmAddonStrategy.defaultValueTemplateConfigMap.create | bool | `true` | |
7676
| hooks.cni.cilium.helmAddonStrategy.defaultValueTemplateConfigMap.name | string | `"default-cilium-cni-helm-values-template"` | |
77+
| hooks.cosi.controller.helmAddonStrategy.defaultValueTemplateConfigMap.create | bool | `true` | |
78+
| hooks.cosi.controller.helmAddonStrategy.defaultValueTemplateConfigMap.name | string | `"default-cosi-controller-helm-values-template"` | |
7779
| hooks.csi.aws-ebs.helmAddonStrategy.defaultValueTemplateConfigMap.create | bool | `true` | |
7880
| hooks.csi.aws-ebs.helmAddonStrategy.defaultValueTemplateConfigMap.name | string | `"default-aws-ebs-csi-helm-values-template"` | |
7981
| hooks.csi.local-path.helmAddonStrategy.defaultValueTemplateConfigMap.create | bool | `true` | |

‎charts/cluster-api-runtime-extensions-nutanix/addons/cosi/controller/values-template.yaml

Whitespace-only changes.

‎charts/cluster-api-runtime-extensions-nutanix/templates/cosi/controller/manifests/helm-addon-installation.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# Copyright 2025 Nutanix. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
{{- if (index .Values.hooks.cosi "controller").helmAddonStrategy.defaultValueTemplateConfigMap.create }}
5+
apiVersion: v1
6+
kind: ConfigMap
7+
metadata:
8+
name: '{{ (index .Values.hooks.cosi "controller").helmAddonStrategy.defaultValueTemplateConfigMap.name }}'
9+
data:
10+
values.yaml: |-
11+
{{- .Files.Get "addons/cosi/controller/values-template.yaml" | nindent 4 }}
12+
{{- end -}}

‎charts/cluster-api-runtime-extensions-nutanix/templates/deployment.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ spec:
4141
- --csi.local-path.helm-addon.default-values-template-configmap-name={{ (index .Values.hooks.csi "local-path").helmAddonStrategy.defaultValueTemplateConfigMap.name }}
4242
- --csi.snapshot-controller.helm-addon.default-values-template-configmap-name={{ (index .Values.hooks.csi "snapshot-controller").helmAddonStrategy.defaultValueTemplateConfigMap.name }}
4343
- --ccm.aws.helm-addon.default-values-template-configmap-name={{ .Values.hooks.ccm.aws.helmAddonStrategy.defaultValueTemplateConfigMap.name }}
44+
- --cosi.controller.helm-addon.default-values-template-configmap-name={{ .Values.hooks.cosi.controller.helmAddonStrategy.defaultValueTemplateConfigMap.name }}
4445
{{- range $k, $v := .Values.hooks.ccm.aws.k8sMinorVersionToCCMVersion }}
4546
- --ccm.aws.aws-ccm-versions={{ $k }}={{ $v }}
4647
{{- end }}

‎charts/cluster-api-runtime-extensions-nutanix/templates/helm-config.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ data:
2323
ChartName: cluster-autoscaler
2424
ChartVersion: 9.43.2
2525
RepositoryURL: '{{ if .Values.helmRepository.enabled }}oci://helm-repository.{{ .Release.Namespace }}.svc/charts{{ else }}https://kubernetes.github.io/autoscaler{{ end }}'
26+
cosi-controller: |
27+
ChartName: cosi
28+
ChartVersion: 0.0.1-alpha.1
29+
RepositoryURL: '{{ if .Values.helmRepository.enabled }}oci://helm-repository.{{ .Release.Namespace }}.svc/charts{{ else }}https://mesosphere.github.io/charts/stable/{{ end }}'
2630
local-path-provisioner-csi: |
2731
ChartName: local-path-provisioner
2832
ChartVersion: 0.0.30

‎charts/cluster-api-runtime-extensions-nutanix/values.schema.json

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -361,6 +361,32 @@
361361
},
362362
"type": "object"
363363
},
364+
"cosi": {
365+
"properties": {
366+
"controller": {
367+
"properties": {
368+
"helmAddonStrategy": {
369+
"properties": {
370+
"defaultValueTemplateConfigMap": {
371+
"properties": {
372+
"create": {
373+
"type": "boolean"
374+
},
375+
"name": {
376+
"type": "string"
377+
}
378+
},
379+
"type": "object"
380+
}
381+
},
382+
"type": "object"
383+
}
384+
},
385+
"type": "object"
386+
}
387+
},
388+
"type": "object"
389+
},
364390
"csi": {
365391
"properties": {
366392
"aws-ebs": {

‎charts/cluster-api-runtime-extensions-nutanix/values.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,12 @@ hooks:
107107
defaultTemplateConfigMap:
108108
create: true
109109
name: default-kube-vip-template
110+
cosi:
111+
controller:
112+
helmAddonStrategy:
113+
defaultValueTemplateConfigMap:
114+
create: true
115+
name: default-cosi-controller-helm-values-template
110116

111117
helmAddonsConfigMap: default-helm-addons-config
112118

‎docs/content/addons/cosi.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
+++
2+
title = " Container Object Storage Interface (COSI)"
3+
icon = "fa-solid fa-eye"
4+
+++
5+
6+
By leveraging CAPI cluster lifecycle hooks, this handler deploys [Container Object Storage Interface] (COSI)
7+
on the new cluster at the `AfterControlPlaneInitialized` phase.
8+
9+
Deployment of COSI is opt-in via the [provider-specific cluster configuration]({{< ref ".." >}}).
10+
11+
The hook uses the [Cluster API Add-on Provider for Helm] to deploy the COSI resources.
12+
13+
## Example
14+
15+
To enable deployment of COSI on a cluster, specify the following values:
16+
17+
```yaml
18+
apiVersion: cluster.x-k8s.io/v1beta1
19+
kind: Cluster
20+
metadata:
21+
name: <NAME>
22+
spec:
23+
topology:
24+
variables:
25+
- name: clusterConfig
26+
value:
27+
addons:
28+
cosi: {}
29+
```
30+
31+
[Container Object Storage Interface]: https://kubernetes.io/blog/2022/09/02/cosi-kubernetes-object-storage-management/
32+
[Cluster API Add-on Provider for Helm]: https://github.com/kubernetes-sigs/cluster-api-addon-provider-helm

‎examples/capi-quick-start/docker-cluster-calico-crs.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ spec:
2727
cni:
2828
provider: Calico
2929
strategy: ClusterResourceSet
30+
cosi: {}
3031
csi:
3132
defaultStorage:
3233
provider: local-path

‎examples/capi-quick-start/docker-cluster-calico-helm-addon.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ spec:
2525
clusterAutoscaler: {}
2626
cni:
2727
provider: Calico
28+
cosi: {}
2829
csi:
2930
defaultStorage:
3031
provider: local-path

0 commit comments

Comments
 (0)