Skip to content

Commit effd1ea

Browse files
jimmidysondkoshkin
jimmidyson
and
dkoshkin
authored
feat: Deploy calico CNI via CRS (#107)
Co-authored-by: Dimitri Koshkin <[email protected]>
1 parent 7297882 commit effd1ea

35 files changed

+911
-160
lines changed

‎.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ openapi_violations.report
1919
.idea/
2020

2121
#ignore report files generated by unittest
22-
*/pkg/**/junit_node*.xml
22+
**/pkg/**/junit_node*.xml
2323

2424
dist/
2525
.local/

‎.go-tools

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
github.com/oligot/[email protected]
2-
sigs.k8s.io/controller-runtime/tools/[email protected]20230611165747-7edfc04cacbd
2+
sigs.k8s.io/controller-runtime/tools/[email protected]20230817155522-304027bcbe4b

‎.golangci.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,3 +78,6 @@ issues:
7878
- source: "flags.Parse|response.WriteError"
7979
linters:
8080
- errcheck
81+
- source: "^// \\+kubebuilder:"
82+
linters:
83+
- lll

‎.goreleaser.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ release:
2323
### Summary
2424
**Full Changelog**: https://github.com/d2iq-labs/{{.ProjectName}}/compare/{{ .PreviousTag }}...{{ .Tag }}
2525
26+
gomod:
27+
proxy: true
28+
2629
builds:
2730
- id: capi-runtime-extensions
2831
dir: ./cmd/capi-runtime-extensions

‎.pre-commit-config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ repos:
3131
stages: [commit]
3232
- id: check-added-large-files
3333
stages: [commit]
34+
exclude: ^pkg/handlers/cni/calico/manifests/tigera-operator-configmap.yaml$
3435
- id: check-case-conflict
3536
stages: [commit]
3637
- id: check-merge-conflict

‎README.md

Lines changed: 10 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -23,16 +23,18 @@ example), run:
2323
make SKIP_BUILD=true dev.run-on-kind
2424
```
2525

26-
To create a cluster with [clusterctl](https://cluster-api.sigs.k8s.io/user/quick-start.html), run:
26+
To create a cluster with [clusterctl](https://cluster-api.sigs.k8s.io/user/quick-start.html), and label it for Calico
27+
CNI at the same time, run:
2728

2829
```shell
29-
env POD_SECURITY_STANDARD_ENABLED=false \
30-
clusterctl generate cluster capi-quickstart \
31-
--flavor development \
32-
--kubernetes-version v1.27.2 \
33-
--control-plane-machine-count=1 \
34-
--worker-machine-count=1 | \
35-
kubectl apply --server-side -f -
30+
clusterctl generate cluster capi-quickstart \
31+
--flavor development \
32+
--kubernetes-version v1.27.2 \
33+
--control-plane-machine-count=1 \
34+
--worker-machine-count=1 | \
35+
gojq --yaml-input --yaml-output \
36+
'. | (select(.kind=="Cluster").metadata.labels["capiext.labs.d2iq.io/cni"]|="calico")' | \
37+
kubectl apply --server-side -f -
3638
```
3739

3840
Wait until control plane is ready:
@@ -55,19 +57,6 @@ kubectl config set-cluster capi-quickstart \
5557
--server=https://$(docker port capi-quickstart-lb 6443/tcp)
5658
```
5759

58-
Deploy Calico to the workload cluster (TODO deploy via lifecycle hook):
59-
60-
```shell
61-
helm repo add --force-update projectcalico https://docs.tigera.io/calico/charts
62-
helm upgrade --install calico projectcalico/tigera-operator \
63-
--version v3.26.1 \
64-
--namespace tigera-operator \
65-
--create-namespace \
66-
--wait \
67-
--wait-for-jobs \
68-
--kubeconfig capd-kubeconfig
69-
```
70-
7160
Wait until all nodes are ready (this indicates that CNI has been deployed successfully):
7261

7362
```shell

‎charts/capi-runtime-extensions/README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,13 @@ A Helm chart for capi-runtime-extensions
3131
| controllers.enableLeaderElection | bool | `false` | |
3232
| deployment.replicas | int | `1` | |
3333
| env | object | `{}` | |
34+
| handlers.CalicoCNI.defaultInstallationConfigMaps.DockerCluster.configMap.content | string | `""` | |
35+
| handlers.CalicoCNI.defaultInstallationConfigMaps.DockerCluster.configMap.name | string | `"calico-cni-installation-dockercluster"` | |
36+
| handlers.CalicoCNI.defaultInstallationConfigMaps.DockerCluster.create | bool | `true` | |
37+
| handlers.CalicoCNI.defaultPodSubnet | string | `"192.168.0.0/16"` | |
38+
| handlers.CalicoCNI.defaultTigeraOperatorConfigMap.name | string | `"tigera-operator"` | |
39+
| handlers.CalicoCNI.enabled | bool | `true` | |
40+
| handlers.ServiceLoadBalancerGC.enabled | bool | `true` | |
3441
| image.pullPolicy | string | `"IfNotPresent"` | |
3542
| image.repository | string | `"ghcr.io/d2iq-labs/capi-runtime-extensions"` | |
3643
| image.tag | string | `""` | |

‎charts/capi-runtime-extensions/templates/cni/calico/manifests/docker/installation.yaml

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Copyright 2023 D2iQ, Inc. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
{{- if and .Values.handlers.CalicoCNI.enabled .Values.handlers.CalicoCNI.defaultInstallationConfigMaps.DockerCluster.create }}
5+
apiVersion: v1
6+
kind: ConfigMap
7+
metadata:
8+
name: '{{ .Values.handlers.CalicoCNI.defaultInstallationConfigMaps.DockerCluster.configMap.name }}'
9+
data:
10+
calico-installation: |
11+
{{- if .Values.handlers.CalicoCNI.defaultInstallationConfigMaps.DockerCluster.configMap.content -}}
12+
{{ .Values.handlers.CalicoCNI.defaultInstallationConfigMaps.DockerCluster.configMap.content | nindent 4}}
13+
{{- else -}}
14+
# This section includes base Calico installation configuration.
15+
# For more information, see: https://docs.projectcalico.org/reference/installation/api
16+
apiVersion: operator.tigera.io/v1
17+
kind: Installation
18+
metadata:
19+
name: default
20+
spec:
21+
cni:
22+
type: Calico
23+
# Configures Calico networking.
24+
calicoNetwork:
25+
# Note: The ipPools section cannot be modified post-install.
26+
ipPools:
27+
- blockSize: 26
28+
cidr: {{ .Values.handlers.CalicoCNI.defaultPodSubnet }}
29+
encapsulation: VXLANCrossSubnet
30+
natOutgoing: Enabled
31+
nodeSelector: all()
32+
nodeMetricsPort: 9091
33+
typhaMetricsPort: 9093
34+
{{- end -}}
35+
{{- end -}}

‎charts/capi-runtime-extensions/templates/cni/calico/manifests/tigera-operator-configmap.yaml

Lines changed: 11 additions & 0 deletions
Large diffs are not rendered by default.

‎charts/capi-runtime-extensions/templates/deployment.yaml

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -27,20 +27,28 @@ spec:
2727
- name: webhook
2828
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default $.Chart.AppVersion }}"
2929
imagePullPolicy: "{{ .Values.image.pullPolicy }}"
30-
{{- if .Values.env }}
31-
env:
32-
{{- range $key, $value := .Values.env }}
33-
- name: "{{ $key }}"
34-
value: "{{ $value }}"
35-
{{- end }}
36-
{{- end }}
3730
args:
3831
- --controllermanager.cert-dir=/controllers-certs/
3932
- --controllermanager.leader-elect={{ if gt (.Values.deployment.replicas | int) 1 }}true{{ else }}{{ .Values.controllers.enableLeaderElection }}{{ end }}
4033
- --runtimehooks.cert-dir=/runtimehooks-certs/
34+
{{- range $key, $value := .Values.handlers }}{{ if $value.enabled }}
35+
- --runtimehooks.enabled-handlers={{ $key }}
36+
{{ end }}{{- end }}
37+
- --runtimehooks.calicocni.defaultsNamespace=$(POD_NAMESPACE)
4138
{{- range $key, $value := .Values.extraArgs }}
4239
- --{{ $key }}={{ $value }}
4340
{{- end }}
41+
env:
42+
- name: POD_NAMESPACE
43+
valueFrom:
44+
fieldRef:
45+
fieldPath: metadata.namespace
46+
{{- if .Values.env }}
47+
{{- range $key, $value := .Values.env }}
48+
- name: "{{ $key }}"
49+
value: "{{ $value }}"
50+
{{- end }}
51+
{{- end }}
4452
ports:
4553
- containerPort: 8443
4654
name: controllers

‎charts/capi-runtime-extensions/templates/role.yaml

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,6 @@ rules:
1010
- ""
1111
resources:
1212
- configmaps
13-
- namespaces
14-
- secrets
1513
verbs:
1614
- create
1715
- delete
@@ -21,10 +19,14 @@ rules:
2119
- update
2220
- watch
2321
- apiGroups:
24-
- cluster.x-k8s.io
22+
- addons.cluster.x-k8s.io
2523
resources:
26-
- clusters
24+
- clusterresourcesets
2725
verbs:
26+
- create
27+
- delete
2828
- get
2929
- list
30+
- patch
31+
- update
3032
- watch

‎charts/capi-runtime-extensions/values.schema.json

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,68 @@
4545
"type": "object",
4646
"additionalProperties": { "type": "string" }
4747
},
48+
"handlers": {
49+
"type": "object",
50+
"properties": {
51+
"CalicoCNI": {
52+
"type": "object",
53+
"properties": {
54+
"defaultInstallationConfigMaps": {
55+
"type": "object",
56+
"properties": {
57+
"DockerCluster": {
58+
"type": "object",
59+
"properties": {
60+
"configMap": {
61+
"type": "object",
62+
"properties": {
63+
"content": {
64+
"type": "string"
65+
},
66+
"name": {
67+
"type": "string"
68+
}
69+
},
70+
"required": ["name"]
71+
},
72+
"create": {
73+
"type": "boolean",
74+
"default": true
75+
}
76+
}
77+
}
78+
}
79+
},
80+
"defaultPodSubnet": {
81+
"type": "string",
82+
"default": "192.168.0.0/16"
83+
},
84+
"defaultTigeraOperatorConfigMap": {
85+
"type": "object",
86+
"properties": {
87+
"name": {
88+
"type": "string"
89+
}
90+
},
91+
"required": ["name"]
92+
},
93+
"enabled": {
94+
"type": "boolean",
95+
"default": true
96+
}
97+
}
98+
},
99+
"ServiceLoadBalancerGC": {
100+
"type": "object",
101+
"properties": {
102+
"enabled": {
103+
"type": "boolean",
104+
"default": true
105+
}
106+
}
107+
}
108+
}
109+
},
48110
"image": {
49111
"type": "object",
50112
"properties": {

‎charts/capi-runtime-extensions/values.yaml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,21 @@
11
# Copyright 2023 D2iQ, Inc. All rights reserved.
22
# SPDX-License-Identifier: Apache-2.0
33

4+
handlers:
5+
CalicoCNI:
6+
enabled: true
7+
defaultPodSubnet: 192.168.0.0/16
8+
defaultTigeraOperatorConfigMap:
9+
name: tigera-operator
10+
defaultInstallationConfigMaps:
11+
DockerCluster:
12+
create: true
13+
configMap:
14+
name: calico-cni-installation-dockercluster
15+
content: ""
16+
ServiceLoadBalancerGC:
17+
enabled: true
18+
419
deployment:
520
replicas: 1
621

‎devbox.lock

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -14,16 +14,16 @@
1414
"version": "3.8.0"
1515
},
1616
"clusterctl@latest": {
17-
"last_modified": "2023-07-23T03:35:12Z",
18-
"resolved": "github:NixOS/nixpkgs/af8cd5ded7735ca1df1a1174864daab75feeb64a#clusterctl",
17+
"last_modified": "2023-08-08T03:07:33Z",
18+
"resolved": "github:NixOS/nixpkgs/844ffa82bbe2a2779c86ab3a72ff1b4176cec467#clusterctl",
1919
"source": "devbox-search",
20-
"version": "1.4.4"
20+
"version": "1.5.0"
2121
},
2222
"crane@latest": {
23-
"last_modified": "2023-06-30T04:44:22Z",
24-
"resolved": "github:NixOS/nixpkgs/3c614fbc76fc152f3e1bc4b2263da6d90adf80fb#crane",
23+
"last_modified": "2023-08-08T03:07:33Z",
24+
"resolved": "github:NixOS/nixpkgs/844ffa82bbe2a2779c86ab3a72ff1b4176cec467#crane",
2525
"source": "devbox-search",
26-
"version": "0.15.2"
26+
"version": "0.16.1"
2727
},
2828
"envsubst@latest": {
2929
"last_modified": "2023-06-30T04:44:22Z",
@@ -140,10 +140,10 @@
140140
"version": "3.11.1"
141141
},
142142
"kubectl@latest": {
143-
"last_modified": "2023-06-30T04:44:22Z",
144-
"resolved": "github:NixOS/nixpkgs/3c614fbc76fc152f3e1bc4b2263da6d90adf80fb#kubectl",
143+
"last_modified": "2023-08-08T03:07:33Z",
144+
"resolved": "github:NixOS/nixpkgs/844ffa82bbe2a2779c86ab3a72ff1b4176cec467#kubectl",
145145
"source": "devbox-search",
146-
"version": "1.27.3"
146+
"version": "1.27.4"
147147
},
148148
"kubernetes-controller-tools@latest": {
149149
"last_modified": "2023-07-23T03:35:12Z",
@@ -170,8 +170,8 @@
170170
"version": "3.3.3"
171171
},
172172
"shfmt@latest": {
173-
"last_modified": "2023-06-30T04:44:22Z",
174-
"resolved": "github:NixOS/nixpkgs/3c614fbc76fc152f3e1bc4b2263da6d90adf80fb#shfmt",
173+
"last_modified": "2023-08-08T03:07:33Z",
174+
"resolved": "github:NixOS/nixpkgs/844ffa82bbe2a2779c86ab3a72ff1b4176cec467#shfmt",
175175
"source": "devbox-search",
176176
"version": "3.7.0"
177177
},

‎go.mod

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ require (
1515
k8s.io/client-go v0.28.0
1616
k8s.io/component-base v0.28.0
1717
k8s.io/klog/v2 v2.100.1
18+
k8s.io/utils v0.0.0-20230406110748-d93618cff8a2
1819
sigs.k8s.io/cluster-api v1.5.0
1920
sigs.k8s.io/controller-runtime v0.15.1
2021
)
@@ -72,7 +73,6 @@ require (
7273
k8s.io/apiextensions-apiserver v0.27.2 // indirect
7374
k8s.io/cluster-bootstrap v0.27.2 // indirect
7475
k8s.io/kube-openapi v0.0.0-20230717233707-2695361300d9 // indirect
75-
k8s.io/utils v0.0.0-20230406110748-d93618cff8a2 // indirect
7676
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
7777
sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
7878
sigs.k8s.io/yaml v1.3.0 // indirect

‎hack/addons/kustomize/tigera-operator/ds-priorityClass.yaml

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Copyright 2023 D2iQ, Inc. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
apiVersion: apps/v1
5+
kind: Deployment
6+
metadata:
7+
name: NA
8+
spec:
9+
template:
10+
spec:
11+
priorityClassName: system-cluster-critical

‎hack/addons/kustomize/tigera-operator/kustomization.yaml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Copyright 2023 D2iQ, Inc. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
apiVersion: kustomize.config.k8s.io/v1beta1
5+
kind: Kustomization
6+
metadata:
7+
name: tigera-operator
8+
sortOptions:
9+
order: fifo
10+
patches:
11+
- path: ds-priorityClass.yaml
12+
target:
13+
kind: Deployment
14+
name: tigera-operator
15+
namespace: tigera-operator
16+
resources:
17+
- tigera-operator.yaml

0 commit comments

Comments
 (0)