Skip to content

Commit 73b1eb7

Browse files
supershalsupershal
authored
test: add encryptionAtRest config in capi-quick-start (#659)
**What problem does this PR solve?**: This PR is stacked on #610 - adds encryptionAtRest variable to all capi-quick-start examples - documentation for encryptionAtRest **How Has This Been Tested?**: <!-- Please describe the tests that you ran to verify your changes. Provide output from the tests and any manual steps needed to replicate the tests. --> Caren e2e tests Manually on docker and aws
1 parent d136154 commit 73b1eb7

19 files changed

+122
-3
lines changed

‎docs/content/customization/generic/encryption-at-rest.md

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
+++
2+
title = "Encryption At Rest"
3+
+++
4+
5+
`encryptionAtRest` variable enables encrypting kubernetes resources at rest using provided encryption provider.
6+
When this variable is set, kuberntetes `secrets` and `configmap`s are encrypted before writing them at `etcd`.
7+
8+
If the `encryptionAtRest` property is not specified, then
9+
the customization will be skipped. The `secrets` and `configmaps` will not be stored as encrypted in `etcd`.
10+
11+
We support following encryption providers
12+
13+
- aescbc
14+
- secretbox
15+
16+
More information about encryption at-rest: [Encrypting Confidential Data at Rest
17+
](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/)
18+
19+
## Example
20+
21+
To encrypt `configmaps` and `secrets` kubernetes resources using `aescbc` encryption provider:
22+
23+
```yaml
24+
apiVersion: cluster.x-k8s.io/v1beta1
25+
kind: Cluster
26+
metadata:
27+
name: <NAME>
28+
spec:
29+
topology:
30+
variables:
31+
- name: clusterConfig
32+
value:
33+
encryptionAtRest:
34+
providers:
35+
- aescbc: {}
36+
```
37+
38+
Applying this configuration will result in
39+
40+
1. `<CLUSTER_NAME>-encryption-config` secret generated.
41+
42+
A secret key for the encryption provider is generated and stored in `<CLUSTER_NAME>-encryption-config` secret.
43+
The APIServer will be configured to use the secret key to encrypt `secrets` and
44+
`configmaps` kubernetes resources before writing them to etcd.
45+
When reading resources from `etcd`, encryption provider that matches the stored data attempts in order to decrypt the data.
46+
CAREN currently does not rotate the key once it generated.
47+
48+
1. Configure APIServer with encryption configuration:
49+
50+
- `KubeadmControlPlaneTemplate`:
51+
52+
```yaml
53+
spec:
54+
kubeadmConfigSpec:
55+
clusterConfiguration:
56+
apiServer:
57+
extraArgs:
58+
encryption-provider-config: /etc/kubernetes/pki/encryptionconfig.yaml
59+
files:
60+
- contentFrom:
61+
secret:
62+
key: config
63+
name: <CLUSTER_NAME>-encryption-config
64+
path: /etc/kubernetes/pki/encryptionconfig.yaml
65+
permissions: "0640"
66+
```

‎examples/capi-quick-start/aws-cluster-calico-crs.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@ spec:
4747
baseOS: ${AMI_LOOKUP_BASEOS}
4848
format: ${AMI_LOOKUP_FORMAT}
4949
org: "${AMI_LOOKUP_ORG}"
50+
encryptionAtRest:
51+
providers:
52+
- aescbc: {}
5053
- name: workerConfig
5154
value:
5255
aws:

‎examples/capi-quick-start/aws-cluster-calico-helm-addon.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@ spec:
4747
baseOS: ${AMI_LOOKUP_BASEOS}
4848
format: ${AMI_LOOKUP_FORMAT}
4949
org: "${AMI_LOOKUP_ORG}"
50+
encryptionAtRest:
51+
providers:
52+
- aescbc: {}
5053
- name: workerConfig
5154
value:
5255
aws:

‎examples/capi-quick-start/aws-cluster-cilium-crs.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@ spec:
4747
baseOS: ${AMI_LOOKUP_BASEOS}
4848
format: ${AMI_LOOKUP_FORMAT}
4949
org: "${AMI_LOOKUP_ORG}"
50+
encryptionAtRest:
51+
providers:
52+
- aescbc: {}
5053
- name: workerConfig
5154
value:
5255
aws:

‎examples/capi-quick-start/aws-cluster-cilium-helm-addon.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,9 @@ spec:
4747
baseOS: ${AMI_LOOKUP_BASEOS}
4848
format: ${AMI_LOOKUP_FORMAT}
4949
org: "${AMI_LOOKUP_ORG}"
50+
encryptionAtRest:
51+
providers:
52+
- aescbc: {}
5053
- name: workerConfig
5154
value:
5255
aws:

‎examples/capi-quick-start/docker-cluster-calico-crs.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ spec:
2929
strategy: ClusterResourceSet
3030
nfd:
3131
strategy: ClusterResourceSet
32+
encryptionAtRest:
33+
providers:
34+
- aescbc: {}
3235
- name: workerConfig
3336
value: {}
3437
version: ${KUBERNETES_VERSION}

‎examples/capi-quick-start/docker-cluster-calico-helm-addon.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ spec:
2929
strategy: HelmAddon
3030
nfd:
3131
strategy: HelmAddon
32+
encryptionAtRest:
33+
providers:
34+
- aescbc: {}
3235
- name: workerConfig
3336
value: {}
3437
version: ${KUBERNETES_VERSION}

‎examples/capi-quick-start/docker-cluster-cilium-crs.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ spec:
2929
strategy: ClusterResourceSet
3030
nfd:
3131
strategy: ClusterResourceSet
32+
encryptionAtRest:
33+
providers:
34+
- aescbc: {}
3235
- name: workerConfig
3336
value: {}
3437
version: ${KUBERNETES_VERSION}

‎examples/capi-quick-start/docker-cluster-cilium-helm-addon.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ spec:
2929
strategy: HelmAddon
3030
nfd:
3131
strategy: HelmAddon
32+
encryptionAtRest:
33+
providers:
34+
- aescbc: {}
3235
- name: workerConfig
3336
value: {}
3437
version: ${KUBERNETES_VERSION}

‎examples/capi-quick-start/nutanix-cluster-calico-crs.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,9 @@ spec:
105105
systemDiskSize: 40Gi
106106
vcpuSockets: 2
107107
vcpusPerSocket: 1
108+
encryptionAtRest:
109+
providers:
110+
- aescbc: {}
108111
imageRegistries:
109112
- credentials:
110113
secretRef:

‎examples/capi-quick-start/nutanix-cluster-calico-helm-addon.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,9 @@ spec:
105105
systemDiskSize: 40Gi
106106
vcpuSockets: 2
107107
vcpusPerSocket: 1
108+
encryptionAtRest:
109+
providers:
110+
- aescbc: {}
108111
imageRegistries:
109112
- credentials:
110113
secretRef:

‎examples/capi-quick-start/nutanix-cluster-cilium-crs.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,9 @@ spec:
105105
systemDiskSize: 40Gi
106106
vcpuSockets: 2
107107
vcpusPerSocket: 1
108+
encryptionAtRest:
109+
providers:
110+
- aescbc: {}
108111
imageRegistries:
109112
- credentials:
110113
secretRef:

‎examples/capi-quick-start/nutanix-cluster-cilium-helm-addon.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,9 @@ spec:
105105
systemDiskSize: 40Gi
106106
vcpuSockets: 2
107107
vcpusPerSocket: 1
108+
encryptionAtRest:
109+
providers:
110+
- aescbc: {}
108111
imageRegistries:
109112
- credentials:
110113
secretRef:

‎hack/examples/bases/aws/cluster/kustomization.yaml.tmpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,9 @@ patches:
4545
- target:
4646
kind: Cluster
4747
path: ../../../patches/aws/config-var.yaml
48+
- target:
49+
kind: Cluster
50+
path: ../../../patches/encryption.yaml
4851

4952
# Delete the clusterclass-specific resources.
5053
- target:

‎hack/examples/bases/docker/cluster/kustomization.yaml.tmpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,3 +35,6 @@ patches:
3535
- target:
3636
kind: Cluster
3737
path: ../../../patches/cluster-autoscaler.yaml
38+
- target:
39+
kind: Cluster
40+
path: ../../../patches/encryption.yaml

‎hack/examples/bases/nutanix/cluster/kustomization.yaml.tmpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,9 @@ patches:
3636
- target:
3737
kind: Cluster
3838
path: ../../../patches/nutanix/initialize-variables.yaml
39+
- target:
40+
kind: Cluster
41+
path: ../../../patches/encryption.yaml
3942

4043
# Remove Additional Trust Bundle ConfigMap
4144
- target:

‎hack/examples/patches/encryption.yaml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Copyright 2024 Nutanix. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
- op: "add"
5+
path: "/spec/topology/variables/0/value/encryptionAtRest"
6+
value:
7+
providers:
8+
- aescbc: {}

‎test/e2e/ownerreference_helpers.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -167,8 +167,8 @@ var (
167167
// https://github.com/kubernetes-sigs/cluster-api/tree/main/docs/book/src/reference/owner_references.md.
168168
KubernetesReferenceAssertions = map[string]func([]metav1.OwnerReference) error{
169169
secretKind: func(owners []metav1.OwnerReference) error {
170-
// TODO:deepakm-ntnx Currently pc-creds, pc-creds-for-csi, dockerhub-credentials
171-
// and registry-creds have unexpected owners which needs more investigation
170+
// TODO:deepakm-ntnx Currently pc-creds, pc-creds-for-csi, dockerhub-credentials,
171+
// registry-creds, and encryption config secrets have unexpected owners which needs more investigation.
172172
return nil
173173
},
174174
configMapKind: func(owners []metav1.OwnerReference) error {

‎test/e2e/quick_start_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ var _ = Describe("Quick start", Serial, func() {
9393
framework.DockerInfraOwnerReferenceAssertions,
9494
framework.KubeadmBootstrapOwnerReferenceAssertions,
9595
framework.KubeadmControlPlaneOwnerReferenceAssertions,
96-
framework.KubernetesReferenceAssertions,
96+
AWSInfraOwnerReferenceAssertions,
9797
NutanixInfraOwnerReferenceAssertions,
9898
AddonReferenceAssertions,
9999
KubernetesReferenceAssertions,

0 commit comments

Comments
 (0)