Skip to content

ci: images tool #822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Sep 9, 2024
Merged

ci: images tool #822

merged 12 commits into from
Sep 9, 2024

Conversation

faiq
Copy link
Contributor

@faiq faiq commented Jul 25, 2024

What problem does this PR solve?:
Adds a tool to fetch all images CAREN uses

Which issue(s) this PR fixes:
Fixes #

How Has This Been Tested?:

I ran the following command from hack/tools/fetch-images

$ go run main.go -chart-directory=../../../charts/cluster-api-runtime-extensions-nutanix/ --helm-chart-configmap=../../../charts/cluster-api-runtime-extensions-nutanix/templates/helm-config.yaml -caren-version=v0.13.1
ghcr.io/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix:v0.13.1
ghcr.io/nutanix-cloud-native/caren-helm-reg:v0.13.1
registry.k8s.io/sig-storage/snapshot-controller:v8.0.1
registry.k8s.io/provider-aws/cloud-controller-manager:v1.27.7
registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.6
registry.k8s.io/provider-aws/cloud-controller-manager:v1.29.3
registry.k8s.io/provider-aws/cloud-controller-manager:v1.30.1
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/node-driver-registrar:v2.11.0-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/external-provisioner:v5.0.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-attacher:v4.6.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-resizer:v1.11.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
registry.k8s.io/autoscaling/cluster-autoscaler:v1.30.0
quay.io/metallb/speaker:v0.14.8
quay.io/frrouting/frr:9.1.0
quay.io/metallb/speaker:v0.14.8
quay.io/metallb/controller:v0.14.8
registry.k8s.io/nfd/node-feature-discovery:v0.16.1-minimal
registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-provisioner:v3.6.3
registry.k8s.io/sig-storage/csi-attacher:v4.4.3
registry.k8s.io/sig-storage/csi-resizer:v1.9.3
registry.k8s.io/sig-storage/csi-snapshotter:v3.0.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.10.0
docker.io/nutanix/ntnx-csi-precheck:3.0.0
quay.io/cilium/cilium:v1.16.0
quay.io/cilium/cilium-envoy@sha256:bd5ff8c66716080028f414ec1cb4f7dc66f40d2fb5a009fff187f4a9b90b566b
quay.io/cilium/operator-generic:v1.16.0
quay.io/cilium/certgen:v0.2.0
docker.io/rancher/local-path-provisioner:v0.0.28
ghcr.io/nutanix-cloud-native/cloud-provider-nutanix/controller:v0.3.2
quay.io/tigera/operator:v1.34.0

Special notes for your reviewer:

Depends on #896.

@faiq faiq changed the base branch from faiq/sync-helm-values-to-templates to faiq/helm-values-include July 25, 2024 21:51
@faiq faiq changed the title Faiq/get images ci: images tool Jul 25, 2024
@faiq faiq force-pushed the faiq/helm-values-include branch from 28da0c0 to 4c52882 Compare July 26, 2024 18:57
@faiq faiq force-pushed the faiq/get-images branch 2 times, most recently from 48b8592 to 8c2e48f Compare July 26, 2024 20:17
@faiq faiq force-pushed the faiq/helm-values-include branch 2 times, most recently from 9b73715 to d1571b8 Compare August 14, 2024 10:03
@faiq faiq force-pushed the faiq/get-images branch from 8c2e48f to a18cf23 Compare August 14, 2024 15:22
@faiq faiq force-pushed the faiq/helm-values-include branch from 891cfa5 to 98f9835 Compare August 14, 2024 18:10
@faiq faiq force-pushed the faiq/get-images branch 3 times, most recently from d759611 to e0f39c3 Compare August 14, 2024 18:55
@faiq faiq force-pushed the faiq/helm-values-include branch 2 times, most recently from d9bb13e to 829e551 Compare August 15, 2024 20:35
@faiq faiq force-pushed the faiq/get-images branch 3 times, most recently from 2edbf28 to 206bb64 Compare August 15, 2024 20:46
@faiq faiq force-pushed the faiq/helm-values-include branch from 2576500 to aa40161 Compare August 15, 2024 22:10
Copy link
Contributor

@supershal supershal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run in locally.

❯ cat caren-images.txt
ghcr.io/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix:v0.0.0-dev
ghcr.io/nutanix-cloud-native/caren-helm-reg:v0.0.0-dev
quay.io/metallb/speaker:v0.14.8
quay.io/frrouting/frr:9.1.0
quay.io/metallb/speaker:v0.14.8
quay.io/metallb/controller:v0.14.8
registry.k8s.io/nfd/node-feature-discovery:v0.16.1-minimal
ghcr.io/nutanix-cloud-native/cloud-provider-nutanix/controller:v0.4.0
registry.k8s.io/sig-storage/snapshot-controller:v8.0.1
quay.io/tigera/operator:v1.34.3
registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.6
registry.k8s.io/provider-aws/cloud-controller-manager:v1.29.3
registry.k8s.io/provider-aws/cloud-controller-manager:v1.30.1
registry.k8s.io/provider-aws/cloud-controller-manager:v1.27.7
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/node-driver-registrar:v2.11.0-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/external-provisioner:v5.0.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-attacher:v4.6.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-resizer:v1.11.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
docker.io/rancher/local-path-provisioner:v0.0.28
quay.io/cilium/cilium:v1.16.1
quay.io/cilium/cilium-envoy@sha256:bd5ff8c66716080028f414ec1cb4f7dc66f40d2fb5a009fff187f4a9b90b566b
quay.io/cilium/operator-generic:v1.16.1
quay.io/cilium/certgen:v0.2.0
registry.k8s.io/autoscaling/cluster-autoscaler:v1.30.0
registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-provisioner:v3.6.3
registry.k8s.io/sig-storage/csi-attacher:v4.4.3
registry.k8s.io/sig-storage/csi-resizer:v1.9.3
registry.k8s.io/sig-storage/csi-snapshotter:v3.0.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.10.0
docker.io/nutanix/ntnx-csi-precheck:3.0.0

Have you already tested this with konvoy e2e airgap tests?

@faiq faiq force-pushed the faiq/helm-values-include branch from aa40161 to 747dafc Compare August 21, 2024 14:20
@faiq faiq force-pushed the faiq/helm-values-include branch 2 times, most recently from 0024538 to 33c4f5b Compare August 29, 2024 15:59
@faiq faiq changed the base branch from faiq/helm-values-include to jimmi/helm-values-include September 3, 2024 17:44
Copy link
Member

@jimmidyson jimmidyson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@faiq faiq enabled auto-merge (squash) September 7, 2024 21:34
@jimmidyson jimmidyson disabled auto-merge September 8, 2024 12:27
@jimmidyson jimmidyson enabled auto-merge (squash) September 8, 2024 12:27
jimmidyson added a commit that referenced this pull request Sep 9, 2024
Calico images are deployed by the tigera-operator and as such are not
discoverable by the helm list images plugin. Using the extra images file
functionality of the helm list images plugin allows for adding these
images as templated strings that will adapt with the chart version as it
is updated in future.

This commit also fixes up some other things:

- Use `filepath` instead of `path` for portabiity
- Output the image list to stdout from make to allow user to decide
  where to redirect list to
- Omit changing dir from image list when running as goreleaser hook
- Slightly more defensive when checking errors

Final output is:

```shell
$ make list-images

docker.io/nutanix/ntnx-csi-precheck:3.0.0
docker.io/nutanix/ntnx-csi:3.0.0
docker.io/rancher/local-path-provisioner:v0.0.28
ghcr.io/nutanix-cloud-native/caren-helm-reg:v0.0.0-dev
ghcr.io/nutanix-cloud-native/cloud-provider-nutanix/controller:v0.4.1
ghcr.io/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix:v0.0.0-dev
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.34.0
public.ecr.aws/eks-distro/kubernetes-csi/external-attacher:v4.6.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-provisioner:v5.0.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-resizer:v1.11.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/node-driver-registrar:v2.11.0-eks-1-30-10
quay.io/calico/apiserver:v3.28.1
quay.io/calico/cni:v3.28.1
quay.io/calico/csi:v3.28.1
quay.io/calico/ctl:v3.28.1
quay.io/calico/kube-controllers:v3.28.1
quay.io/calico/node-driver-registrar:v3.28.1
quay.io/calico/node:v3.28.1
quay.io/calico/pod2daemon-flexvol:v3.28.1
quay.io/calico/typha:v3.28.1
quay.io/cilium/certgen:v0.2.0
quay.io/cilium/cilium-envoy@sha256:bd5ff8c66716080028f414ec1cb4f7dc66f40d2fb5a009fff187f4a9b90b566b
quay.io/cilium/cilium:v1.16.1
quay.io/cilium/operator-generic:v1.16.1
quay.io/frrouting/frr:9.1.0
quay.io/metallb/controller:v0.14.8
quay.io/metallb/speaker:v0.14.8
quay.io/tigera/operator:v1.34.3
registry.k8s.io/autoscaling/cluster-autoscaler:v1.30.0
registry.k8s.io/nfd/node-feature-discovery:v0.16.4
registry.k8s.io/provider-aws/cloud-controller-manager:v1.27.9
registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.9
registry.k8s.io/provider-aws/cloud-controller-manager:v1.29.6
registry.k8s.io/provider-aws/cloud-controller-manager:v1.30.2
registry.k8s.io/provider-aws/cloud-controller-manager:v1.31.0
registry.k8s.io/sig-storage/csi-attacher:v4.4.3
registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.10.0
registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.3
registry.k8s.io/sig-storage/csi-provisioner:v3.6.3
registry.k8s.io/sig-storage/csi-resizer:v1.9.3
registry.k8s.io/sig-storage/csi-snapshotter:v3.0.3
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/snapshot-controller:v8.1.0
```

Depends on #822.
@jimmidyson jimmidyson disabled auto-merge September 9, 2024 12:48
@jimmidyson jimmidyson enabled auto-merge (squash) September 9, 2024 12:52
@jimmidyson jimmidyson merged commit 700ea07 into main Sep 9, 2024
38 checks passed
@jimmidyson jimmidyson deleted the faiq/get-images branch September 9, 2024 13:20
jimmidyson added a commit that referenced this pull request Sep 9, 2024
Calico images are deployed by the tigera-operator and as such are not
discoverable by the helm list images plugin. Using the extra images file
functionality of the helm list images plugin allows for adding these
images as templated strings that will adapt with the chart version as it
is updated in future.

This commit also fixes up some other things:

- Use `filepath` instead of `path` for portabiity
- Output the image list to stdout from make to allow user to decide
  where to redirect list to
- Omit changing dir from image list when running as goreleaser hook
- Slightly more defensive when checkinbg errors

Depends on #822.
@github-actions github-actions bot mentioned this pull request Sep 12, 2024
dlipovetsky pushed a commit that referenced this pull request Sep 25, 2024
🤖 I have created a release *beep* *boop*
---


## 0.16.0 (2024-09-25)

<!-- Release notes generated using configuration in .github/release.yaml
at main -->

## What's Changed
### Exciting New Features 🎉
* feat: Build with go 1.23 by @jimmidyson in
#889
* feat: Enable Hubble Relay in Cilium deployment via CAAPH by
@dlipovetsky in
#899
* feat: Extract CAAPH values templates to files by @jimmidyson in
#896
* feat: Build with go 1.23.1 by @jimmidyson in
#913
* feat: Support node taints per nodepool and control plane by
@jimmidyson in
#909
### Fixes 🔧
* fix: Remove deprecated toleration for node-role.kubernetes.io/master
by @jimmidyson in
#895
* fix: Do not use digests for Cilium images by @jimmidyson in
#901
### Other Changes
* test: Bump Kubernetes versions for tests by @jimmidyson in
#893
* ci: images tool by @faiq in
#822
* build: Include Calico images in image list by @jimmidyson in
#903
* build: Use upstream packages again from upstream by @jimmidyson in
#908
* ci(main): enable creating release-please PR from release branches by
@supershal in
#912
* docs: Enable dark mode, add Nutanix color, and header links by
@jimmidyson in
#915


**Full Changelog**:
v0.15.0...v0.16.0

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
dlipovetsky pushed a commit that referenced this pull request Sep 27, 2024
**What problem does this PR solve?**:
Adds a tool to fetch all images CAREN uses

**Which issue(s) this PR fixes**:
Fixes #

**How Has This Been Tested?**:
<!--
Please describe the tests that you ran to verify your changes.
Provide output from the tests and any manual steps needed to replicate
the tests.
-->
I ran the following command from `hack/tools/fetch-images`

```
$ go run main.go -chart-directory=../../../charts/cluster-api-runtime-extensions-nutanix/ --helm-chart-configmap=../../../charts/cluster-api-runtime-extensions-nutanix/templates/helm-config.yaml -caren-version=v0.13.1
ghcr.io/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix:v0.13.1
ghcr.io/nutanix-cloud-native/caren-helm-reg:v0.13.1
registry.k8s.io/sig-storage/snapshot-controller:v8.0.1
registry.k8s.io/provider-aws/cloud-controller-manager:v1.27.7
registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.6
registry.k8s.io/provider-aws/cloud-controller-manager:v1.29.3
registry.k8s.io/provider-aws/cloud-controller-manager:v1.30.1
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/node-driver-registrar:v2.11.0-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/external-provisioner:v5.0.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-attacher:v4.6.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-resizer:v1.11.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
registry.k8s.io/autoscaling/cluster-autoscaler:v1.30.0
quay.io/metallb/speaker:v0.14.8
quay.io/frrouting/frr:9.1.0
quay.io/metallb/speaker:v0.14.8
quay.io/metallb/controller:v0.14.8
registry.k8s.io/nfd/node-feature-discovery:v0.16.1-minimal
registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-provisioner:v3.6.3
registry.k8s.io/sig-storage/csi-attacher:v4.4.3
registry.k8s.io/sig-storage/csi-resizer:v1.9.3
registry.k8s.io/sig-storage/csi-snapshotter:v3.0.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.10.0
docker.io/nutanix/ntnx-csi-precheck:3.0.0
quay.io/cilium/cilium:v1.16.0
quay.io/cilium/cilium-envoy@sha256:bd5ff8c66716080028f414ec1cb4f7dc66f40d2fb5a009fff187f4a9b90b566b
quay.io/cilium/operator-generic:v1.16.0
quay.io/cilium/certgen:v0.2.0
docker.io/rancher/local-path-provisioner:v0.0.28
ghcr.io/nutanix-cloud-native/cloud-provider-nutanix/controller:v0.3.2
quay.io/tigera/operator:v1.34.0
```


**Special notes for your reviewer**:
<!--
Use this to provide any additional information to the reviewers.
This may include:
- Best way to review the PR.
- Where the author wants the most review attention on.
- etc.
-->

Depends on #896.
dlipovetsky pushed a commit that referenced this pull request Sep 27, 2024
**What problem does this PR solve?**:
Adds a tool to fetch all images CAREN uses

**Which issue(s) this PR fixes**:
Fixes #

**How Has This Been Tested?**:
<!--
Please describe the tests that you ran to verify your changes.
Provide output from the tests and any manual steps needed to replicate
the tests.
-->
I ran the following command from `hack/tools/fetch-images`

```
$ go run main.go -chart-directory=../../../charts/cluster-api-runtime-extensions-nutanix/ --helm-chart-configmap=../../../charts/cluster-api-runtime-extensions-nutanix/templates/helm-config.yaml -caren-version=v0.13.1
ghcr.io/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix:v0.13.1
ghcr.io/nutanix-cloud-native/caren-helm-reg:v0.13.1
registry.k8s.io/sig-storage/snapshot-controller:v8.0.1
registry.k8s.io/provider-aws/cloud-controller-manager:v1.27.7
registry.k8s.io/provider-aws/cloud-controller-manager:v1.28.6
registry.k8s.io/provider-aws/cloud-controller-manager:v1.29.3
registry.k8s.io/provider-aws/cloud-controller-manager:v1.30.1
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/node-driver-registrar:v2.11.0-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
public.ecr.aws/ebs-csi-driver/aws-ebs-csi-driver:v1.33.0
public.ecr.aws/eks-distro/kubernetes-csi/external-provisioner:v5.0.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-attacher:v4.6.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/external-resizer:v1.11.1-eks-1-30-10
public.ecr.aws/eks-distro/kubernetes-csi/livenessprobe:v2.13.0-eks-1-30-10
registry.k8s.io/autoscaling/cluster-autoscaler:v1.30.0
quay.io/metallb/speaker:v0.14.8
quay.io/frrouting/frr:9.1.0
quay.io/metallb/speaker:v0.14.8
quay.io/metallb/controller:v0.14.8
registry.k8s.io/nfd/node-feature-discovery:v0.16.1-minimal
registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.9.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-provisioner:v3.6.3
registry.k8s.io/sig-storage/csi-attacher:v4.4.3
registry.k8s.io/sig-storage/csi-resizer:v1.9.3
registry.k8s.io/sig-storage/csi-snapshotter:v3.0.3
docker.io/nutanix/ntnx-csi:3.0.0
registry.k8s.io/sig-storage/livenessprobe:v2.11.0
registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.10.0
docker.io/nutanix/ntnx-csi-precheck:3.0.0
quay.io/cilium/cilium:v1.16.0
quay.io/cilium/cilium-envoy@sha256:bd5ff8c66716080028f414ec1cb4f7dc66f40d2fb5a009fff187f4a9b90b566b
quay.io/cilium/operator-generic:v1.16.0
quay.io/cilium/certgen:v0.2.0
docker.io/rancher/local-path-provisioner:v0.0.28
ghcr.io/nutanix-cloud-native/cloud-provider-nutanix/controller:v0.3.2
quay.io/tigera/operator:v1.34.0
```


**Special notes for your reviewer**:
<!--
Use this to provide any additional information to the reviewers.
This may include:
- Best way to review the PR.
- Where the author wants the most review attention on.
- etc.
-->

Depends on #896.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants