fix: issues with mindthegap deplotment (#660)

**What problem does this PR solve?**:

This adds trust-manager to allow us to use self hosted helm charts in
mindthegap. See:
kubernetes-sigs/cluster-api-addon-provider-helm#104

We can think about this in a couple of different parts:

1. Installing trust-manager and CRDs. We need to install trust manager
so that we can create a certificate bundle with our self signed cert to
caaph-controller-manager. We pull the CRDs separately because we are
unable to use them in our templates because they aren't available at
that time. See the comment in the script for more details.

2. Modifying CAAPH to use the bundle. This is done through a job it has
a corresponding Service Account/Role/Role Binding.

3. Various little fixes. ie changing repository url and versions to
using `v`

I tested this by creating a docker cluster.

The helm chart proxies use this value

```yaml
- apiVersion: addons.cluster.x-k8s.io/v1alpha1
  kind: HelmChartProxy
  metadata:
    creationTimestamp: "2024-05-14T23:41:09Z"
    finalizers:
    - helmchartproxy.addons.cluster.x-k8s.io
    generation: 1
    name: node-feature-discovery-docker-cluster-cilium-helm-addon
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: Cluster
      name: docker-cluster-cilium-helm-addon
      uid: e0e943e4-6d63-4d85-8d37-f63160ff93e3
    resourceVersion: "3282"
    uid: e04b6b62-5f39-4528-87f7-5928d2afd4dd
  spec:
    chartName: node-feature-discovery
    clusterSelector:
      matchLabels:
        cluster.x-k8s.io/cluster-name: docker-cluster-cilium-helm-addon
    namespace: node-feature-discovery
    options:
      enableClientCache: false
      install:
        createNamespace: true
      timeout: 10m0s
      upgrade:
        maxHistory: 10
    releaseName: node-feature-discovery
    repoURL: oci://mindthegap.mindthegap.svc/charts
```

**Which issue(s) this PR fixes**:
Fixes #

**How Has This Been Tested?**:
<!--
Please describe the tests that you ran to verify your changes.
Provide output from the tests and any manual steps needed to replicate
the tests.
-->

**Special notes for your reviewer**:
<!--
Use this to provide any additional information to the reviewers.
This may include:
- Best way to review the PR.
- Where the author wants the most review attention on.
- etc.
-->

Author: faiq

charts/cluster-api-runtime-extensions-nutanix/templates/helm-config.yaml

@@ -10,39 +10,39 @@ data:
   cilium: |
     ChartName: cilium
     ChartVersion: 1.15.0
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
   cluster-autoscaler: |
     ChartName: cluster-autoscaler
     ChartVersion: 9.37.0
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
   local-path-provisioner-csi: |
     ChartName: local-path-provisioner
     ChartVersion: v0.0.29
     RepositoryURL: https://charts.containeroo.ch
   metallb: |
     ChartName: metallb
     ChartVersion: v0.14.5
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
   nfd: |
     ChartName: node-feature-discovery
-    ChartVersion: v0.15.2
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    ChartVersion: 0.15.2
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
   nutanix-ccm: |
     ChartName: nutanix-cloud-provider
     ChartVersion: 0.3.3
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
   nutanix-snapshot-csi: |
     ChartName: nutanix-csi-snapshot
     ChartVersion: 6.3.2
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
   nutanix-storage-csi: |
     ChartName: nutanix-csi-storage
-    ChartVersion: v3.0.0-beta.1912
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    ChartVersion: 3.0.0-beta.1912
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
   tigera-operator: |
     ChartName: tigera-operator
     ChartVersion: v3.28.0
-    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc
+    RepositoryURL: oci://mindthegap.{{ .Release.Namespace }}.svc/charts
 kind: ConfigMap
 metadata:
   creationTimestamp: null

charts/cluster-api-runtime-extensions-nutanix/templates/helm-chart-svc.yaml renamed to charts/cluster-api-runtime-extensions-nutanix/templates/mindthegap.yaml

@@ -1,6 +1,8 @@
-# Copyright 2023 D2iQ, Inc. All rights reserved.
+# Copyright 2024 Nutanix. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
-
+# This file contains the manifests to run a mindthegap deployment which contains helm charts for our addons.
+# The pod is built via goreleaser with configuration from hack/addons.
+#
 apiVersion: cert-manager.io/v1
 kind: Issuer
 metadata:
@@ -12,12 +14,12 @@ spec:
 apiVersion: cert-manager.io/v1
 kind: Certificate
 metadata:
-  name: mindthegap-example-tls
+  name: mindthegap-tls
   namespace: {{ .Release.Namespace }}
 spec:
   dnsNames:
-  - mindthegap.default.svc
-  - mindthegap.default.svc.cluster.local
+  - mindthegap.{{ .Release.Namespace }}.svc
+  - mindthegap.{{ .Release.Namespace }}.svc.cluster.local
   issuerRef:
     name: mindthegap-selfsigned
     kind: Issuer