feat: adds a mindthegap container and deployment

[faiq]

What problem does this PR solve?:

This mindthegap container is going to serve the helm charts for any helm addons we have.

Which issue(s) this PR fixes:
https://jira.nutanix.com/browse/NCN-100596
https://jira.nutanix.com/browse/NCN-100597

How Has This Been Tested?:

Special notes for your reviewer:

[faiq]

01:23 PM  faiqus @ archlinux  ~/go/src/github.com/d2iq-labs/cluster-api-runtime-extensions-nutanix  ⏎ 130  faiq/adds-mindthegap-container⚡?+ 
$ kubectl get po
NAME                                                      READY   STATUS    RESTARTS   AGE
cluster-api-runtime-extensions-nutanix-68f76fb8cf-6zngc   1/1     Running   0          5m51s
mindthegap-6ff84f74cc-68x6d                               1/1     Running   0          21m

01:23 PM  faiqus @ archlinux  ~/go/src/github.com/d2iq-labs/cluster-api-runtime-extensions-nutanix  faiq/adds-mindthegap-container⚡?+ 
$ kubectl get cm ^C

01:23 PM  faiqus @ archlinux  ~/go/src/github.com/d2iq-labs/cluster-api-runtime-extensions-nutanix  ⏎ 130  faiq/adds-mindthegap-container⚡?+ 
$ kubectl get cm default-helm-addons-config -o yaml
apiVersion: v1
data:
  cilium: |
    ChartName: cilium
    ChartVersion: 1.15.0
    RepositoryURL: registry.default.svc
  cluster-autoscaler: |
    ChartName: cluster-autoscaler
    ChartVersion: 9.35.0
    RepositoryURL: registry.default.svc
  metallb: |
    ChartName: metallb
    ChartVersion: v0.14.5
    RepositoryURL: registry.default.svc
  nfd: |
    ChartName: node-feature-discovery
    ChartVersion: v0.15.2
    RepositoryURL: registry.default.svc
  nutanix-ccm: |
    ChartName: nutanix-cloud-provider
    ChartVersion: 0.3.3
    RepositoryURL: registry.default.svc
  nutanix-snapshot-csi: |
    ChartName: nutanix-csi-snapshot
    ChartVersion: 6.3.2
    RepositoryURL: registry.default.svc
  nutanix-storage-csi: |
    ChartName: nutanix-csi-storage
    ChartVersion: v3.0.0-beta.1912
    RepositoryURL: registry.default.svc
  tigera-operator: |
    ChartName: tigera-operator
    ChartVersion: v3.26.4
    RepositoryURL: registry.default.svc
kind: ConfigMap

[deepakm-ntnx]

Could you please document in the description, general workflow with and without mindthegap from customer perspective. need a writeup to undestand from newbie perspective

[faiq]

the workflow should be transparent to the user. this is an internal implementation detail of how helm charts are served

[supershal]

🎉 Thanks for solving very hard problem with airgap usecase.

[dkoshkin]

Great job getting this across the line. and thanks for taking in my previous suggestions!

[jimmidyson]

Could we combine this with helmRegistryImage please? So the config looks like:

inClusterHelmOCIRegistry:
  enabled: true
  image:
    repository: repository: ghcr.io/nutanix-cloud-native/caren-helm-reg
    tag: ""
    pullPolicy: IfNotPresent

[jimmidyson]

Simpler to just use os.WriteFile

Suggested change

	f, err = os.OpenFile(fullOutputfilePath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o666)
	if err != nil {
	log.Error(err, "failed to create file")
	}
	defer f.Close()
	_, err = bytes.NewBuffer(b).WriteTo(f)
	if err != nil {
	log.Error(err, "failed to write to file")
	}
	if err := os.WriteFile(fullOutputfilePath, b, 0o644); err != nil {
	log.Error(err, "failed to write mindthegap config file")
	}

[jimmidyson]

Let's add tolerations and node affinity (preferredDuringSchedulingIgnoredDuringExecution) to allow running on CP nodes but prefer to run on non-CP nodes.