feat: use registryMirror addon as Containerd mirror

[dkoshkin]

What problem does this PR solve?:
Depends on #1116

Automatically sets the registryMirror as a Containerd mirror. We're not updating the Cluster object because this should not be a user controller configuration and the IP used as the mirror is determined based on the addon handler and the Service CIDRs.

Tested in a Docker cluster:

$ kubectl port-forward \
  --address=0.0.0.0 \
  --namespace registry-mirror-system \
  pod/registry-mirror-docker-registry-0 5000:5000
# Push an image tag that doesn't exist in dockerhub  
$ crane copy nginx:latest 0.0.0.0:5000/library/nginx:dkoshkin --insecure
$ kubectl run nginx-working --image=docker.io/library/nginx:dkoshkin
$ kubectl run nginx-should-be-broken --image=docker.io/library/nginx:dne
$ kubectl get pods 
NAME                                                              READY   STATUS              RESTARTS   AGE
cluster-autoscaler-0196931c-cb53-7abf-aa89-49c82c42ced5-86w5j8c   0/1     ContainerCreating   0          19m
nginx-should-be-broken                                            0/1     ErrImagePull        0          11m
nginx-working                                                     1/1     Running             0          11m

Which issue(s) this PR fixes:
Fixes #

How Has This Been Tested?:

Special notes for your reviewer:

The base branch was changed.

[github-actions]

This PR/issue depends on:

• feat: add registry addon #1116
  By Dependent Issues (🤖). Happy coding!

[supershal]

LGTM

[manoj-nutanix]

one q. if user removes registry config from cluster spec after deployment, will it generate new containerd patch without bootstrap registry config and apply it to live cluster?

[dkoshkin]

one q. if user removes registry config from cluster spec after deployment, will it generate new containerd patch without bootstrap registry config and apply it to live cluster?

Thats correct, let me create a ticket, maybe we should have a validation webhook here to prevent this. Not sure though because there are lots of other input we expect clients to understand what changing it means and nothing prevents it there.

[manoj-nutanix]

one q. if user removes registry config from cluster spec after deployment, will it generate new containerd patch without bootstrap registry config and apply it to live cluster?

Thats correct, let me create a ticket, maybe we should have a validation webhook here to prevent this. Not sure though because there are lots of other input we expect clients to understand what changing it means and nothing prevents it there.

Makes sense. Either we block users to not remove registry config from cluster spec or we should ensure that removal will cleanup everything and it becomes connected site cluster. It would be good to have integration/e2e test for this behavior.