podpresets: Create a pod preset to automount hostpaths and set HOME for tools

This adds one pod preset to each new tool namespace. One will use the label
selector "toolforge: tool" to be an
easy to remember and use label for custom manifests where people use them
and for webservice to use.

Anything in a tool namespace that uses the label will have
this spec injected into pods:
  env:
    - name: HOME
      value: /data/project/<toolname>
  volumeMounts:
    - mountPath: /public/dumps/
      name: dumps
      readOnly: true
    - mountPath: /data/project/
      name: home
    - mountPath: /etc/wmcs-project
      name: wmcs-project
      readOnly: true
    - mountPath: /data/scratch/
      name: scratch
    - mountPath: /etc/ldap.conf
      readOnly: true
      name: etcldap-conf
    - mountPath: /etc/ldap.yaml
      name: etcldap-yaml
      readOnly: true
    - mountPath: /etc/novaobserver.yaml
      name: etcnovaobserver-yaml
      readOnly: true
    - mountPath: /var/lib/sss/pipes
      name: sssd-pipes
  volumes:
    - hostPath:
        path: /public/dumps
        type: Directory
      name: dumps
    - hostPath:
        path: /data/project
        type: Directory
      name: home
    - hostPath:
        path: /etc/wmcs-project
        type: File
      name: wmcs-project
    - hostPath:
        path: /data/scratch
        type: Directory
      name: scratch
    - hostPath:
        path: /etc/ldap.conf
        type: File
      name: etcldap-conf
    - hostPath:
        path: /etc/ldap.yaml
        type: File
      name: etcldap-yaml
    - hostPath:
        path: /etc/novaobserver.yaml
        type: File
      name: etcnovaobserver-yaml
    - hostPath:
        path: /var/lib/sss/pipes
        type: Directory
      name: sssd-pipes

Bug: T215678
Change-Id: Icf264d22a9e544d35d675f4fb5424da13eb085eb

Author: bstorm

README.md

@@ -58,8 +58,7 @@ The steps are below:
 4. Fix that until it works, if it didn't.
 5. Run `vagrant forward-port 1389 389` to expose the vagrant VMs LDAP to the
    host.
-6. Start minikube with `minikube start
-   --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy`. It
+6. Start minikube with `minikube start --kubernetes-version=1.15.5 --extra-config=apiserver.runtime-config=settings.k8s.io/v1alpha1=true --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy,PodPreset`. It
    will fail to finish initializing because PodSecurityPolicy complicates
    everything until the next step.
 7. Run `kubectl apply -f <path to
@@ -92,13 +91,13 @@ The steps are below:
     pod! After this, things become a bit more familiar in terms of python
     testing.
 17. Run `source venv/bin/activate`
-18. Start recording tests! Delete the cassettes in the pod shell with `rm tests/cassettes` just to make sure you have a clean slate and run `pytest --in-k8s`.  This will
+18. Start recording tests! Delete the cassettes in the pod shell with `rm tests/cassettes/*` just to make sure you have a clean slate and run `pytest --in-k8s`.  This will
     **fail** on one of the API tests.  The reason is that this doesn't have an
     excellent teardown when actually running against an API server just yet.
     It should have only failed on a single test.
 19. In another terminal on your local machine run `kubectl delete ns
     tool-blurp` to clean up what is upsetting that last test.
-20. In your kubernetes pod terminal run `rm tests/cassettes/test_tool_renewal`. Now record only that test as a VCR cassette with `pytest --in-k8s -k "test_tool_renewal"`.  If that succeeded, you have
+20. In your kubernetes pod terminal run `rm tests/cassettes/test_tool_renewal.yaml`. Now record only that test as a VCR cassette with `pytest --in-k8s -k "test_tool_renewal"`.  If that succeeded, you have
     a good set of mocks ("cassettes") to run later.
 21. You now need to get those cassettes from the pod to your host and into the
     git repository. There are several ways to do that. The easy and reliable way is to copy them all to `/data/project` inside the pod like `cp -r tests/cassettes /data/project/` to get them on the minikube VM.  Then, log out of your pod terminal (since that should all be done if all your tests passed), delete the cassettes in your active repo (`rm tests/cassettes/*`), and replace them from the minikube vm with `scp -i $(minikube ssh-key) docker@$(minikube ip):/data/project/cassettes/* tests/cassettes/`

betaservice.yaml

@@ -29,6 +29,17 @@ rules:
   - patch
   - update
   - watch
+- apiGroups:
+  - settings.k8s.io
+  resources:
+  - podpresets
+  verbs:
+  - create
+  - get
+  - list
+  - patch
+  - update
+  - watch
 - apiGroups:
   - extensions
   resources:

maintain_kubeusers/maintain_kubeusers.py

@@ -63,6 +63,7 @@ def __init__(self):
         self.certs = client.CertificatesV1beta1Api()
         self.rbac = client.RbacAuthorizationV1Api()
         self.extensions = client.ExtensionsV1beta1Api()
+        self.settings_api = client.SettingsV1alpha1Api()
 
     def get_cluster_info(self):
         c_info = self.core.read_namespaced_config_map(
@@ -195,6 +196,129 @@ def approve_cert(self, user):
         else:
             logging.error("Certificate creation stalled or failed for %s", user)
 
+    def create_presets(self, user):
+        try:
+            _ = self.settings_api.create_namespaced_pod_preset(
+                namespace="tool-{}".format(user),
+                body=client.V1alpha1PodPreset(
+                    api_version="settings.k8s.io/v1alpha1",
+                    kind="PodPreset",
+                    metadata=client.V1ObjectMeta(name="mount-toolforge-vols"),
+                    spec=client.V1alpha1PodPresetSpec(
+                        selector=client.V1LabelSelector(
+                            match_labels={
+                                "toolforge": "tool"
+                            }
+                        ),
+                        env=[
+                            client.V1EnvVar(
+                                name="HOME",
+                                value="/data/project/{}".format(user)
+                            ),
+                        ],
+                        volumes=[
+                            client.V1Volume(
+                                name="dumps",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/public/dumps", type="Directory"
+                                ),
+                            ),
+                            client.V1Volume(
+                                name="home",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/data/project", type="Directory"
+                                ),
+                            ),
+                            client.V1Volume(
+                                name="wmcs-project",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/etc/wmcs-project", type="File"
+                                ),
+                            ),
+                            client.V1Volume(
+                                name="scratch",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/data/scratch", type="Directory"
+                                ),
+                            ),
+                            client.V1Volume(
+                                name="etcldap-conf",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/etc/ldap.conf", type="File"
+                                ),
+                            ),
+                            client.V1Volume(
+                                name="etcldap-yaml",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/etc/ldap.yaml", type="File"
+                                ),
+                            ),
+                            client.V1Volume(
+                                name="etcnovaobserver-yaml",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/etc/novaobserver.yaml", type="File"
+                                ),
+                            ),
+                            client.V1Volume(
+                                name="sssd-pipes",
+                                host_path=client.V1HostPathVolumeSource(
+                                    path="/var/lib/sss/pipes", type="Directory"
+                                ),
+                            ),
+                        ],
+                        volume_mounts=[
+                            client.V1VolumeMount(
+                                name="dumps",
+                                mount_path="/public/dumps",
+                                read_only=True,
+                            ),
+                            client.V1VolumeMount(
+                                name="home", mount_path="/data/project"
+                            ),
+                            client.V1VolumeMount(
+                                name="wmcs-project",
+                                mount_path="/etc/wmcs-project",
+                                read_only=True,
+                            ),
+                            client.V1VolumeMount(
+                                name="scratch", mount_path="/data/scratch"
+                            ),
+                            client.V1VolumeMount(
+                                name="etcldap-conf",
+                                mount_path="/etc/ldap.conf",
+                                read_only=True,
+                            ),
+                            client.V1VolumeMount(
+                                name="etcldap-yaml",
+                                mount_path="/etc/ldap.yaml",
+                                read_only=True,
+                            ),
+                            client.V1VolumeMount(
+                                name="etcnovaobserver-yaml",
+                                mount_path="/etc/novaobserver.yaml",
+                                read_only=True,
+                            ),
+                            client.V1VolumeMount(
+                                name="sssd-pipes",
+                                mount_path="/var/lib/sss/pipes",
+                            ),
+                        ],
+                    ),
+                ),
+            )
+        except ApiException as api_ex:
+            if api_ex.status == 409 and "AlreadyExists" in api_ex.body:
+                logging.info(
+                    "PodPreset mount-toolforge-vols in tool-%s already exists",
+                    user,
+                )
+                return
+
+            logging.error(
+                "Could not create PodPreset mount-toolforge-vols for %s", user
+            )
+            raise
+
     def create_namespace(self, user):
         """
         Creates a namespace for the given user if it doesn't exist
@@ -520,6 +644,7 @@ def process_rbac(self, user):
     def add_user_access(self, user):
         self.generate_psp(user)
         self.create_namespace(user.name)
+        self.create_presets(user.name)
         self.process_rbac(user.name)
         self.create_configmap(user)
 

service.yaml

@@ -29,6 +29,17 @@ rules:
   - patch
   - update
   - watch
+- apiGroups:
+  - settings.k8s.io
+  resources:
+  - podpresets
+  verbs:
+  - create
+  - get
+  - list
+  - patch
+  - update
+  - watch
 - apiGroups:
   - extensions
   resources: