Support for disk labels parameter

[katsew]

Problem

I use PVC and PV resources on GKE, and I also use PersistentDisk(PD) for ComputeEngine instances.
I usually filter PDs by labels when I delete it.
However, the PD which created by PV does not have labels for filtering.
This makes me difficult to filter to delete PDs.

Solution

Adding parameter to StorageClass and use it in this driver.

https://kubernetes.io/docs/concepts/storage/storage-classes/#gce-pd
e.g.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: slow
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-standard
  replication-type: none
  disk-labels: 'foo=123,bar=456' # Add this parameter to set labels

Additional Info

I read the related issue on external-provisioner, but in my use case, I think it's better to implement on this driver, since it depends on GCP PD.

Sample implementation:
https://github.com/katsew/gcp-compute-persistent-disk-csi-driver/tree/disk-labels-support/pkg

[katsew]

Sorry, I didn't read the doc.
I didn't realize that this is not an official repo.
Close this issue because it's not compatible with current GCP CSI.

[msau42]

Hi @katsew, regardless of the feature request for in-tree vs CSI driver, I would like to understand your use cases.

Do you want the labels on the K8s PersistentVolume object, or the GCP Persistent Disk object? Do you want to label per app, per-namespace, or per cluster?

[katsew]

@msau42

Thanks for the reaction!
I think there is a different use case on k8s PV and GCP PD for labeling.
My main goal of this issue needs GCP PD labeling, because the name of the PD created from PV is totally randomize and cannot see what the PD is for.

For GCP PD, I want per-app, per-cluster labels to determine which app creates a disk.
I usually create a StorageClass per app, so I can fulfill my requirement by adding parameters to a StorageClass object.
I filter PD by labels like gcloud compute disks list --filter="labels.cluster=dev-cluster AND NOT labels.app=foo ...".

For k8s PV, I want to cascade labels which PVC has, and it can be per-app, per-namespace labels.
In this case, I just want to run kubectl get pv -l app=foo,deployTo=namespace rather than kubectl get pv | grep part-of-pv-name.
It's useful for filtering PVs by app and their namespace.

[msau42]

Sorry for the late response. Yes, I think the request makes sense. For labeling GCP PD, we can add a new StorageClass parameter for it.

For the K8s labeling case, you can get the PV that a PVC uses from PVC.Spec.Name. Is that sufficient to search for PVs by labels?

/kind feature
/help

[katsew]

@msau42

Thanks for the response.

For the K8s labeling case, you can get the PV that a PVC uses from PVC.Spec.Name. Is that sufficient to search for PVs by labels?

This could help finding PVs by labels, but could not filtering these.

kubectl get pvc -l app=foo,deployTo=namespace -o jsonpath="{range .items[*]}{.spec.volumeName}{'\n'}{end}"

I think adding labels from PVCs to PVs gives more sufficient way to filter PVs by labels.

kubectl get pv -l app=foo,deployTo=namespace -o jsonpath='{range .items[?(@.spec.persistentVolumeReclaimPolicy=="Delete")]}{.metadata.name}{"\n"}{end}'

[teralype]

I also think that propagating Kubernetes labels from the PVC to PV and from the PV to the actual GCE PD would increase discoverability. At the moment, now I'm force to create GCE PDs myself, then a Kubernetes PV, which is far from ideal and does not work well with Pod auto-scaling.

[teralype]

I also think that propagating Kubernetes labels from the PVC to PV and from the PV to the actual GCE PD would increase discoverability. At the moment, now I'm force to create GCE PDs myself, then a Kubernetes PV, which is far from ideal and does not work well with Pod auto-scaling.

[msau42]

Putting here as fyi, GKE has a feature to label all GCP resources on a per cluster basis. This may help partially with your use case. https://cloud.google.com/kubernetes-engine/docs/how-to/creating-managing-labels

We are also considering adding some k8s metadata like PVC name and namespace to the disk. #387

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[davidz627]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[msau42]

/lifecycle frozen

[ernestas2k]

Any comments on the status?

We're in need to put custom labels for special identification purposes (matters not related to kubernetes).

Looking into other drivers:

1. aws drivers support tags via startup args.
2. azure drivers support tags via request.Parameters["tags"] = "key1=val1,key2=val2" and seems to be comfortable to use.

Should we start working on request.Parameters["labels"] = "key1=val1,key2=val2" that translates into GCE Disk labels?

We also consider forking the project for this improvement to get it available ASAP.

[msau42]

Adding a storageclass parameter seems reasonable to me. Contributions are greatly appreciated!

[ernestas2k]

I have opened PR to address this ticket, please take a look.

[mattcary]

I've taken over #469 and will submit a new PR shortly.

[mattcary]

/remove-help