Skip to content

ReadWriteMany not working for Hyperdisk Balanced HA #2101

Open
@kastriotdobratiqi

Description

@kastriotdobratiqi

I am trying to set up a Hyperdisk Balanced HA persistent volume by following the docs and it is not working. Created the following resources:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: balanced-ha-storage
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
  type: hyperdisk-balanced-high-availability
  provisioned-throughput-on-create: "250Mi"
  provisioned-iops-on-create: "7000"
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: podpvc
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: balanced-ha-storage
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-server-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        volumeMounts:
        - mountPath: /var/lib/www/html
          name: mypvc
      volumes:
      - name: mypvc
        persistentVolumeClaim:
          claimName: podpvc
          readOnly: false

This results on the following:

Name:          podpvc
Namespace:     default
StorageClass:  balanced-ha-storage
Status:        Pending
Volume:        
Labels:        <none>
Annotations:   volume.beta.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
               volume.kubernetes.io/selected-node: selected.node
               volume.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       web-server-deployment-7df4f6469-b7g65
Events:
  Type     Reason                Age                From                                                                                              Message
  ----     ------                ----               ----                                                                                              -------
  Normal   WaitForFirstConsumer  95s (x2 over 99s)  persistentvolume-controller                                                                       waiting for first consumer to be created before binding
  Normal   WaitForPodScheduled   65s (x2 over 80s)  persistentvolume-controller                                                                       waiting for pod web-server-deployment-7df4f6469-b7g65 to be scheduled
  Normal   Provisioning          28s (x6 over 59s)  pd.csi.storage.gke.io_gke-40e91c2458b04c4b82aa-1635-2733-vm_4cc477aa-e2d0-4001-b565-1b9c05869d3a  External provisioner is provisioning volume for claim "default/podpvc"
  Warning  ProvisioningFailed    28s (x3 over 52s)  pd.csi.storage.gke.io_gke-40e91c2458b04c4b82aa-1635-2733-vm_4cc477aa-e2d0-4001-b565-1b9c05869d3a  failed to provision volume with StorageClass "balanced-ha-storage": rpc error: code = InvalidArgument desc = VolumeCapabilities is invalid: specified multi writer with mount access type
  Normal   ExternalProvisioning  5s (x5 over 59s)   persistentvolume-controller                                                                       Waiting for a volume to be created either by the external provisioner 'pd.csi.storage.gke.io' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.

This is where it is failing:

if mnt != nil && mod == csi.VolumeCapability_AccessMode_MULTI_NODE_MULTI_WRITER {

Mounting this seems to not be working. Is that by design?

I can make the provisioning work if I treat the volume as a raw block by adding volumeMode: Block and then instead of mounting it as a FileSystem use:

        volumeDevices:
        - devicePath: /var/lib/www/html
          name: mypvc

But again is this how it is supposed to work or is it a bug, given this works on ReadWriteOnce access mode.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions