Skip to content

Fix Random CMEK Disk Creation Failure (disk already exists) #563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 28, 2020
Merged

Fix Random CMEK Disk Creation Failure (disk already exists) #563

merged 1 commit into from
Jul 28, 2020

Conversation

saad-ali
Copy link
Contributor

What type of PR is this?

/kind bug

What this PR does / why we need it:
This PR fixes the issue where provisioning of GCE PDs with CMEK used to sometimes fails with disk already exists with same name error.

Which issue(s) this PR fixes:

Fixes #558

Special notes for your reviewer:

Added unit test to prevent future regression.

Manually verified by setting csi-provisioner timeout to 1 seconds:

  Type     Reason                 Age                From                                                                                                 Message
  ----     ------                 ----               ----                                                                                                 -------
  Normal   ExternalProvisioning   32s (x2 over 34s)  persistentvolume-controller                                                                          waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
  Warning  ProvisioningFailed     32s (x2 over 33s)  pd.csi.storage.gke.io_gke-cluster-1-default-pool-4cede575-43h6_5ca3c339-e79e-4f83-acc5-97267112065f  failed to provision volume with StorageClass "csi-gce-pd-cmek": rpc error: code = DeadlineExceeded desc = context deadline exceeded
  Warning  ProvisioningFailed     30s (x2 over 32s)  pd.csi.storage.gke.io_gke-cluster-1-default-pool-4cede575-43h6_5ca3c339-e79e-4f83-acc5-97267112065f  failed to provision volume with StorageClass "csi-gce-pd-cmek": rpc error: code = Aborted desc = An operation with the given Volume ID projects/test-project/zones/us-central1-c/disks/pvc-0a3d4780-ce27-11ea-b569-42010a8000c9 already exists
  Normal   Provisioning           22s (x5 over 34s)  pd.csi.storage.gke.io_gke-cluster-1-default-pool-4cede575-43h6_5ca3c339-e79e-4f83-acc5-97267112065f  External provisioner is provisioning volume for claim "default/pvc-demo"
  Normal   ProvisioningSucceeded  21s                pd.csi.storage.gke.io_gke-cluster-1-default-pool-4cede575-43h6_5ca3c339-e79e-4f83-acc5-97267112065f  Successfully provisioned volume pvc-0a3d4780-ce27-11ea-b569-42010a8000c9

Does this PR introduce a user-facing change?:

Fixed issue where provisioning of GCE PDs with CMEK used to sometimes fails with `disk already exists with same name` error.

/assign @msau42

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. labels Jul 25, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: saad-ali

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested review from jingxu97 and verult July 25, 2020 03:36
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 25, 2020
msau42
msau42 reviewed Jul 27, 2020
View reviewed changes
@saad-ali
Copy link
Contributor Author

PTAL

msau42
msau42 reviewed Jul 28, 2020
View reviewed changes
Copy link
Contributor

@msau42 msau42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 28, 2020
@saad-ali
Copy link
Contributor Author

/test pull-gcp-compute-persistent-disk-csi-driver-kubernetes-integration

@k8s-ci-robot k8s-ci-robot merged commit f476c98 into kubernetes-sigs:master Jul 28, 2020
@msau42 msau42 mentioned this pull request Aug 5, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msau42 msau42 msau42 left review comments

@jingxu97 jingxu97 Awaiting requested review from jingxu97

@verult verult Awaiting requested review from verult

Assignees

@msau42 msau42

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CMEK Disk Creation Sometimes Fails with "disk already exists with same name"
3 participants
@saad-ali @k8s-ci-robot @msau42