Skip to content

change GetDisk error reporting to temporary in CreateVolume codepath #1558

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 11, 2024
Merged

change GetDisk error reporting to temporary in CreateVolume codepath #1558

merged 1 commit into from
Jan 11, 2024

Conversation

leiyiz
Copy link
Contributor

@leiyiz leiyiz commented Jan 2, 2024 •
edited
Loading

What type of PR is this?

/kind bug

What this PR does / why we need it:

When we do CreateVolume, the csi provisioner expects either "final" or "temporary" errors, if we report final error when disk creation could be ongoing or the disk is already created, then a PVC deletion before successful volume creation could lead to disk leakage as DeleteVolume won't happen.

The temporary errors are defined here: https://github.com/kubernetes-csi/external-provisioner/blob/master/pkg/controller/controller.go#L1904

This PR changes all the GCE API calls in CreateVolume code path that potentially happens AFTER insertDisk API calls happened to report a non-final error.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 2, 2024
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. release-note-none Denotes a PR that doesn't merit a release note. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jan 2, 2024
@leiyiz leiyiz requested review from msau42 and mattcary and removed request for tyuchn and saikat-royc January 2, 2024 19:47
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jan 2, 2024
@leiyiz
Copy link
Contributor Author

leiyiz commented Jan 5, 2024

@mattcary addressed the comments, can you take another look

mattcary
mattcary reviewed Jan 10, 2024
View reviewed changes
pkg/gce-pd-csi-driver/controller.go
@@ -324,7 +324,8 @@ func (gceCS *GCEControllerServer) CreateVolume(ctx context.Context, req *csi.Cre
existingDisk, err := gceCS.CloudProvider.GetDisk(ctx, gceCS.CloudProvider.GetDefaultProject(), volKey, gceAPIVersion)
if err != nil {
if !gce.IsGCEError(err, "notFound") {
return nil, common.LoggedError("CreateVolume, failed to getDisk when validating: ", err)
// failed to GetDisk, however the Disk may already be created, the error code should be non-Final
return nil, common.LoggedError("CreateVolume, failed to getDisk when validating: ", status.Error(codes.Unavailable, err.Error()))
}
}
if err == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I hate that you can't comment on lines outside the PR...)

On 345 there's a !ready check that returns an internal error code --- we should probably turn this into a temporary error?

You know, I think this is going to screw up our slo filtering :-/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and on line 423

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(offline discussion: our slo filtering will be fine)

@mattcary
Copy link
Contributor

/lgtm
/approve

thanks!

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 11, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: leiyiz, mattcary

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit f78e4ff into kubernetes-sigs:master Jan 11, 2024
@mattcary
Copy link
Contributor

/cherry-pick release-1.12

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Feb 12, 2024
Merged
@k8s-infra-cherrypick-robot
Copy link

@mattcary: new pull request created: #1600

In response to this:

/cherry-pick release-1.12

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mattcary
Copy link
Contributor

/cherry-pick release-1.11

@mattcary
Copy link
Contributor

/cherry-pick release-1.10

@k8s-infra-cherrypick-robot
Copy link

@mattcary: new pull request created: #1601

In response to this:

/cherry-pick release-1.11

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

This was referenced Feb 12, 2024
Merged
Merged
@k8s-infra-cherrypick-robot
Copy link

@mattcary: new pull request created: #1602

In response to this:

/cherry-pick release-1.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mattcary
Copy link
Contributor

/cherry-pick release-1.9

@mattcary
Copy link
Contributor

/cherry-pick release-1.8

@k8s-infra-cherrypick-robot
Copy link

@mattcary: new pull request created: #1603

In response to this:

/cherry-pick release-1.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-infra-cherrypick-robot k8s-infra-cherrypick-robot mentioned this pull request Feb 12, 2024
Merged
@k8s-infra-cherrypick-robot
Copy link

@mattcary: #1558 failed to apply on top of branch "release-1.8":

Applying: change GetDisk error reporting to temporary in CreateVolume codepath
Using index info to reconstruct a base tree...
M	pkg/gce-cloud-provider/compute/gce-compute.go
M	pkg/gce-pd-csi-driver/controller.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/gce-pd-csi-driver/controller.go
CONFLICT (content): Merge conflict in pkg/gce-pd-csi-driver/controller.go
Auto-merging pkg/gce-cloud-provider/compute/gce-compute.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 change GetDisk error reporting to temporary in CreateVolume codepath
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherry-pick release-1.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mattcary added a commit that referenced this pull request Feb 13, 2024
@mattcary
Merge pull request #1604 from mattcary/orphan-18
d67a539 
Cherry-pick #1558 to release-1.8
@pwschuurman pwschuurman mentioned this pull request Mar 27, 2024
Merged
@amacaskill amacaskill mentioned this pull request Apr 8, 2024
Merged
@riteshghorse riteshghorse mentioned this pull request Aug 7, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattcary mattcary mattcary left review comments

@msau42 msau42 Awaiting requested review from msau42

Assignees

@mattcary mattcary

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@leiyiz @mattcary @k8s-ci-robot @k8s-infra-cherrypick-robot