[pvc] potential workaround for mount volume failure #13594
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
started the job as gitpod-build-pavel-13353.1 because the annotations in the pull request description changed |
Furisto
approved these changes
Oct 4, 2022
jenting
reviewed
Oct 4, 2022
| // if we failed to mount volume, we need to re-create the pod | ||
| // ref: https://github.com/gitpod-io/gitpod/issues/13353 | ||
| // ref: https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/issues/608 | ||
| if strings.Contains(pod.Status.Reason, "MountVolume.MountDevice failed for volume") { |
There was a problem hiding this comment.
Could we have a log when we-manager detects the issue happens? So we could know the number of times we hit this issue.
There was a problem hiding this comment.
We already have it. On line 361.
There was a problem hiding this comment.
Since we haven't reproduced it yet, and also we did not dump the pod YAML manifest when the last time Toru was able to reproduce. I worried that the ws-manager did not recreate the pod because one of the conditions mismatch
gitpod/components/ws-manager/pkg/manager/manager.go
Lines 703 to 707 in e3d9622
roboquat
added
deployed: workspace
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Potential workaround for random mount volume failures.
Related Issue(s)
Fixes #13353
How to test
It is very hard to repro this issue.
So another way to test this PR is to ensure opening workspace works.
Release Notes
Documentation
Werft options:
If enabled this will build
install/previewValid options are
all,workspace,webapp,ide