Skip to content

e2e test should take GKE node version as an input #603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 3, 2020
+14 −0
Merged

e2e test should take GKE node version as an input #603

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions test/k8s-integration/cluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,10 @@ func clusterUpGKE(gceZone, gceRegion string, numNodes int, imageType string, use
cmdParams = append(cmdParams, "--enable-autorepair")
}

if isVariableSet(gkeNodeVersion) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should there be validation against specifying both release channel and node version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that's an error. I just ran

gcloud beta container clusters create --release-channel rapid --node-version 1.16.8-gke.15 skewed

and it succeeded. It might be useful to test such skewed deployments (although it looks like one might need to be careful about auto-upgrade---maybe that should be automatically disabled whenever both release channel and node version are specified? Or maybe we should just disable auto-upgrade all the time anyway as we probably never want that to happen during a test?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works if we specify release channel and node version. Master version picks the release version, while nodes pickup the node version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok thanks for checking. Yeah in hindsight if this were not the case, the skew tests we have planned with node version = master version - 2 would be tough.

Disabling auto-upgrade all the time makes sense to me. Removes the possible confusion during the rare occasion when a short-lived test cluster is running while an auto-upgrade is being pushed out.

Copy link
Member Author

@saikat-royc saikat-royc Sep 2, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see error when using release channels, and --no-enable-autoupgrade option.
e.g.

gcloud beta container clusters create gcp-pd-csi-driver-test-cluster --zone us-central1-c --num-nodes 3 --quiet --machine-type n1-standard-2 --release-channel rapid --enable-autorepair --no-enable-autoupgrade --node-version 1.16 --addons GcePersistentDiskCsiDriver
...
Auto_upgrade cannot be false when release_channel RAPID is set

however --no-enable-autoupgrade works with cluster-version and node-version option

gcloud beta container clusters create gcp-pd-csi-driver-test-cluster --zone us-central1-c --num-nodes 3 --quiet --machine-type n1-standard-2 --image-type cos --cluster-version latest --no-enable-autoupgrade --node-version 1.16 --addons GcePersistentDiskCsiDriver

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's defer the autoupgrade fiddling until we understand the use cases a bit better. It may be that it's not a problem for tests, we'd have to dig into the details more.

cmdParams = append(cmdParams, "--node-version", *gkeNodeVersion)
}

if useManagedDriver {
// PD CSI Driver add on is enabled only in gcloud beta.
cmdParams = append([]string{"beta"}, cmdParams...)
Expand Down
1 change: 1 addition & 0 deletions test/k8s-integration/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ var (
imageType = flag.String("image-type", "cos", "the image type to use for the cluster")
gkeReleaseChannel = flag.String("gke-release-channel", "", "GKE release channel to be used for cluster deploy. One of 'rapid', 'stable' or 'regular'")
gkeTestClusterName = flag.String("gke-cluster-name", "gcp-pd-csi-driver-test-cluster", "GKE cluster name")
gkeNodeVersion = flag.String("gke-node-version", "", "GKE cluster worker node version")

// Test infrastructure flags
boskosResourceType = flag.String("boskos-resource-type", "gce-project", "name of the boskos resource type to reserve")
Expand Down
5 changes: 5 additions & 0 deletions test/run-k8s-integration-ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ readonly image_type=${IMAGE_TYPE:-cos}
readonly use_gke_managed_driver=${USE_GKE_MANAGED_DRIVER:-false}
readonly gke_release_channel=${GKE_RELEASE_CHANNEL:-""}
readonly teardown_driver=${GCE_PD_TEARDOWN_DRIVER:-true}
readonly gke_node_version=${GKE_NODE_VERSION:-}

export GCE_PD_VERBOSITY=9

Expand Down Expand Up @@ -57,4 +58,8 @@ else
base_cmd="${base_cmd} --gce-region=${gce_region}"
fi

if [ -z "$gke_node_version" ]; then
base_cmd="${base_cmd} --gke-node-version=${gke_node_version}"
fi

eval "$base_cmd"
4 changes: 4 additions & 0 deletions test/run-k8s-integration.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ readonly image_type=${IMAGE_TYPE:-cos}
readonly use_gke_managed_driver=${USE_GKE_MANAGED_DRIVER:-false}
readonly gke_release_channel=${GKE_RELEASE_CHANNEL:-""}
readonly teardown_driver=${GCE_PD_TEARDOWN_DRIVER:-true}
readonly gke_node_version=${GKE_NODE_VERSION:-}

export GCE_PD_VERBOSITY=9

Expand Down Expand Up @@ -57,4 +58,7 @@ else
base_cmd="${base_cmd} --gce-region=${gce_region}"
fi

if [ -z "$gke_node_version" ]; then
base_cmd="${base_cmd} --gke-node-version=${gke_node_version}"
fi
eval "$base_cmd"