Add windows driver installation support #510
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,6 +3,5 @@ commonLabels: | |
| namespace: | ||
| gce-pd-csi-driver | ||
| resources: | ||
| - controller.yaml | ||
| - setup-cluster.yaml | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| apiVersion: policy/v1beta1 | ||
| kind: PodSecurityPolicy | ||
| metadata: | ||
| name: csi-gce-pd-node-psp | ||
| spec: | ||
| allowedHostPaths: | ||
| - pathPrefix: "/var/lib/kubelet/plugins_registry/" | ||
| - pathPrefix: "/var/lib/kubelet" | ||
| - pathPrefix: "/var/lib/kubelet/plugins/pd.csi.storage.gke.io/" | ||
| - pathPrefix: "/dev" | ||
| - pathPrefix: "/etc/udev" | ||
| - pathPrefix: "/lib/udev" | ||
| - pathPrefix: "/run/udev" | ||
| - pathPrefix: "/sys" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| kind: DaemonSet | ||
|
There was a problem hiding this comment. this file can be removed now right? |
||
| apiVersion: apps/v1 | ||
| metadata: | ||
| name: csi-gce-pd-node | ||
| spec: | ||
| template: | ||
| spec: | ||
| hostNetwork: true | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
| bases: | ||
| - ../../../base | ||
| namespace: | ||
| gce-pd-csi-driver | ||
| resources: | ||
| - node.yaml | ||
| patchesStrategicMerge: | ||
| - allowedHostPaths.yaml | ||
| images: | ||
| - name: gke.gcr.io/gcp-compute-persistent-disk-csi-driver | ||
| # Don't change stable image without changing pdImagePlaceholder in | ||
| # test/k8s-integration/main.go | ||
| newName: gke.gcr.io/gcp-compute-persistent-disk-csi-driver | ||
| newTag: "v0.7.0-gke.0" | ||
| - name: gke.gcr.io/csi-provisioner | ||
| newName: gke.gcr.io/csi-provisioner | ||
| newTag: "v1.5.0-gke.0" | ||
| - name: gke.gcr.io/csi-attacher | ||
| newName: gke.gcr.io/csi-attacher | ||
| newTag: "v2.1.1-gke.0" | ||
| - name: gke.gcr.io/csi-node-driver-registrar | ||
| newName: gke.gcr.io/csi-node-driver-registrar | ||
| newTag: "v1.2.0-gke.0" | ||
| - name: gke.gcr.io/csi-resizer | ||
| newName: gke.gcr.io/csi-resizer | ||
| newTag: "v0.4.0-gke.0" | ||
| - name: gke.gcr.io/csi-snapshotter | ||
| newName: gke.gcr.io/csi-snapshotter | ||
| newTag: "v2.1.1-gke.0" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| kind: DaemonSet | ||
|
There was a problem hiding this comment. this file can be removed now right? |
||
| apiVersion: apps/v1 | ||
| metadata: | ||
| name: csi-gce-pd-node | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - name: csi-driver-registrar | ||
| args: | ||
| - "--v=5" | ||
| - "--csi-address=/csi/csi.sock" | ||
| - "--kubelet-registration-path=/var/lib/kubelet/plugins/pd.csi.storage.gke.io/csi.sock" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| apiVersion: policy/v1beta1 | ||
| kind: PodSecurityPolicy | ||
| metadata: | ||
| name: csi-gce-pd-node-psp | ||
| spec: | ||
| allowedHostPaths: | ||
| - pathPrefix: \var\lib\kubelet | ||
|
There was a problem hiding this comment. I think this is going to replace the current PSP with these rules, instead of add, which is not desirable if we're going to be running both linux + windows. There was a problem hiding this comment. good point, will try to make a merge here. There was a problem hiding this comment. actually, I think it is better to have separate policy for linux and windows instead of merging |
||
| - pathPrefix: \var\lib\kubelet\plugins_registry | ||
| - pathPrefix: \var\lib\kubelet\plugins\pd.csi.storage.gke.io | ||
| - pathPrefix: \\.\pipe\csi-proxy-disk-v1alpha1 | ||
| - pathPrefix: \\.\pipe\csi-proxy-volume-v1alpha1 | ||
| - pathPrefix: \\.\pipe\csi-proxy-filesystem-v1alpha1 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
|
There was a problem hiding this comment. Are you going to need multiple overlays for windows too (ie staging, stable, dev)? The challenge I see with this structure is that we now have 2 places that we need to keep in sync when we update overlays. For example, if we make a update to staging, then we need to update both windows and linux staging overlays. There was a problem hiding this comment. So with one overlay, we have both windows and linux together. What if user need to deploy with mixed OS and versions, e.g, linux stable, and windows dev etc? Or just because some versions are only available in linux, not windows? There was a problem hiding this comment. I think we'll still need to have different overlays for dev/alpha/staging/stable and we can control the versions used in each. For example, the dev overlay could use stable versions for linux, and unstable versions for windows. And in terms of development flow, we can add Windows first to the dev overlay, then once we're ready, add it to staging, and then when we're ready to release, add it to stable. There was a problem hiding this comment. Maybe I miss something here, so you think user will not need a mixed version for windows and linux. If they want to deploy stable version, it will be stable version for both linux and windows? But what if windows don't have stable version yet, user must deploy dev version for both linux and windows, instead of one stable version and one dev version? There was a problem hiding this comment. Also in the case user already deploy a Linux driver, and want to add a windows driver (might be a different version from Linux), with one overlay, it will try to deploy both linux and windows. Some setup might be messed up with different versions of deployment? In the current layout, we have common things for both linux and windows in the base, and different things in overlay. So new changes, either common to linux and windows, which will go to base, or different from linux and windows, and we need to update them wherever applicable. So I don't see there is more work needed compared to one overlay which contains both linux and windows. Also there might be cases that some setup is only applicable for linux, and we might accidentally break windows setup if we don't pay attention to the windows part? There was a problem hiding this comment. I have been thinking the structure again last night :), and realized this is more like a multi-stage process. So instead of base/overlay structure, I propose to use the following as multiple stages and use them as bases. I am working on this structure now and will update. Please let me know what you think about this structure.
There was a problem hiding this comment. I'm not quite sure how this would work out but am curious to see. One thing to note is that alpha/staging/stable is not necessarily just different versions. Different features may be added depending on their maturity.
In terms of looking at the driver itself, I think we should consider both Linux + Windows parts of the driver to be part of one complete driver instead of 2 independent drivers. So considering that Windows is still alpha right now, I think it is fine if it uses the Linux alpha versions as well (which currently don't have any difference from stable). I don't think a user will want to have a mixed Linux + Windows cluster where the Linux part is stable and the Windows part is alpha. I would expect a user to have 2 separate clusters: a Linux-only cluster used for production, and an experimental Windows-only cluster.
I agree with one deployment and one place to define all the versions, it's easier to make sure the versions are in-sync with each other. |
||
| kind: Kustomization | ||
| bases: | ||
| - ../../../base | ||
| namespace: | ||
| gce-pd-csi-driver | ||
| resources: | ||
| - node.yaml | ||
| patchesStrategicMerge: | ||
| - allowedHostPaths.yaml | ||
| images: | ||
| - name: gke.gcr.io/gcp-compute-persistent-disk-csi-driver | ||
| # Don't change stable image without changing pdImagePlaceholder in | ||
| # test/k8s-integration/main.go | ||
| newName: gke.gcr.io/gcp-compute-persistent-disk-csi-driver | ||
| newTag: "v0.7.0-gke.0" | ||
| - name: gke.gcr.io/gcp-compute-persistent-disk-csi-driver-win | ||
| # Temporarly set to the private repo. Will swtich to public one | ||
| # once it is available. | ||
| newName: gcr.io/jing-k8s-dev/gce-pd-windows-2019 | ||
| newTag: "0.2.0" | ||
| - name: gke.gcr.io/csi-provisioner | ||
| newName: gke.gcr.io/csi-provisioner | ||
| newTag: "v1.5.0-gke.0" | ||
| - name: gke.gcr.io/csi-attacher | ||
| newName: gke.gcr.io/csi-attacher | ||
| newTag: "v2.1.1-gke.0" | ||
| - name: gke.gcr.io/csi-node-driver-registrar | ||
| newName: gcr.io/k8s-staging-csi/csi-node-driver-registrar | ||
| newTag: "amd64-windows-v20200428-v1.3.0-26-g510710d5" | ||
| - name: gke.gcr.io/csi-resizer | ||
| newName: gke.gcr.io/csi-resizer | ||
| newTag: "v0.4.0-gke.0" | ||
| - name: gke.gcr.io/csi-snapshotter | ||
| newName: gke.gcr.io/csi-snapshotter | ||
| newTag: "v2.1.1-gke.0" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This means that the controller may not get rescheduled if the Node becomes unhealthy. Is that what we want?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point, will remove it. I added it because I see node has it. And my test setup has linux tainted.