You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> ## *This page is out-of-date and under active development.*
4
-
5
3
This page describes to CSI driver developers how to deploy their driver onto a Kubernetes cluster.
6
4
7
5
## Overview
8
-
There are three components plus the kubelet that enable CSI drivers to provide storage to Kubernetes. These components are sidecar containers which are responsible for communication with both Kubernetes and the CSI driver, making the appropriate CSI calls for their respectful Kubernetes events.
A CSI driver is typically deployed in Kubernetes as two components:
8
+
a controller component and a per-node component.
12
9
13
-
Sidecar containers manage Kubernetes events and make the appropriate calls to the CSI driver. These are the _external attacher_, _external provisioner_, _external snapshotter_ and the _driver registrar_.
10
+
### Controller Plugin
14
11
15
-
### External Attacher
16
-
[external-attacher](https://github.com/kubernetes-csi/external-attacher) is a sidecar container that watches Kubernetes _VolumeAttachment_ objects and triggers CSI _ControllerPublish_ and _ControllerUnpublish_ operations against a driver endpoint. As of this writing, the external attacher does not support leader election and therefore there can be only one running per CSI driver. For more information please read [_Attaching and Detaching_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#attaching-and-detaching).
12
+
The controller component can be deployed as a Deployment or StatefulSet on
13
+
any node in the cluster. It consists of the CSI driver that implements the
14
+
CSI Controller service and one or more
15
+
[sidecar containers](sidecar-containers.md). These controller
16
+
sidecar containers typically interact with Kubernetes objects and make calls
17
+
to the driver's CSI Controller service.
17
18
18
-
Note, even though this is called the _external attacher_, its function is to call the CSI API calls _ControllerPublish_ and _ControllerUnpublish_. These calls most likely will occur in a node which is _not_ the one that will mount the volume. For this reason, many CSI drivers do not support these calls, instead doing the attach/detach and mount/unmount both in the CSI _NodePublish_ and _NodeUnpublish_ calls done by the kubelet at the node which is supposed to mount.
19
+
It generally does not need direct access to the host and can perform all its
20
+
operations through the Kubernetes API and external control plane services.
21
+
Multiple copies of the controller component can be deployed for HA, however
22
+
it is recommended to use leader election to ensure there is only one active
23
+
controller at a time.
19
24
20
-
### External Provisioner
21
-
[external-provisioner](https://github.com/kubernetes-csi/external-provisioner) is a Sidecar container that watches Kubernetes _PersistentVolumeClaim_ objects and triggers CSI _CreateVolume_ and _DeleteVolume_ operations against a driver endpoint. For more information please read [_Provisioning and Deleting_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#provisioning-and-deleting).
25
+
Controller sidecars include the external-provisioner, external-attacher,
26
+
external-snapshotter, and external-resizer. Including a sidecar in the
27
+
deployment may be optional. See each sidecar's page for more details.
22
28
23
-
###External Snapshotter
24
-
[external-snapshotter](https://github.com/kubernetes-csi/external-snapshotter) is a Sidecar container that watches Kubernetes _VolumeSnapshot_ objects and triggers CSI _CreateSnapshot_ and _DeleteSnapshot_ operations against a driver endpoint. For more information please read [_Snapshot Design Proposal_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-snapshot.md).
[driver-registrar](https://github.com/kubernetes-csi/driver-registrar) is a sidecar container that registers the CSI driver with kubelet, and adds the drivers custom NodeId to a label on the Kubernetes Node API Object. It does this by communicating with the _Identity_ service on the CSI driver and also calling the CSI _GetNodeId_ operation. The driver registrar must have the Kubernetes name for the node set through the environment variable `KUBE_NODE_NAME` as follows:
32
+
Sidecar containers manage Kubernetes events and make the appropriate
33
+
calls to the CSI driver. The calls are made by sharing a UNIX domain socket
34
+
through an emptyDir volume between the sidecars and CSI Driver.
28
35
29
-
```yaml
30
-
- name: csi-driver-registrar
31
-
imagePullPolicy: Always
32
-
image: quay.io/k8scsi/driver-registrar:v0.2.0
33
-
args:
34
-
- "--v=5"
35
-
- "--csi-address=$(ADDRESS)"
36
-
env:
37
-
- name: ADDRESS
38
-
value: /csi/csi.sock
39
-
- name: KUBE_NODE_NAME
40
-
valueFrom:
41
-
fieldRef:
42
-
fieldPath: spec.nodeName
43
-
volumeMounts:
44
-
- name: socket-dir
45
-
mountPath: /csi
46
-
```
36
+
#### RBAC Rules
37
+
38
+
Most controller sidecars interact with Kubernetes objects and therefore need
39
+
to set RBAC policies. Each sidecar repository contains example RBAC
40
+
configurations.
41
+
42
+
### Node Plugin
43
+
44
+
The node component should be deployed on every node in the cluster through a
45
+
DaemonSet. It consists of the CSI driver that implements the CSI Node service and the
The Kubernetes kubelet runs on every node and is responsible for making the CSI calls _NodePublish_ and _NodeUnpublish_. These calls mount and unmount the storage volume from the storage system, making it available to the Pod to consume. As shown in the _external-attacher_, most CSI drivers choose to implement both their attach/detach and mount/unmount calls in the _NodePublish_ and _NodeUnpublish_ calls. They do this because the kubelet makes the request on the node which is to consume the volume.
52
+
The Kubernetes kubelet runs on every node and is responsible for making the CSI
53
+
Node service calls. These calls mount and unmount the storage volume from the
54
+
storage system, making it available to the Pod to consume. Kubelet makes calls
55
+
to the CSI driver through a UNIX domain socket shared on the host via a HostPath
56
+
volume. There is also a second UNIX domain socket that the node-driver-registrar
57
+
uses to register the CSI driver to kubelet.
58
+
59
+
#### Driver Volume Mounts
60
+
The node plugin needs direct access to the host for making block devices and/or
61
+
filesystem mounts available to the Kubernetes kubelet.
52
62
53
-
### Mount point
54
-
The mount point used by the CSI driver must be set to _Bidirectional_. See the example below:
63
+
The mount point used by the CSI driver must be set to _Bidirectional_ to allow Kubelet
64
+
on the host to see mounts created by the CSI driver container. See the example below:
55
65
56
66
```yaml
57
-
volumeMounts:
58
-
- name: socket-dir
59
-
mountPath: /csi
60
-
- name: mountpoint-dir
61
-
mountPath: /var/lib/kubelet/pods
62
-
mountPropagation: "Bidirectional"
63
-
volumes:
67
+
containers:
68
+
- name: my-csi-driver
69
+
...
70
+
volumeMounts:
64
71
- name: socket-dir
65
-
hostPath:
66
-
path: /var/lib/kubelet/plugins/csi-hostpath
67
-
type: DirectoryOrCreate
72
+
mountPath: /csi
68
73
- name: mountpoint-dir
69
-
hostPath:
70
-
path: /var/lib/kubelet/pods
71
-
type: Directory
74
+
mountPath: /var/lib/kubelet/pods
75
+
mountPropagation: "Bidirectional"
76
+
- name: node-driver-registrar
77
+
...
78
+
volumeMounts:
79
+
- name: registration-dir
80
+
mountPath: /registration
81
+
volumes:
82
+
# This volume is where the socket for kubelet->driver communication is done
83
+
- name: socket-dir
84
+
hostPath:
85
+
path: /var/lib/kubelet/plugins/<driver-name>
86
+
type: DirectoryOrCreate
87
+
# This volume is where the driver mounts volumes
88
+
- name: mountpoint-dir
89
+
hostPath:
90
+
path: /var/lib/kubelet/pods
91
+
type: Directory
92
+
# This volume is where the node-driver-registrar registers the plugin
93
+
# with kubelet
94
+
- name: registration-dir
95
+
hostPath:
96
+
path: /var/lib/kubelet/plugins_registry
97
+
type: Directory
72
98
```
73
99
74
-
### RBAC Rules
75
-
Side car containers need the appropriate permissions to be able to access and manipulate Kubernetes objects. Here are the RBAC rules needed:
Deploying a CSI driver onto Kubernetes is highlighted in detail in [_Recommended Mechanism for Deploying CSI Drivers on Kubernetes_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md#recommended-mechanism-for-deploying-csi-drivers-on-kubernetes).
@@ -121,10 +122,8 @@ Another feature that CSI depends on is mount propagation. It allows the sharing
121
122
122
123
### Examples
123
124
124
-
- Simple deployment example using a single pod for all components: see the [hostpath example](Example.html).
125
-
- Full deployment example using a _DaemonSet_ for the node plugin and _StatefulSet_ for the controller plugin: check the [NFS driver deployment files][nfs-driver].
125
+
- Simple deployment example using a single pod for all components: see the [hostpath example](example.html).
126
+
- Full deployment example using a _DaemonSet_ for the node plugin and _StatefulSet_ for the controller plugin: TODO
126
127
127
128
## More information
128
129
For more information, please read [_CSI Volume Plugins in Kubernetes Design Doc_](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md).
0 commit comments