Skip to content

chore: add docs re docker inside envbuilder-built-envs #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,11 @@ Provide the encoded JSON config to envbuilder:
DOCKER_CONFIG_BASE64=ewoJImF1dGhzIjogewoJCSJodHRwczovL2luZGV4LmRvY2tlci5pby92MS8iOiB7CgkJCSJhdXRoIjogImJhc2U2NCBlbmNvZGVkIHRva2VuIgoJCX0KCX0KfQo=
```

### Docker-in-Docker

See [here](./docs/docker.md) for instructions on running Docker containers inside
environments built by Envbuilder.

## Git Authentication

Two methods of authentication are supported:
Expand Down
127 changes: 127 additions & 0 deletions docs/docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Docker inside Envbuilder

There are a number of approaches you can use to have access to a Docker daemon
from inside Envbuilder:

## Docker Outside of Docker (DooD)

**Security:** None
**Convenience:** High

This approach re-uses the host Docker socket and passes it inside the container.
It is the simplest approach, but offers **no security** -- any process inside the
container that can connect to the Docker socket will have access to the
underlying host.
Only use it if you are the only person using the Docker socket (for example, if
you are experimenting on your own workstation).

Example:

```console
docker run -it --rm \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/01_dood \
-e ENVBUILDER_INIT_SCRIPT=bash \
-v /var/run/docker.socket:/var/run/docker.socket \
ghcr.io/coder/envbuilder:latest
```


## Docker-in-Docker (DinD)

**Security:** Low
**Convenience:** High

This approach entails running a Docker daemon inside the container.
This requires a privileged container to run, and therefore has a wide potential
attack surface.

Example:

> Note that due to a lack of init system, the Docker daemon
> needs to be started separately inside the container. In this example, we
> create a custom entrypoint to start the Docker daemon in the background and
> call this entrypoint via `ENVBUILDER_INIT_SCRIPT`.

```console
docker run -it --rm \
--privileged \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/02_dind \
-e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
ghcr.io/coder/envbuilder:latest
```

### DinD via Devcontainer Feature

The above can also be accomplished using the [`docker-in-docker` Devcontainer
feature](https://github.com/devcontainers/features/tree/main/src/docker-in-docker).

> Note: we still need the custom entrypoint to start the docker startup script.
> See https://github.com/devcontainers/features/blob/main/src/docker-in-docker/devcontainer-feature.json#L60

Example:

```console
docker run -it --rm \
--privileged \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/03_dind_feature \
-e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
ghcr.io/coder/envbuilder:latest
```

## Rootless DinD

**Security:** Medium
**Convenience:** Medium

This approach runs a Docker daemon in *rootless* mode.
While this still requires a privileged container, this allows you to restrict
usage of the `root` user inside the container, as the Docker daemon will be run
under a "fake" root user (via `rootlesskit`). The user inside the workspace can
then be a 'regular' user without root permissions.

> Note: Once again, we use a custom entrypoint via `ENVBUILDER_INIT_SCRIPT` to
> start the Docker daemon via `rootlesskit`.

Example:

```console
docker run -it --rm \
--privileged \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/04_dind_rootless \
-e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
ghcr.io/coder/envbuilder:latest
```

## Docker-in-Docker using Sysbox

**Security:** High
**Convenience:** Low for infra admins, high for users

This approach requires installing the [`sysbox-runc` container
runtime](https://github.com/nestybox/sysbox/blob/master/docs/user-guide/install-package.md).
This is an alternative container runtime that provides additional benefits,
including transparently enabling Docker inside workspaces. Most notably, it
**does not require a privileged container**, so you can allow developers root
access inside their workspaces, if required.

Example:
```console
docker run -it --rm \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/02_dind \
-e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
--runtime sysbox-runc \
ghcr.io/coder/envbuilder:latest
```

For further information on Sysbox, please consult the [Sysbox
Documentation](https://github.com/nestybox/sysbox/blob/master/docs/user-guide/README.md).
2 changes: 2 additions & 0 deletions examples/docker/01_dood/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
FROM ubuntu:noble
RUN apt-get update && apt-get install -y docker.io
5 changes: 5 additions & 0 deletions examples/docker/01_dood/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"build": {
"dockerfile": "Dockerfile"
}
}
6 changes: 6 additions & 0 deletions examples/docker/02_dind/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM ubuntu:noble
RUN apt-get update && \
apt-get install -y curl apt-transport-https && \
curl -fsSL https://get.docker.com/ | sh -s -
ADD entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
5 changes: 5 additions & 0 deletions examples/docker/02_dind/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"build": {
"dockerfile": "Dockerfile"
}
}
7 changes: 7 additions & 0 deletions examples/docker/02_dind/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

set -euo pipefail

nohup dockerd > /var/log/docker.log 2>&1 &

exec bash --login
3 changes: 3 additions & 0 deletions examples/docker/03_dind_feature/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM ubuntu:noble
ADD entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
8 changes: 8 additions & 0 deletions examples/docker/03_dind_feature/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"build": {
"dockerfile": "Dockerfile"
},
"features": {
"ghcr.io/devcontainers/features/docker-in-docker:2": {}
}
}
7 changes: 7 additions & 0 deletions examples/docker/03_dind_feature/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

set -euo pipefail

/usr/local/share/docker-init.sh

exec bash --login
24 changes: 24 additions & 0 deletions examples/docker/04_dind_rootless/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
FROM ubuntu:noble
# Based on UID of ubuntu user in container.
ENV XDG_RUNTIME_DIR /run/user/1000
ENV DOCKER_HOST unix:///${XDG_RUNTIME_DIR}/docker.sock
# Setup as root
RUN apt-get update && \
# Install prerequisites
apt-get install -y apt-transport-https curl iproute2 uidmap && \
# Install Docker
curl -fsSL https://get.docker.com/ | sh -s - && \
# Add ubuntu user to docker group
usermod -aG docker ubuntu && \
# Create the XDG_RUNTIME_DIR for our user and set DOCKER_HOST
mkdir -p ${XDG_RUNTIME_DIR} && \
chown ubuntu:ubuntu ${XDG_RUNTIME_DIR}

# Setup rootless mode as the ubuntu user.
USER ubuntu
RUN dockerd-rootless-setuptool.sh install && \
docker context use rootless && \
mkdir -p /home/ubuntu/.local/share/docker
# Add our custom entrypoint
ADD entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
5 changes: 5 additions & 0 deletions examples/docker/04_dind_rootless/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"build": {
"dockerfile": "Dockerfile"
}
}
8 changes: 8 additions & 0 deletions examples/docker/04_dind_rootless/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash

set -euo pipefail

# Start the rootless docker daemon as a non-root user
nohup rootlesskit --net=slirp4netns --mtu=1500 --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run dockerd > "/tmp/dockerd-rootless.log" 2>&1 &

exec bash --login