Skip to content

chore: add docs re docker inside envbuilder-built-envs #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,11 @@ Provide the encoded JSON config to envbuilder:
DOCKER_CONFIG_BASE64=ewoJImF1dGhzIjogewoJCSJodHRwczovL2luZGV4LmRvY2tlci5pby92MS8iOiB7CgkJCSJhdXRoIjogImJhc2U2NCBlbmNvZGVkIHRva2VuIgoJCX0KCX0KfQo=
```

### Docker-in-Docker

See [here](./docs/docker.md) for instructions on running Docker containers inside
environments built by Envbuilder.

## Git Authentication

Two methods of authentication are supported:
Expand Down
108 changes: 108 additions & 0 deletions docs/docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# Docker inside Envbuilder

There are a number of approaches you can use to have access to a Docker daemon
from inside Envbuilder:

## Docker Outside of Docker (DooD)

**Security:** None
**Convenience:** High

This approach re-uses the host Docker socket and passes it inside the container.
It is the simplest approach, but offers **no security** -- any process inside the
container that can connect to the Docker socket will have access to the
underlying host.
Only use it if you are the only person using the Docker socket (for example, if
you are experimenting on your own workstation).

Example:

```shell
docker run -it --rm \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/01_dood \
-e ENVBUILDER_INIT_SCRIPT=bash \
ghcr.io/coder/envbuilder:latest
```


## Docker-in-Docker (DinD)

**Security:** Low
**Convenience:** High

This approach entails running a Docker daemon inside the container.
This requires a privileged container to run, and therefore has a wide potential
attack surface.

Example:

> Note that due to a lack of init system, the Docker daemon
> needs to be started separately inside the container. In this example, we
> create a custom entrypoint to start the Docker daemon in the background and
> call this entrypoint via `ENVBUILDER_INIT_SCRIPT`.

```shell
docker run -it --rm \
--privileged \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/02_dind \
-e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
ghcr.io/coder/envbuilder:latest
```

This can also be accomplished using the [`docker-in-docker` Devcontainer feature](https://github.com/devcontainers/features/tree/main/src/docker-in-docker).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just off the top of my head, this won't work OOTB, envbuilder doesn't run the entrypoint defined here:

https://github.com/devcontainers/features/blob/67c10a660868260c284b02f2878dcfdcfd91279f/src/docker-in-docker/devcontainer-feature.json#L60

Mounts may also require special consideration. 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the custom entrypoint is still required here. 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should point it out here, otherwise we open ourselves up to support requests for feature X not working.

Later on, we should think about how we can support launching these entrypoints from envbulider.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a separate example 03_dind_feature for this. Works fine with volume mounts as far as I tested, too.


## Rootless DinD

**Security:** Medium
**Convenience:** Medium

This approach runs a Docker daemon in *rootless* mode.
While this still requires a privileged container, this allows you to restrict
usage of the `root` user inside the container, as the Docker daemon will be run
under a "fake" root user (via `rootlesskit`). The user inside the workspace can
then be a 'regular' user without root permissions.

> Note: Once again, we use a custom entrypoint via `ENVBUILDER_INIT_SCRIPT` to
> start the Docker daemon via `rootlesskit`.

Example:

```
docker run -it --rm \
--privileged \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/03_dind_rootless \
-e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
ghcr.io/coder/envbuilder:latest
```

## Docker-in-Docker using Sysbox

**Security:** High
**Convenience:** Low for infra admins, high for users

This approach requires installing the [`sysbox-runc` container
runtime](https://github.com/nestybox/sysbox/blob/master/docs/user-guide/install-package.md).
This is an alternative container runtime that provides additional benefits,
including transparently enabling Docker inside workspaces. Most notably, it
**does not require a privileged container**, so you can allow developers root
access inside their workspaces, if required.

Example:
```
docker run -it --rm \
-v /tmp/envbuilder:/workspaces \
-e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
-e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/02_dind \
-e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
--runtime sysbox-runc \
ghcr.io/coder/envbuilder:latest
```

For further information on Sysbox, please consult the [Sysbox
Documentation](https://github.com/nestybox/sysbox/blob/master/docs/user-guide/README.md).
2 changes: 2 additions & 0 deletions examples/docker/01_dood/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
FROM ubuntu:latest
RUN apt-get update && apt-get install -y docker.io
5 changes: 5 additions & 0 deletions examples/docker/01_dood/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"build": {
"dockerfile": "Dockerfile"
}
}
6 changes: 6 additions & 0 deletions examples/docker/02_dind/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
FROM ubuntu:latest
RUN apt-get update && \
apt-get install -y curl apt-transport-https && \
curl -fsSL https://get.docker.com/ | sh -s -
ADD entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
5 changes: 5 additions & 0 deletions examples/docker/02_dind/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"build": {
"dockerfile": "Dockerfile"
}
}
7 changes: 7 additions & 0 deletions examples/docker/02_dind/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

set -euo pipefail

nohup dockerd > /var/log/docker.log 2>&1 &

exec bash --login
22 changes: 22 additions & 0 deletions examples/docker/03_dind_rootless/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
FROM ubuntu:latest
## Install prerequisites
RUN apt-get update && \
apt-get install -y apt-transport-https curl iproute2 uidmap
# Install Docker daemon
RUN curl -fsSL https://get.docker.com/ | sh -s -
# Add the ubuntu user to the Docker group
RUN usermod -aG docker ubuntu
# Add our custom entrypoint
ADD entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
# Create the XDG_RUNTIME_DIR for our user and set DOCKER_HOST
ENV XDG_RUNTIME_DIR /run/user/1000
RUN mkdir -p ${XDG_RUNTIME_DIR} && chown ubuntu:ubuntu ${XDG_RUNTIME_DIR}
ENV DOCKER_HOST unix:///${XDG_RUNTIME_DIR}/docker.sock
# Setup rootless mode as the ubuntu user.
USER ubuntu
RUN dockerd-rootless-setuptool.sh install
# Switch the ubuntu user to use the rootless docker context
RUN docker context use rootless
RUN mkdir -p /home/ubuntu/.local/share/docker
ENTRYPOINT ["/entrypoint.sh"]
5 changes: 5 additions & 0 deletions examples/docker/03_dind_rootless/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"build": {
"dockerfile": "Dockerfile"
}
}
8 changes: 8 additions & 0 deletions examples/docker/03_dind_rootless/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash

set -euo pipefail

# Start the rootless docker daemon as a non-root user
nohup rootlesskit --net=slirp4netns --mtu=1500 --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run dockerd > "/tmp/dockerd-rootless.log" 2>&1 &

exec bash --login