Skip to content

Give the dockerfile some love (2/x) #94

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed

Give the dockerfile some love (2/x) #94

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8dd6993
Stuff env version variables together
bebehei Feb 15, 2019
bf79218
Export DEBIAN_FRONTEND in every installation layer
bebehei Feb 15, 2019
6d1e898
Put the LaTeX stuff in one image
bebehei Feb 15, 2019
e6ff154
Remove transient dependencies from texlive-full
bebehei Feb 15, 2019
c86a528
Flatten conda and pyenv installs
bebehei Feb 15, 2019
48a0d14
Add users at the beginning
bebehei Feb 15, 2019
88ba5cc
Move env variable block to the highest possible layer
bebehei Feb 15, 2019
8b018e4
Unite python pip installation in one layer
bebehei Feb 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 61 additions & 72 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,23 @@ FROM ubuntu:18.04
LABEL mantainer="Read the Docs <[email protected]>"
LABEL version="5.0.0rc1"

ENV DEBIAN_FRONTEND noninteractive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one is still needed for when you jump into the image and run apt install x directly, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's the reason why it's done via export ....

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, running

docker run -it readthedocs/build:latest /bin/bash

I suppose that we still want to run with the noninteractive, don't we?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running it with docker run -i implies to be interactive, setting DEBIAN_FRONTEND=noninteracive is contradicting. 😉

See the official answer moby/moby#4032 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I know, but I want to have the same behavior when running apt install X from docker run -i than when that command is ran from Read the Docs build process internally (a os.system call at setup.py for example). Otherwise, it's hard to be sure that what I'm testing locally will work in production.

ENV APPDIR /app
ENV LANG C.UTF-8

# Versions, and expose labels for external usage
ENV PYTHON_VERSION_27 2.7.15
ENV PYTHON_VERSION_36 3.6.8
ENV PYTHON_VERSION_37 3.7.2
ENV CONDA_VERSION 4.5.12
LABEL python.version_27=$PYTHON_VERSION_27
LABEL python.version_36=$PYTHON_VERSION_36
LABEL python.version_37=$PYTHON_VERSION_37
LABEL conda.version=$CONDA_VERSION
# UID and GID from readthedocs/user
RUN groupadd --gid 205 docs \
&& useradd --gid 205 -m --uid 1005 docs

# System dependencies
RUN apt-get -y update
RUN apt-get -y install \
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y update \
&& apt-get -y install \
software-properties-common \
vim

# Install requirements
RUN apt-get -y install \
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install \
build-essential \
bzr \
curl \
Expand Down Expand Up @@ -61,7 +56,8 @@ RUN apt-get -y install \

# pyenv extra requirements
# https://github.com/pyenv/pyenv/wiki/Common-build-problems
RUN apt-get install -y \
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install \
liblzma-dev \
libncurses5-dev \
libncursesw5-dev \
Expand All @@ -73,24 +69,11 @@ RUN apt-get install -y \
wget \
xz-utils

# LaTeX -- split to reduce image layer size
RUN apt-get -y install \
texlive-fonts-extra
RUN apt-get -y install \
texlive-latex-extra-doc \
texlive-pictures-doc \
texlive-publishers-doc
RUN apt-get -y install \
texlive-lang-english \
texlive-lang-japanese
RUN apt-get -y install \
texlive-full
RUN apt-get -y install \
# Install LateX packages
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install \
fonts-symbola \
latex-cjk-chinese-arphic-bkai00mp \
latex-cjk-chinese-arphic-gbsn00lp \
latex-cjk-chinese-arphic-gkai00mp \
texlive-fonts-recommended
texlive-full

# plantuml: is to support sphinxcontrib-plantuml
# https://pypi.org/project/sphinxcontrib-plantuml/
Expand All @@ -104,73 +87,79 @@ RUN apt-get -y install \
#
# swig: is required for different purposes
# https://github.com/rtfd/readthedocs-docker-images/issues/15
RUN apt-get -y install \
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install \
imagemagick \
librsvg2-bin \
plantuml \
swig

# Install Python tools/libs
RUN apt-get -y install \
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install \
python-pip \
&& pip install -U \
auxlib \
virtualenv

# sphinx-js dependencies: jsdoc and typedoc (TypeScript support)
RUN apt-get -y install \
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install \
nodejs \
npm \
&& npm install --global \
jsdoc \
typedoc

# UID and GID from readthedocs/user
RUN groupadd --gid 205 docs
RUN useradd -m --uid 1005 --gid 205 docs

USER docs
WORKDIR /home/docs

# Versions, and expose labels for external usage
ENV PYTHON_VERSION_27=2.7.15 \
PYTHON_VERSION_36=3.6.8 \
PYTHON_VERSION_37=3.7.2 \
CONDA_VERSION=4.5.12
LABEL python.version_27=$PYTHON_VERSION_27 \
python.version_36=$PYTHON_VERSION_36 \
python.version_37=$PYTHON_VERSION_37 \
conda.version=$CONDA_VERSION

# Install Conda
RUN curl -O https://repo.continuum.io/miniconda/Miniconda2-${CONDA_VERSION}-Linux-x86_64.sh
RUN bash Miniconda2-${CONDA_VERSION}-Linux-x86_64.sh -b -p /home/docs/.conda/
ENV PATH $PATH:/home/docs/.conda/bin
RUN rm -f Miniconda2-${CONDA_VERSION}-Linux-x86_64.sh
RUN curl -O https://repo.continuum.io/miniconda/Miniconda2-${CONDA_VERSION}-Linux-x86_64.sh \
&& bash Miniconda2-${CONDA_VERSION}-Linux-x86_64.sh -b -p /home/docs/.conda/ \
&& rm -f Miniconda2-${CONDA_VERSION}-Linux-x86_64.sh
ENV PATH=$PATH:/home/docs/.conda/bin

# Install pyenv
RUN wget https://github.com/pyenv/pyenv/archive/master.zip
RUN unzip master.zip && \
rm -f master.zip && \
mv pyenv-master ~docs/.pyenv
ENV PYENV_ROOT /home/docs/.pyenv
ENV PATH /home/docs/.pyenv/shims:$PATH:/home/docs/.pyenv/bin
RUN curl -L https://github.com/pyenv/pyenv/archive/master.tar.gz \
| tar xz \
&& mv pyenv-master .pyenv
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to keep the absolute path here (~docs/.pyenv) to avoid potential future problems.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather use ~/.pyenv, ok? Specifying the docs user should be redundant.

ENV PYENV_ROOT=/home/docs/.pyenv \
PATH=/home/docs/.pyenv/shims:$PATH:/home/docs/.pyenv/bin

# Install supported Python versions
RUN pyenv install $PYTHON_VERSION_27 && \
pyenv install $PYTHON_VERSION_37 && \
pyenv install $PYTHON_VERSION_36 && \
pyenv global \
$PYTHON_VERSION_27 \
$PYTHON_VERSION_37 \
$PYTHON_VERSION_36

WORKDIR /tmp

RUN pyenv local $PYTHON_VERSION_27 && \
pyenv exec pip install --no-cache-dir -U pip && \
pyenv exec pip install --no-cache-dir --only-binary numpy,scipy numpy scipy && \
pyenv exec pip install --no-cache-dir pandas matplotlib virtualenv

RUN pyenv local $PYTHON_VERSION_37 && \
pyenv exec pip install --no-cache-dir -U pip && \
pyenv exec pip install --no-cache-dir --only-binary numpy,scipy numpy scipy && \
pyenv exec pip install --no-cache-dir pandas matplotlib virtualenv

RUN pyenv local $PYTHON_VERSION_36 && \
pyenv exec pip install --no-cache-dir -U pip && \
pyenv exec pip install --no-cache-dir --only-binary numpy,scipy numpy scipy && \
pyenv exec pip install --no-cache-dir pandas matplotlib virtualenv
RUN set -x \
&& pyenv install "${PYTHON_VERSION_27}" \
&& pyenv install "${PYTHON_VERSION_37}" \
&& pyenv install "${PYTHON_VERSION_36}" \
&& pyenv global \
"${PYTHON_VERSION_27}" \
"${PYTHON_VERSION_37}" \
"${PYTHON_VERSION_36}" \
&& true

RUN set -x \
&& cd /tmp \
&& for env in "${PYTHON_VERSION_27}" "${PYTHON_VERSION_37}" "${PYTHON_VERSION_36}"; \
do pyenv local "${env}" \
&& pyenv exec pip install --no-cache-dir -U pip \
&& pyenv exec pip install --no-cache-dir --only-binary numpy,scipy \
matplotlib \
numpy \
pandas \
scipy \
virtualenv \
; done

WORKDIR /

Expand Down