Skip to content

Backport PR #49981 on branch 1.5.x (Streamline docker usage) #50421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@
// You can edit these settings after create using File > Preferences > Settings > Remote.
"settings": {
"terminal.integrated.shell.linux": "/bin/bash",
"python.condaPath": "/opt/conda/bin/conda",
"python.pythonPath": "/opt/conda/bin/python",
"python.pythonPath": "/usr/local/bin/python",
"python.formatting.provider": "black",
"python.linting.enabled": true,
"python.linting.flake8Enabled": true,
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/code-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ jobs:
run: docker build --pull --no-cache --tag pandas-dev-env .

- name: Show environment
run: docker run -w /home/pandas pandas-dev-env mamba run -n pandas-dev python -c "import pandas as pd; print(pd.show_versions())"
run: docker run --rm pandas-dev-env python -c "import pandas as pd; print(pd.show_versions())"

requirements-dev-text-installable:
name: Test install requirements-dev.txt
Expand Down
49 changes: 10 additions & 39 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,42 +1,13 @@
FROM quay.io/condaforge/mambaforge
FROM python:3.10.8
WORKDIR /home/pandas

# if you forked pandas, you can pass in your own GitHub username to use your fork
# i.e. gh_username=myname
ARG gh_username=pandas-dev
ARG pandas_home="/home/pandas"
RUN apt-get update && apt-get -y upgrade
RUN apt-get install -y build-essential

# Avoid warnings by switching to noninteractive
ENV DEBIAN_FRONTEND=noninteractive
# hdf5 needed for pytables installation
RUN apt-get install -y libhdf5-dev

# Configure apt and install packages
RUN apt-get update \
&& apt-get -y install --no-install-recommends apt-utils git tzdata dialog 2>&1 \
#
# Configure timezone (fix for tests which try to read from "/etc/localtime")
&& ln -fs /usr/share/zoneinfo/Etc/UTC /etc/localtime \
&& dpkg-reconfigure -f noninteractive tzdata \
#
# cleanup
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*

# Switch back to dialog for any ad-hoc use of apt-get
ENV DEBIAN_FRONTEND=dialog

# Clone pandas repo
RUN mkdir "$pandas_home" \
&& git clone "https://github.com/$gh_username/pandas.git" "$pandas_home" \
&& cd "$pandas_home" \
&& git remote add upstream "https://github.com/pandas-dev/pandas.git" \
&& git pull upstream main

# Set up environment
RUN mamba env create -f "$pandas_home/environment.yml"

# Build C extensions and pandas
SHELL ["mamba", "run", "--no-capture-output", "-n", "pandas-dev", "/bin/bash", "-c"]
RUN cd "$pandas_home" \
&& export \
&& python setup.py build_ext -j 4 \
&& python -m pip install --no-build-isolation -e .
RUN python -m pip install --upgrade pip
RUN python -m pip install --use-deprecated=legacy-resolver \
-r https://raw.githubusercontent.com/pandas-dev/pandas/main/requirements-dev.txt
CMD ["/bin/bash"]
30 changes: 9 additions & 21 deletions doc/source/development/contributing_environment.rst
Original file line number Diff line number Diff line change
Expand Up @@ -216,34 +216,22 @@ with a full pandas development environment.

Build the Docker image::

# Build the image pandas-yourname-env
docker build --tag pandas-yourname-env .
# Or build the image by passing your GitHub username to use your own fork
docker build --build-arg gh_username=yourname --tag pandas-yourname-env .
# Build the image
docker build -t pandas-dev .

Run Container::

# Run a container and bind your local repo to the container
docker run -it -w /home/pandas --rm -v path-to-local-pandas-repo:/home/pandas pandas-yourname-env
# This command assumes you are running from your local repo
# but if not alter ${PWD} to match your local repo path
docker run -it --rm -v ${PWD}:/home/pandas pandas-dev

Then a ``pandas-dev`` virtual environment will be available with all the development dependencies.
When inside the running container you can build and install pandas the same way as the other methods

.. code-block:: shell

root@... :/home/pandas# conda env list
# conda environments:
#
base * /opt/conda
pandas-dev /opt/conda/envs/pandas-dev

.. note::
If you bind your local repo for the first time, you have to build the C extensions afterwards.
Run the following command inside the container::

python setup.py build_ext -j 4
.. code-block:: bash

You need to rebuild the C extensions anytime the Cython code in ``pandas/_libs`` changes.
This most frequently occurs when changing or merging branches.
python setup.py build_ext -j 4
python -m pip install -e . --no-build-isolation --no-use-pep517

*Even easier, you can integrate Docker with the following IDEs:*

Expand Down