Skip to content

Commit 33114f5

Browse files
author
MarcoGorelli
committed
rewrite contributing environment guide
1 parent a047fb6 commit 33114f5

File tree

1 file changed

+57
-115
lines changed

1 file changed

+57
-115
lines changed

doc/source/development/contributing_environment.rst

+57-115
Original file line numberDiff line numberDiff line change
@@ -15,50 +15,25 @@ locally before pushing your changes. It's recommended to also install the :ref:`
1515
.. contents:: Table of contents:
1616
:local:
1717

18+
.. _contributing.c_compiler:
1819

19-
Option 1: creating an environment without Docker
20-
------------------------------------------------
20+
Step 1: install a C compiler
21+
----------------------------
2122

22-
Installing a C compiler
23-
~~~~~~~~~~~~~~~~~~~~~~~
24-
25-
pandas uses C extensions (mostly written using Cython) to speed up certain
26-
operations. To install pandas from source, you need to compile these C
27-
extensions, which means you need a C compiler. This process depends on which
28-
platform you're using.
29-
30-
If you have setup your environment using :ref:`mamba <contributing.mamba>`, the packages ``c-compiler``
31-
and ``cxx-compiler`` will install a fitting compiler for your platform that is
32-
compatible with the remaining mamba packages. On Windows and macOS, you will
33-
also need to install the SDKs as they have to be distributed separately.
34-
These packages will automatically be installed by using the ``pandas``
35-
``environment.yml`` file.
23+
How to do this will depend on your platform. If you choose to user ``Docker``
24+
in the next step, then you can skip this step.
3625

3726
**Windows**
3827

39-
You will need `Build Tools for Visual Studio 2019
40-
<https://visualstudio.microsoft.com/downloads/>`_.
41-
42-
.. warning::
43-
You DO NOT need to install Visual Studio 2019.
44-
You only need "Build Tools for Visual Studio 2019" found by
45-
scrolling down to "All downloads" -> "Tools for Visual Studio 2019".
46-
In the installer, select the "C++ build tools" workload.
47-
48-
You can install the necessary components on the commandline using
49-
`vs_buildtools.exe <https://download.visualstudio.microsoft.com/download/pr/9a26f37e-6001-429b-a5db-c5455b93953c/460d80ab276046de2455a4115cc4e2f1e6529c9e6cb99501844ecafd16c619c4/vs_BuildTools.exe>`_:
50-
51-
.. code::
28+
You will need `Build Tools for Visual Studio <https://visualstudio.microsoft.com/downloads/>`_.
5229

53-
vs_buildtools.exe --quiet --wait --norestart --nocache ^
54-
--installPath C:\BuildTools ^
55-
--add "Microsoft.VisualStudio.Workload.VCTools;includeRecommended" ^
56-
--add Microsoft.VisualStudio.Component.VC.v141 ^
57-
--add Microsoft.VisualStudio.Component.VC.v141.x86.x64 ^
58-
--add Microsoft.VisualStudio.Component.Windows10SDK.17763
30+
Note: you do not need to install Visual Studio.
31+
You only need "Build Tools for Visual Studio" found by
32+
scrolling down to "All downloads" -> "Tools for Visual Studio".
33+
In the installer, select the "C++ build tools" workload.
5934

60-
To setup the right paths on the commandline, call
61-
``"C:\BuildTools\VC\Auxiliary\Build\vcvars64.bat" -vcvars_ver=14.16 10.0.17763.0``.
35+
Alternatively, you could use the `WSL <https://learn.microsoft.com/en-us/windows/wsl/install>`_
36+
and consult the ``Linux`` instructions below.
6237

6338
**macOS**
6439

@@ -69,12 +44,7 @@ https://devguide.python.org/setup/#macos
6944

7045
**Linux**
7146

72-
For Linux-based :ref:`mamba <contributing.mamba>` installations, you won't have to install any
73-
additional components outside of the mamba environment. The instructions
74-
below are only needed if your setup isn't based on mamba environments.
75-
76-
Some Linux distributions will come with a pre-installed C compiler. To find out
77-
which compilers (and versions) are installed on your system::
47+
You probably already have a C compiler installed. You can check this with::
7848

7949
# for Debian/Ubuntu:
8050
dpkg --list | grep compiler
@@ -83,67 +53,42 @@ which compilers (and versions) are installed on your system::
8353

8454
`GCC (GNU Compiler Collection) <https://gcc.gnu.org/>`_, is a widely used
8555
compiler, which supports C and a number of other languages. If GCC is listed
86-
as an installed compiler nothing more is required. If no C compiler is
87-
installed (or you wish to install a newer version) you can install a compiler
88-
(GCC in the example code below) with::
89-
90-
# for recent Debian/Ubuntu:
91-
sudo apt install build-essential
92-
# for Red Had/RHEL/CentOS/Fedora
93-
yum groupinstall "Development Tools"
56+
as an installed compiler nothing more is required.
9457

95-
For other Linux distributions, consult your favorite search engine for
96-
compiler installation instructions.
58+
If no C compiler is installed, or you wish to upgrade, or you're using a different
59+
Linux distribution, consult your favorite search engine for compiler installation/update
60+
instructions.
9761

9862
Let us know if you have any difficulties by opening an issue or reaching out on our contributor
9963
community :ref:`Slack <community.slack>`.
10064

101-
.. _contributing.mamba:
65+
.. _contributing.isolated_environment:
10266

103-
Option 1a: using mamba (recommended)
104-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
67+
Step 2: create an isolated environment
68+
----------------------------------------
10569

106-
Now create an isolated pandas development environment:
70+
Before we begin, please:
10771

108-
* Install `mamba <https://mamba.readthedocs.io/en/latest/installation.html>`_
109-
* Make sure your mamba is up to date (``mamba update mamba``)
11072
* Make sure that you have :any:`cloned the repository <contributing.forking>`
11173
* ``cd`` to the pandas source directory
11274

113-
We'll now kick off a three-step process:
75+
.. _contributing.mamba:
76+
77+
Option 1: using mamba (recommended)
78+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
11479

115-
1. Install the build dependencies
116-
2. Build and install pandas
117-
3. Install the optional dependencies
80+
* Install `mamba <https://mamba.readthedocs.io/en/latest/installation.html>`_
81+
* Make sure your mamba is up to date (``mamba update mamba``)
11882

11983
.. code-block:: none
12084
12185
# Create and activate the build environment
12286
mamba env create --file environment.yml
12387
mamba activate pandas-dev
12488
125-
# Build and install pandas
126-
python setup.py build_ext -j 4
127-
python -m pip install -e . --no-build-isolation --no-use-pep517
89+
Option 2: using pip
90+
~~~~~~~~~~~~~~~~~~~
12891

129-
At this point you should be able to import pandas from your locally built version::
130-
131-
$ python
132-
>>> import pandas
133-
>>> print(pandas.__version__) # note: the exact output may differ
134-
1.5.0.dev0+1355.ge65a30e3eb.dirty
135-
136-
This will create the new environment, and not touch any of your existing environments,
137-
nor any existing Python installation.
138-
139-
To return to your root environment::
140-
141-
mamba deactivate
142-
143-
Option 1b: using pip
144-
~~~~~~~~~~~~~~~~~~~~
145-
146-
If you aren't using mamba for your development environment, follow these instructions.
14792
You'll need to have at least the :ref:`minimum Python version <install.version>` that pandas supports.
14893
You also need to have ``setuptools`` 51.0.0 or later to build pandas.
14994

@@ -162,10 +107,6 @@ You also need to have ``setuptools`` 51.0.0 or later to build pandas.
162107
# Install the build dependencies
163108
python -m pip install -r requirements-dev.txt
164109
165-
# Build and install pandas
166-
python setup.py build_ext -j 4
167-
python -m pip install -e . --no-build-isolation --no-use-pep517
168-
169110
**Unix**/**macOS with pyenv**
170111

171112
Consult the docs for setting up pyenv `here <https://github.com/pyenv/pyenv>`__.
@@ -174,7 +115,6 @@ Consult the docs for setting up pyenv `here <https://github.com/pyenv/pyenv>`__.
174115
175116
# Create a virtual environment
176117
# Use an ENV_DIR of your choice. We'll use ~/Users/<yourname>/.pyenv/versions/pandas-dev
177-
178118
pyenv virtualenv <version> <name-to-give-it>
179119
180120
# For instance:
@@ -186,42 +126,29 @@ Consult the docs for setting up pyenv `here <https://github.com/pyenv/pyenv>`__.
186126
# Now install the build dependencies in the cloned pandas repo
187127
python -m pip install -r requirements-dev.txt
188128
189-
# Build and install pandas
190-
python setup.py build_ext -j 4
191-
python -m pip install -e . --no-build-isolation --no-use-pep517
192-
193129
**Windows**
194130

195131
Below is a brief overview on how to set-up a virtual environment with Powershell
196132
under Windows. For details please refer to the
197133
`official virtualenv user guide <https://virtualenv.pypa.io/en/latest/user_guide.html#activators>`__.
198134

199-
Use an ENV_DIR of your choice. We'll use ~\\virtualenvs\\pandas-dev where
200-
'~' is the folder pointed to by either $env:USERPROFILE (Powershell) or
201-
%USERPROFILE% (cmd.exe) environment variable. Any parent directories
135+
Use an ENV_DIR of your choice. We'll use ``~\\virtualenvs\\pandas-dev`` where
136+
``~`` is the folder pointed to by either ``$env:USERPROFILE`` (Powershell) or
137+
``%USERPROFILE%`` (cmd.exe) environment variable. Any parent directories
202138
should already exist.
203139

204140
.. code-block:: powershell
205-
206141
# Create a virtual environment
207142
python -m venv $env:USERPROFILE\virtualenvs\pandas-dev
208-
209143
# Activate the virtualenv. Use activate.bat for cmd.exe
210144
~\virtualenvs\pandas-dev\Scripts\Activate.ps1
211-
212145
# Install the build dependencies
213146
python -m pip install -r requirements-dev.txt
214147
215-
# Build and install pandas
216-
python setup.py build_ext -j 4
217-
python -m pip install -e . --no-build-isolation --no-use-pep517
218-
219-
Option 2: creating an environment using Docker
220-
----------------------------------------------
148+
Option 3: using Docker
149+
~~~~~~~~~~~~~~~~~~~~~~
221150

222-
Instead of manually setting up a development environment, you can use `Docker
223-
<https://docs.docker.com/get-docker/>`_ to automatically create the environment with just several
224-
commands. pandas provides a ``DockerFile`` in the root directory to build a Docker image
151+
pandas provides a ``DockerFile`` in the root directory to build a Docker image
225152
with a full pandas development environment.
226153

227154
**Docker Commands**
@@ -238,13 +165,6 @@ Run Container::
238165
# but if not alter ${PWD} to match your local repo path
239166
docker run -it --rm -v ${PWD}:/home/pandas pandas-dev
240167

241-
When inside the running container you can build and install pandas the same way as the other methods
242-
243-
.. code-block:: bash
244-
245-
python setup.py build_ext -j 4
246-
python -m pip install -e . --no-build-isolation --no-use-pep517
247-
248168
*Even easier, you can integrate Docker with the following IDEs:*
249169

250170
**Visual Studio Code**
@@ -258,3 +178,25 @@ See https://code.visualstudio.com/docs/remote/containers for details.
258178
Enable Docker support and use the Services tool window to build and manage images as well as
259179
run and interact with containers.
260180
See https://www.jetbrains.com/help/pycharm/docker.html for details.
181+
182+
Step 3: build and install pandas
183+
--------------------------------
184+
185+
You can now run::
186+
187+
# Build and install pandas
188+
python setup.py build_ext -j 4
189+
python -m pip install -e . --no-build-isolation --no-use-pep517
190+
191+
At this point you should be able to import pandas from your locally built version::
192+
193+
$ python
194+
>>> import pandas
195+
>>> print(pandas.__version__) # note: the exact output may differ
196+
1.5.0.dev0+1355.ge65a30e3eb.dirty
197+
198+
This will create the new environment, and not touch any of your existing environments,
199+
nor any existing Python installation.
200+
201+
Note that you will need to repeat this step each time the C extensions change, for example
202+
if you modified them or if you did a fetch and merge from ``upstream/main``.

0 commit comments

Comments
 (0)