Skip to content

DOC streamline contributing guide a bit #52328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions doc/source/development/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ that is assigned, feel free to kindly ask the current assignee if you can take i

We have several :ref:`contributor community <community>` communication channels, which you are
welcome to join, and ask questions as you figure things out. Among them are regular meetings for
new contributors, dev meetings, a dev mailing list, and a slack for the contributor community.
new contributors, dev meetings, a dev mailing list, and a Slack for the contributor community.
All pandas contributors are welcome to these spaces, where they can connect with each other. Even
maintainers who have been with us for a long time felt just like you when they started out, and
are happy to welcome you and support you as you get to know how we work, and where things are.
Expand Down Expand Up @@ -308,8 +308,9 @@ default commit message will open, and you can simply save and quit this file.
If there are merge conflicts, you need to solve those conflicts. See for
example at https://help.github.com/articles/resolving-a-merge-conflict-using-the-command-line/
for an explanation on how to do this.
Once the conflicts are merged and the files where the conflicts were solved are
added, you can run ``git commit`` to save those fixes.
Once the conflicts are resolved, you should do:
1. ``git add -u`` to stage any files you've updated;
2. ``git commit`` to finish the merge.

If you have uncommitted changes at the moment you want to update the branch with
main, you will need to ``stash`` them prior to updating (see the
Expand All @@ -324,7 +325,7 @@ request by pushing to the branch on GitHub::
Autofixing formatting errors
----------------------------

We use several styling checks (e.g. ``black``, ``flake8``, ``isort``) which are run after
We use several styling checks (e.g. ``black``, ``ruff``, ``isort``) which are run after
you make a pull request.

To automatically fix formatting errors on each commit you make, you can
Expand Down
78 changes: 18 additions & 60 deletions doc/source/development/contributing_codebase.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,40 +18,32 @@ tools will be run to check your code for stylistic errors.
Generating any warnings will cause the test to fail.
Thus, good style is a requirement for submitting code to pandas.

There is a tool in pandas to help contributors verify their changes before
contributing them to the project::
There are a couple of tools in pandas to help contributors verify their changes
before contributing to the project

./ci/code_checks.sh

The script validates the doctests, formatting in docstrings, and
imported modules. It is possible to run the checks independently by using the
parameters ``docstrings``, ``code``, and ``doctests``
(e.g. ``./ci/code_checks.sh doctests``).
- ``./ci/code_checks.sh``: a script validates the doctests, formatting in docstrings,
and imported modules. It is possible to run the checks independently by using the
parameters ``docstrings``, ``code``, and ``doctests``
(e.g. ``./ci/code_checks.sh doctests``);
- ``pre-commit``, which we go into detail on in the next section.

In addition, because a lot of people use our library, it is important that we
do not make sudden changes to the code that could have the potential to break
a lot of user code as a result, that is, we need it to be as *backwards compatible*
as possible to avoid mass breakages.

In addition to ``./ci/code_checks.sh``, some extra checks (including static type
checking) are run by ``pre-commit`` - see :ref:`here <contributing.pre-commit>`
for how to run them.

.. _contributing.pre-commit:

Pre-commit
----------

Additionally, :ref:`Continuous Integration <contributing.ci>` will run code formatting checks
like ``black``, ``ruff``,
``isort``, and ``cpplint`` and more using `pre-commit hooks <https://pre-commit.com/>`_
``isort``, and ``cpplint`` and more using `pre-commit hooks <https://pre-commit.com/>`_.
Any warnings from these checks will cause the :ref:`Continuous Integration <contributing.ci>` to fail; therefore,
it is helpful to run the check yourself before submitting code. This
can be done by installing ``pre-commit``::

pip install pre-commit

and then running::
can be done by installing ``pre-commit`` (which should already have happened if you followed the instructions
in :ref:`Setting up your development environment <contributing_environment>`) and then running::

pre-commit install

Expand All @@ -63,17 +55,17 @@ remain up-to-date with our code checks as they change.
Note that if needed, you can skip these checks with ``git commit --no-verify``.

If you don't want to use ``pre-commit`` as part of your workflow, you can still use it
to run its checks with::
to run its checks with one of the following::

pre-commit run --files <files you have modified>
pre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files

without needing to have done ``pre-commit install`` beforehand.

If you want to run checks on all recently committed files on upstream/main you can use::

pre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files
Finally, we also have some slow pre-commit checks, which don't run on each commit
but which do run during continuous integration. You can trigger them manually with::

without needing to have done ``pre-commit install`` beforehand.
pre-commit run --hook-stage manual --all-files

.. note::

Expand Down Expand Up @@ -170,43 +162,9 @@ pandas strongly encourages the use of :pep:`484` style type hints. New developme
Style guidelines
~~~~~~~~~~~~~~~~

Type imports should follow the ``from typing import ...`` convention. Some types do not need to be imported since :pep:`585` some builtin constructs, such as ``list`` and ``tuple``, can directly be used for type annotations. So rather than

.. code-block:: python

import typing

primes: typing.List[int] = []

You should write

.. code-block:: python

primes: list[int] = []

``Optional`` should be avoided in favor of the shorter ``| None``, so instead of

.. code-block:: python

from typing import Union

maybe_primes: list[Union[int, None]] = []

or

.. code-block:: python

from typing import Optional

maybe_primes: list[Optional[int]] = []

You should write

.. code-block:: python

from __future__ import annotations # noqa: F404

maybe_primes: list[int | None] = []
Type imports should follow the ``from typing import ...`` convention.
Your code may be automatically re-written to use some modern constructs (e.g. using the built-in ``list`` instead of ``typing.List``)
by the :ref:`pre-commit checks <contributing.pre-commit>`.

In some cases in the code base classes may define class variables that shadow builtins. This causes an issue as described in `Mypy 1775 <https://github.com/python/mypy/issues/1775#issuecomment-310969854>`_. The defensive solution here is to create an unambiguous alias of the builtin and use that without your annotation. For example, if you come across a definition like

Expand Down
10 changes: 5 additions & 5 deletions doc/source/development/contributing_environment.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ locally before pushing your changes. It's recommended to also install the :ref:`
Step 1: install a C compiler
----------------------------

How to do this will depend on your platform. If you choose to use ``Docker``
How to do this will depend on your platform. If you choose to use ``Docker`` or ``GitPod``
in the next step, then you can skip this step.

**Windows**
Expand Down Expand Up @@ -213,6 +213,10 @@ You can now run::
python setup.py build_ext -j 4
python -m pip install -e . --no-build-isolation --no-use-pep517

.. note::
You will need to repeat this step each time the C extensions change, for example
if you modified any file in ``pandas/_libs`` or if you did a fetch and merge from ``upstream/main``.

At this point you should be able to import pandas from your locally built version::

$ python
Expand All @@ -222,7 +226,3 @@ At this point you should be able to import pandas from your locally built versio

This will create the new environment, and not touch any of your existing environments,
nor any existing Python installation.

.. note::
You will need to repeat this step each time the C extensions change, for example
if you modified any file in ``pandas/_libs`` or if you did a fetch and merge from ``upstream/main``.