Skip to content

Commit 769ca9f

Browse files
lwasserlwasser
committed
Fix: final edits to packaging section
1 parent 08609d9 commit 769ca9f

6 files changed

+205
-244
lines changed

‎package-structure-code/complex-python-package-builds.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,3 +141,12 @@ pdm and poetry both rely on setuptools for C extensions. pdm's support claims to
141141
142142
https://pdm.fming.dev/latest/pyproject/build/#build-platform-specific-wheels
143143
-->
144+
145+
146+
<!-- https://github.com/pyOpenSci/python-package-guide/pull/23#discussion_r1071541329
147+
ELI: A complex build could mean running a python script that processes some data file and produces a pure python module file.
148+
149+
Probably not common in the scientific community specifically, but I've seen quite a few setup.py files that contain custom build stages which e.g. build gettext locale catalogs.
150+
151+
The main point is that it is more "complex" than simply copying files or directories as-is into the built wheel.
152+
-->

‎package-structure-code/intro.md

Lines changed: 31 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,10 @@
11
# Python package structure information
22

3-
If you plan to submit a package for review to pyOpenSci and are looking for
4-
some guidance on package structure, code formats and style, then this section is for you.
5-
6-
## Guidelines for pyOpenSci's packaging recommendations
7-
8-
<!-- Might belong on the LANDING page for this entire guide?-->
9-
10-
There are some differing opinions on what Python package structure should
11-
look like and what tools to use across the Python ecosystem.
12-
13-
In this guide, we have made decisions around suggested standards and required
14-
standards, based upon the commonly used approaches in the scientific Python
15-
community. Our goal is to help standardize packaging across this ecosystem.
16-
17-
In some cases the suggestions here may diverge from those in the non-scientific parts of the Python ecosystem.
3+
This section provides guidance on your Python package's structure, code formats and style. It also reviews the various packaging tools that you can use to
4+
support building and publishing your package.
185

196
```{note}
20-
The suggestions for package layout in this section are made with the
21-
intent of being helpful; they are not specific requirements for your
22-
package to be reviewed and accepted into our ecosystem.
23-
```
24-
25-
In all cases, we try to align our suggestions with the most current, accepted
26-
[PEP's (Python Enhancement Protocols)](https://peps.python.org/pep-0000/) and the [scientific-python community specs](https://scientific-python.org/specs/).
27-
If you plan to submit a package for review to pyOpenSci and are looking for
28-
some guidance on package structure, code formats and style, then this section
29-
is for you.
30-
31-
<!-- TODO: move this either to the top of this section or the landing page?-->
32-
33-
```{note}
34-
Have a look at the
7+
If you are considering submitting a package for peer review, have a look at the
358
bare-minimum [editor checks](https://www.pyopensci.org/peer-review-guide/software-peer-review-guide/editor-in-chief-guide.html#editor-checklist-template) that pyOpenSci
369
performs before a review begins. These checks are useful to explore
3710
for both authors planning to submit a package to us for review and for
@@ -40,37 +13,50 @@ anyone who is just getting started with creating a Python package.
4013
In general these are basic items that should be in any open software repository.
4114
```
4215

16+
## What you will learn here
17+
18+
In this section of our Python packaging guide, we:
19+
20+
* Provide an overview of the options available to you when packaging your tool
21+
* Suggest tools and approaches that both meet your needs and also support existing standards.
22+
* Suggest tools and approaches that will allow you to expand upon a workflow that may begin as a pure Python tool and evolve into a tool that requires addition layers of complexity in the packaging build.
23+
* Align our suggestions with the most current, accepted
24+
[PEPs (Python Enhancement Protocols)](https://peps.python.org/pep-0000/) and the [scientific-python community SPECs](https://scientific-python.org/specs/).
25+
* In an effort to maintain consistency within our community , we also align with existing best practices being implemented by developers of core Scientific Python packages such as Numpy, SciPy and others.
26+
4327
## Guidelines for pyOpenSci's packaging recommendations
4428

4529
<!-- Might belong on the LANDING page for this entire guide?-->
4630

47-
Python is a flexible programming language that is used across numerous
48-
disciplines and domains. Python is so flexible that it is one of the few
49-
languages that can be used to wrap around other languages.
31+
The flexibility of the Python programming language lends itself to a diverse
32+
range of tool options for creating a Python package. Python is so flexible that
33+
it is one of the few languages that can be used to wrap around other languages.
5034

5135
If you are building a pure Python package, then your packaging setup can be
5236
simple. However, some scientific packages have complex requirements as they may
5337
need to support extensions or tools written in other languages such as C or C++.
5438

5539
To support the many different uses of Python, there are many ways to create a
56-
Python package.
40+
Python package. In this guide, we suggest approaches for packaging approaches and tools based
41+
upon:
5742

58-
The ecosystem is dynamic, and constantly evolving to support
59-
the numerous needs that developers (and scientists) have using Python.
43+
1. What we think will be best and easiest to adopt for those who are newer to packaging
44+
2. Tools that we think are well maintained and documented.
45+
3. A shared goal of standardizing packaging approaches across this (scientific) Python ecosystem.
6046

61-
This dynamic yet flexible environment is what many love about Python.
47+
Here, we also try to align our suggestions with the most current, accepted
48+
[Python community](https://packaging.python.org/en/latest/) and [scientific community](https://scientific-python.org/specs/).
6249

63-
## What you will learn here
6450

65-
In this section of our Python packaging guide, we try to:
51+
```{admonition} Suggestions in this guide are not pyOpenSci review requirements
52+
:class: important
6653
67-
* Provide an overview of the options available to you when packaging your tool
68-
* Suggest tools and approaches that both meet your needs and also support existing standards.
69-
* Suggest tools and approaches that will allow you to expand upon a workflow that may begin as a pure Python tool and evolve into a tool that requires addition layers of complexity in the packaging build.
70-
* Align our suggestions with the most current, accepted
71-
[PEPs (Python Enhancement Protocols)](https://peps.python.org/pep-0000/) and the [scientific-python community SPECs](https://scientific-python.org/specs/).
72-
* In an effort to maintain consistency within our community , we also align with existing best practices being implemented by developers of core Scientific Python packages such as Numpy, SciPy and others.
54+
The suggestions for package layout in this section are made with the
55+
intent of being helpful; they are not specific requirements for your
56+
package to be reviewed and accepted into our pyOpenSci open source ecosystem.
7357
58+
Please check out our [package scope page](https://www.pyopensci.org/software-peer-review/about/package-scope.html) and [review requirements in our author guide](https://www.pyopensci.org/software-peer-review/how-to/author-guide.html#) if you are looking for Python package review requirements!
59+
```
7460

7561
<!--
7662
```{tip}

‎package-structure-code/pyproject-toml-python-package-metadata.md

Lines changed: 36 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -1,73 +1,44 @@
11
# Use a pyproject.toml file for your package configuration & metadata
22

3-
The standard file that Python packages use to specify build requirements and
4-
metadata is called a pyproject.toml. The pyproject.toml file has become the
5-
standard for declaring Python package metadata (including dependencies) rather
6-
than using a setup.py file (or a setup.py + setup.cfg file).
3+
The standard file that Python packages use to [specify build requirements and
4+
metadata is called a **pyproject.toml**](https://packaging.python.org/en/latest/specifications/declaring-project-metadata/). Adding metadata, build requirements
5+
and package dependencies to a **pyproject.toml** file replaces storing that
6+
information in a setup.py or setup.cfg file.
77

8-
As such you should try to [include all project based metadata and build system specifications in a `pyproject.toml` file.](https://packaging.python.org/en/latest/specifications/declaring-project-metadata/) Using setup.py to manage both package set up and
9-
hold metadata [can cause problems with package development.](https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html)
10-
11-
12-
```{admonition} Benefits of using a pyproject.toml file
13-
:class: tip
14-
15-
1. Because setup.py has a mixture of code and metadata, it will be run twice when
16-
building your package. First it will be run to extract metadata (dependencies). Then it will be run to build your package.
17-
1. Including your package's metadata in a separate human-readable `pyproject.toml` format also allows someone to view the project's metadata without
18-
running any Python code.
19-
```
20-
21-
A pyproject.toml is written in [TOML (Tom's Obvious, Minimal Language) format](https://toml.io/en/). TOML is an easy-to-read structure that is founded on key: value pairs.
22-
23-
Each section in the pyproject.toml file contains a `[table identifier]`.
8+
The **pyproject.toml** file is written in [TOML (Tom's Obvious, Minimal Language) format](https://toml.io/en/). TOML is an easy-to-read structure that is founded on key: value pairs. Each section in the **pyproject.toml** file contains a `[table identifier]`.
249
Below that table identifier are key value pairs that
2510
support configuration for that particular table.
2611

12+
### Benefits of using a pyproject.toml file
13+
14+
Including your package's metadata in a separate human-readable **pyproject.toml**
15+
format also allows someone to view the project's metadata in a GitHub repository.
16+
2717
<!-- setup.cfg for project metadata is being deprecated - set setuptools guide and
2818
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
2919
pypa -
3020
https://packaging.python.org/en/latest/specifications/declaring-project-metadata/ -->
3121

32-
```{note}
33-
<!-- [PEP518 describes the move away from setup.py to the pyproject.toml file.](https://peps.python.org/pep-0518/) -->
34-
Python package standards are moving away from including both package metadata
35-
and Python code needed to set up a package in the same **setup.py** file.
36-
Instead we are moving towards using a **proproject.toml** file.
22+
```{admonition} Setup.py is still useful for complex package builds
23+
:class: tip
3724

38-
In some cases where a build is complex, a **setup.py** file may still be
39-
required. While this guide will not cover complex builds, we will provide
25+
Using **setup.py** to manage package builds and metadata [can cause problems with package development](https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html).
26+
In some cases where a Python package build is complex, a **setup.py** file may
27+
be required. While this guide will not cover complex builds, we will provide
4028
resources working with complex builds in the future.
4129

42-
<!-- https://github.com/pyOpenSci/python-package-guide/pull/23#discussion_r1071541329
43-
ELI: A complex build could mean running a python script that processes some data file and produces a pure python module file.
44-
45-
Probably not common in the scientific community specifically, but I've seen quite a few setup.py files that contain custom build stages which e.g. build gettext locale catalogs.
46-
47-
The main point is that it is more "complex" than simply copying files or directories as-is into the built wheel.
48-
-->
4930
```
5031

51-
## Example pyproject.toml
52-
53-
Below is an example build configuration for a Python project. This setup
54-
requires:
55-
56-
* **setuptools** to create the package structure,
57-
* **wheel** which is used by `setuptools` to create the [**.whl** (wheel) file](https://realpython.com/python-wheels/).
58-
* **setuptools build** to "build" the package
59-
* **setuptools_scm** to manage package version updates
60-
61-
In the example below `[build-system]` is the first table
62-
of values. It has two keys that specify the build front end and backend for a package:
32+
### Example pyproject.toml for building using PDM
33+
Below is an example build configuration for a Python project. This example
34+
package setup uses:
6335

64-
1. `requires =`
65-
1. `build-backend =`
36+
* **pdm.pep517.api** to build the [package's SDist and wheels](python-package-distribution-files-sdist-wheel)
6637

6738
```
6839
[build-system]
69-
requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
70-
build-backend = "setuptools.build_meta"
40+
requires = ["pdm-pep517>=1.0.0"]
41+
build-backend = "pdm.pep517.api"
7142
7243
[project]
7344
name = "examplePy"
@@ -85,25 +56,29 @@ dependencies = [
8556
"dependency-package-name-2",
8657
]
8758
```
59+
Notice that dependencies are specified in this file.
8860

61+
### Example pyproject.toml for building using setuptools
8962

90-
Notice that you also can specify dependencies in this file.
91-
63+
The package metadata including authors, keywords, etc is also easy to read.
64+
Below you can see the same toml file that uses a different build system (setuptools).
65+
Notice how simple it is to swap out the tools needed to build this package!
9266

93-
A major benefit of the pyproject.toml file is that it makes is transparent
67+
In this example package setup you use:
9468

95-
1. what build system you are using to create your package
96-
2. what dependencies you need
69+
* **setuptools** to build the [package's SDist and wheels](python-package-distribution-files-sdist-wheel)
70+
* **setuptools_scm** to manage package version updates using version control tags
9771

72+
In the example below `[build-system]` is the first table
73+
of values. It has two keys that specify the build front end and backend for a package:
9874

99-
The package metadata including authors, keywords, etc is also easy to read.
100-
Below you can see the same toml file that uses a different build system (PDM).
101-
Notice how simple it is to swap out the tools needed to build this package!
75+
1. `requires =`
76+
1. `build-backend =`
10277

10378
```
10479
[build-system]
105-
requires = ["pdm-pep517>=1.0.0"]
106-
build-backend = "pdm.pep517.api"
80+
requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
81+
build-backend = "setuptools.build_meta"
10782
10883
[project]
10984
name = "examplePy"
@@ -125,18 +100,5 @@ dependencies = [
125100

126101

127102
```{note}
128-
[Click here to read about our packaging documentation requirements.](/package-structure-code/python-package-build-tools)
103+
[Click here to read about our packaging build tools including PDM, setuptools, Poetry and Hatch.](/package-structure-code/python-package-build-tools)
129104
```
130-
131-
<!-- TODO: add link to section on build tools when it exists and
132-
turn this into button:
133-
134-
We discuss build tools for python package more here.
135-
-->
136-
137-
138-
139-
<!-- TODO:
140-
1. add some links to packages that are using a purely toml config
141-
1. link to our example package once it's further along
142-
-->

0 commit comments

Comments
 (0)