Skip to content

Commit 6e0c5b7

Browse files
simonjayhawkinssimonjayhawkins
committed
Merge remote-tracking branch 'upstream/main' into bisect
2 parents 9471705 + d924d0b commit 6e0c5b7

File tree

178 files changed

+2729
-773
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

178 files changed

+2729
-773
lines changed

‎.github/workflows/codeql.yml

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
name: CodeQL
2+
on:
3+
schedule:
4+
# every day at midnight
5+
- cron: "0 0 * * *"
6+
7+
concurrency:
8+
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
9+
cancel-in-progress: true
10+
11+
jobs:
12+
analyze:
13+
runs-on: ubuntu-latest
14+
permissions:
15+
actions: read
16+
contents: read
17+
security-events: write
18+
19+
strategy:
20+
fail-fast: false
21+
matrix:
22+
language:
23+
- python
24+
25+
steps:
26+
- uses: actions/checkout@v3
27+
- uses: github/codeql-action/init@v2
28+
with:
29+
languages: ${{ matrix.language }}
30+
- uses: github/codeql-action/autobuild@v2
31+
- uses: github/codeql-action/analyze@v2

‎.github/workflows/sdist.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -79,9 +79,9 @@ jobs:
7979
run: |
8080
case "${{matrix.python-version}}" in
8181
3.8)
82-
pip install numpy==1.19.5 ;;
82+
pip install numpy==1.20.3 ;;
8383
3.9)
84-
pip install numpy==1.19.5 ;;
84+
pip install numpy==1.20.3 ;;
8585
3.10)
8686
pip install numpy==1.21.2 ;;
8787
esac

‎.pre-commit-config.yaml

+14-4
Original file line numberDiff line numberDiff line change
@@ -85,32 +85,42 @@ repos:
8585
- repo: local
8686
hooks:
8787
- id: pyright
88+
# note: assumes python env is setup and activated
8889
name: pyright
8990
entry: pyright
90-
# note: assumes python env is setup and activated
9191
language: node
9292
pass_filenames: false
9393
types: [python]
9494
stages: [manual]
9595
additional_dependencies: &pyright_dependencies
96-
- [email protected].262
96+
- [email protected].264
9797
- id: pyright_reportGeneralTypeIssues
98+
# note: assumes python env is setup and activated
9899
name: pyright reportGeneralTypeIssues
99100
entry: pyright --skipunannotated -p pyright_reportGeneralTypeIssues.json
100-
# note: assumes python env is setup and activated
101101
language: node
102102
pass_filenames: false
103103
types: [python]
104104
stages: [manual]
105105
additional_dependencies: *pyright_dependencies
106106
- id: mypy
107+
# note: assumes python env is setup and activated
107108
name: mypy
108109
entry: mypy
109-
# note: assumes python env is setup and activated
110110
language: system
111111
pass_filenames: false
112112
types: [python]
113113
stages: [manual]
114+
- id: stubtest
115+
# note: assumes python env is setup and activated
116+
# note: requires pandas dev to be installed
117+
name: mypy (stubtest)
118+
entry: python
119+
language: system
120+
pass_filenames: false
121+
types: [pyi]
122+
args: [scripts/run_stubtest.py]
123+
stages: [manual]
114124
- id: flake8-rst
115125
name: flake8-rst
116126
description: Run flake8 on code snippets in docstrings or RST files

‎ci/code_checks.sh

+2-2
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,8 @@ fi
7878
### DOCSTRINGS ###
7979
if [[ -z "$CHECK" || "$CHECK" == "docstrings" ]]; then
8080

81-
MSG='Validate docstrings (EX04, GL01, GL02, GL03, GL04, GL05, GL06, GL07, GL09, GL10, PR03, PR04, PR05, PR06, PR08, PR09, PR10, RT01, RT04, RT05, SA02, SA03, SA04, SS01, SS02, SS03, SS04, SS05)' ; echo $MSG
82-
$BASE_DIR/scripts/validate_docstrings.py --format=actions --errors=EX04,GL01,GL02,GL03,GL04,GL05,GL06,GL07,GL09,GL10,PR03,PR04,PR05,PR06,PR08,PR09,PR10,RT01,RT04,RT05,SA02,SA03,SA04,SS01,SS02,SS03,SS04,SS05
81+
MSG='Validate docstrings (EX04, GL01, GL02, GL03, GL04, GL05, GL06, GL07, GL09, GL10, PR03, PR04, PR05, PR06, PR08, PR09, PR10, RT01, RT04, RT05, SA02, SA03, SA04, SS01, SS02, SS03, SS04, SS05, SS06)' ; echo $MSG
82+
$BASE_DIR/scripts/validate_docstrings.py --format=actions --errors=EX04,GL01,GL02,GL03,GL04,GL05,GL06,GL07,GL09,GL10,PR03,PR04,PR05,PR06,PR08,PR09,PR10,RT01,RT04,RT05,SA02,SA03,SA04,SS01,SS02,SS03,SS04,SS05,SS06
8383
RET=$(($RET + $?)) ; echo $MSG "DONE"
8484

8585
fi

‎ci/deps/actions-310-numpydev.yaml

+1-2
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,7 @@ dependencies:
1616
- pytz
1717
- pip
1818
- pip:
19-
#- cython # TODO: don't install from master after Cython 3.0.0a11 is released
20-
- "git+https://github.com/cython/cython.git@master"
19+
- "cython"
2120
- "--extra-index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple"
2221
- "--pre"
2322
- "numpy"

‎ci/deps/actions-310.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ dependencies:
55
- python=3.10
66

77
# test dependencies
8-
- cython=0.29.30
8+
- cython>=0.29.30
99
- pytest>=6.0
1010
- pytest-cov
1111
- pytest-xdist>=1.31

‎ci/deps/actions-38-downstream_compat.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ dependencies:
66
- python=3.8
77

88
# test dependencies
9-
- cython=0.29.30
9+
- cython>=0.29.30
1010
- pytest>=6.0
1111
- pytest-cov
1212
- pytest-xdist>=1.31

‎ci/deps/actions-38-minimum_versions.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ dependencies:
77
- python=3.8.0
88

99
# test dependencies
10-
- cython=0.29.30
10+
- cython>=0.29.30
1111
- pytest>=6.0
1212
- pytest-cov
1313
- pytest-xdist>=1.31
@@ -17,7 +17,7 @@ dependencies:
1717

1818
# required dependencies
1919
- python-dateutil=2.8.1
20-
- numpy=1.19.5
20+
- numpy=1.20.3
2121
- pytz=2020.1
2222

2323
# optional dependencies

‎ci/deps/actions-38.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ dependencies:
55
- python=3.8
66

77
# test dependencies
8-
- cython=0.29.30
8+
- cython>=0.29.30
99
- pytest>=6.0
1010
- pytest-cov
1111
- pytest-xdist>=1.31

‎ci/deps/actions-39.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ dependencies:
55
- python=3.9
66

77
# test dependencies
8-
- cython=0.29.30
8+
- cython>=0.29.30
99
- pytest>=6.0
1010
- pytest-cov
1111
- pytest-xdist>=1.31

‎ci/deps/circle-38-arm64.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ dependencies:
55
- python=3.8
66

77
# test dependencies
8-
- cython=0.29.30
8+
- cython>=0.29.30
99
- pytest>=6.0
1010
- pytest-cov
1111
- pytest-xdist>=1.31

‎doc/source/conf.py

+15-11
Original file line numberDiff line numberDiff line change
@@ -50,23 +50,25 @@
5050
# sphinxext.
5151

5252
extensions = [
53-
"sphinx.ext.autodoc",
54-
"sphinx.ext.autosummary",
55-
"sphinx.ext.doctest",
56-
"sphinx.ext.extlinks",
57-
"sphinx.ext.todo",
58-
"numpydoc", # handle NumPy documentation formatted docstrings
53+
"contributors", # custom pandas extension
5954
"IPython.sphinxext.ipython_directive",
6055
"IPython.sphinxext.ipython_console_highlighting",
6156
"matplotlib.sphinxext.plot_directive",
62-
"sphinx.ext.intersphinx",
57+
"numpydoc",
58+
"sphinx_copybutton",
59+
"sphinx_panels",
60+
"sphinx_toggleprompt",
61+
"sphinx.ext.autodoc",
62+
"sphinx.ext.autosummary",
6363
"sphinx.ext.coverage",
64-
"sphinx.ext.mathjax",
64+
"sphinx.ext.doctest",
65+
"sphinx.ext.extlinks",
6566
"sphinx.ext.ifconfig",
67+
"sphinx.ext.intersphinx",
6668
"sphinx.ext.linkcode",
69+
"sphinx.ext.mathjax",
70+
"sphinx.ext.todo",
6771
"nbsphinx",
68-
"sphinx_panels",
69-
"contributors", # custom pandas extension
7072
]
7173

7274
exclude_patterns = [
@@ -144,6 +146,9 @@
144146
# already loads it
145147
panels_add_bootstrap_css = False
146148

149+
# https://sphinx-toggleprompt.readthedocs.io/en/stable/#offset
150+
toggleprompt_offset_right = 35
151+
147152
# Add any paths that contain templates here, relative to this directory.
148153
templates_path = ["../_templates"]
149154

@@ -453,7 +458,6 @@
453458
# extlinks alias
454459
extlinks = {
455460
"issue": ("https://github.com/pandas-dev/pandas/issues/%s", "GH"),
456-
"wiki": ("https://github.com/pandas-dev/pandas/wiki/%s", "wiki "),
457461
}
458462

459463

‎doc/source/ecosystem.rst

+3-3
Original file line numberDiff line numberDiff line change
@@ -591,12 +591,12 @@ Library Accessor Classes Description
591591
Development tools
592592
-----------------
593593

594-
`pandas-stubs <https://github.com/VirtusLab/pandas-stubs>`__
595-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
594+
`pandas-stubs <https://github.com/pandas-dev/pandas-stubs>`__
595+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
596596

597597
While pandas repository is partially typed, the package itself doesn't expose this information for external use.
598598
Install pandas-stubs to enable basic type coverage of pandas API.
599599

600600
Learn more by reading through :issue:`14468`, :issue:`26766`, :issue:`28142`.
601601

602-
See installation and usage instructions on the `github page <https://github.com/VirtusLab/pandas-stubs>`__.
602+
See installation and usage instructions on the `github page <https://github.com/pandas-dev/pandas-stubs>`__.

‎doc/source/getting_started/install.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,7 @@ Dependencies
235235
================================================================ ==========================
236236
Package Minimum supported version
237237
================================================================ ==========================
238-
`NumPy <https://numpy.org>`__ 1.19.5
238+
`NumPy <https://numpy.org>`__ 1.20.3
239239
`python-dateutil <https://dateutil.readthedocs.io/en/stable/>`__ 2.8.1
240240
`pytz <https://pypi.org/project/pytz/>`__ 2020.1
241241
================================================================ ==========================

‎doc/source/getting_started/intro_tutorials/10_text_data.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,7 @@ applied to integers, so no ``str`` is used.
179179
180180
Based on the index name of the row (``307``) and the column (``Name``),
181181
we can do a selection using the ``loc`` operator, introduced in the
182-
`tutorial on subsetting <3_subset_data.ipynb>`__.
182+
:ref:`tutorial on subsetting <10min_tut_03_subset>`.
183183

184184
.. raw:: html
185185

‎doc/source/reference/general_functions.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -85,4 +85,4 @@ Importing from other DataFrame libraries
8585
.. autosummary::
8686
:toctree: api/
8787

88-
api.exchange.from_dataframe
88+
api.interchange.from_dataframe

‎doc/source/reference/style.rst

+1
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ Style application
4141
Styler.applymap_index
4242
Styler.format
4343
Styler.format_index
44+
Styler.relabel_index
4445
Styler.hide
4546
Styler.concat
4647
Styler.set_td_classes

‎doc/source/user_guide/io.rst

+23-17
Original file line numberDiff line numberDiff line change
@@ -107,9 +107,10 @@ index_col : int, str, sequence of int / str, or False, optional, default ``None`
107107
string name or column index. If a sequence of int / str is given, a
108108
MultiIndex is used.
109109

110-
Note: ``index_col=False`` can be used to force pandas to *not* use the first
111-
column as the index, e.g. when you have a malformed file with delimiters at
112-
the end of each line.
110+
.. note::
111+
``index_col=False`` can be used to force pandas to *not* use the first
112+
column as the index, e.g. when you have a malformed file with delimiters at
113+
the end of each line.
113114

114115
The default value of ``None`` instructs pandas to guess. If the number of
115116
fields in the column header row is equal to the number of fields in the body
@@ -182,15 +183,16 @@ General parsing configuration
182183
+++++++++++++++++++++++++++++
183184

184185
dtype : Type name or dict of column -> type, default ``None``
185-
Data type for data or columns. E.g. ``{'a': np.float64, 'b': np.int32}``
186-
(unsupported with ``engine='python'``). Use ``str`` or ``object`` together
187-
with suitable ``na_values`` settings to preserve and
188-
not interpret dtype.
186+
Data type for data or columns. E.g. ``{'a': np.float64, 'b': np.int32, 'c': 'Int64'}``
187+
Use ``str`` or ``object`` together with suitable ``na_values`` settings to preserve
188+
and not interpret dtype. If converters are specified, they will be applied INSTEAD
189+
of dtype conversion.
190+
189191
.. versionadded:: 1.5.0
190192

191-
Support for defaultdict was added. Specify a defaultdict as input where
192-
the default determines the dtype of the columns which are not explicitly
193-
listed.
193+
Support for defaultdict was added. Specify a defaultdict as input where
194+
the default determines the dtype of the columns which are not explicitly
195+
listed.
194196
engine : {``'c'``, ``'python'``, ``'pyarrow'``}
195197
Parser engine to use. The C and pyarrow engines are faster, while the python engine
196198
is currently more feature-complete. Multithreading is currently only supported by
@@ -283,7 +285,9 @@ parse_dates : boolean or list of ints or names or list of lists or dict, default
283285
* If ``[[1, 3]]`` -> combine columns 1 and 3 and parse as a single date
284286
column.
285287
* If ``{'foo': [1, 3]}`` -> parse columns 1, 3 as date and call result 'foo'.
286-
A fast-path exists for iso8601-formatted dates.
288+
289+
.. note::
290+
A fast-path exists for iso8601-formatted dates.
287291
infer_datetime_format : boolean, default ``False``
288292
If ``True`` and parse_dates is enabled for a column, attempt to infer the
289293
datetime format to speed up the processing.
@@ -1593,8 +1597,10 @@ of multi-columns indices.
15931597
15941598
pd.read_csv("mi2.csv", header=[0, 1], index_col=0)
15951599
1596-
Note: If an ``index_col`` is not specified (e.g. you don't have an index, or wrote it
1597-
with ``df.to_csv(..., index=False)``, then any ``names`` on the columns index will be *lost*.
1600+
.. note::
1601+
If an ``index_col`` is not specified (e.g. you don't have an index, or wrote it
1602+
with ``df.to_csv(..., index=False)``, then any ``names`` on the columns index will
1603+
be *lost*.
15981604

15991605
.. ipython:: python
16001606
:suppress:
@@ -3048,15 +3054,15 @@ Read in the content of the "books.xml" as instance of ``StringIO`` or
30483054
df = pd.read_xml(bio)
30493055
df
30503056
3051-
Even read XML from AWS S3 buckets such as Python Software Foundation's IRS 990 Form:
3057+
Even read XML from AWS S3 buckets such as NIH NCBI PMC Article Datasets providing
3058+
Biomedical and Life Science Jorurnals:
30523059

30533060
.. ipython:: python
30543061
:okwarning:
30553062
30563063
df = pd.read_xml(
3057-
"s3://irs-form-990/201923199349319487_public.xml",
3058-
xpath=".//irs:Form990PartVIISectionAGrp",
3059-
namespaces={"irs": "http://www.irs.gov/efile"}
3064+
"s3://pmc-oa-opendata/oa_comm/xml/all/PMC1236943.xml",
3065+
xpath=".//journal-meta",
30603066
)
30613067
df
30623068

‎doc/source/whatsnew/v1.4.4.rst

+3
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ including other versions of pandas.
1515
Fixed regressions
1616
~~~~~~~~~~~~~~~~~
1717
- Fixed regression in :func:`concat` materializing :class:`Index` during sorting even if :class:`Index` was already sorted (:issue:`47501`)
18+
- Fixed regression in :meth:`DataFrame.loc` not updating the cache correctly after values were set (:issue:`47867`)
1819
- Fixed regression in setting ``None`` or non-string value into a ``string``-dtype Series using a mask (:issue:`47628`)
1920
-
2021

@@ -26,6 +27,8 @@ Bug fixes
2627
~~~~~~~~~
2728
- The :class:`errors.FutureWarning` raised when passing arguments (other than ``filepath_or_buffer``) as positional in :func:`read_csv` is now raised at the correct stacklevel (:issue:`47385`)
2829
- Bug in :meth:`DataFrame.to_sql` when ``method`` was a ``callable`` that did not return an ``int`` and would raise a ``TypeError`` (:issue:`46891`)
30+
- Bug in :func:`read_xml` when reading XML files with Chinese character tags and would raise ``XMLSyntaxError`` (:issue:`47902`)
31+
- Bug in :meth:`loc.__getitem__` with a list of keys causing an internal inconsistency that could lead to a disconnect between ``frame.at[x, y]`` vs ``frame[y].loc[x]`` (:issue:`22372`)
2932

3033
.. ---------------------------------------------------------------------------
3134

0 commit comments

Comments
 (0)