Merge branch 'master' of https://github.com/pandas-dev/pandas into cln-arith

Author: jbrockmendel

.pre-commit-config.yaml

@@ -21,10 +21,12 @@ repos:
           - file
         args: [--append-config=flake8/cython-template.cfg]
 -   repo: https://github.com/PyCQA/isort
-    rev: 5.2.2
+    rev: 5.6.0
     hooks:
     -   id: isort
         exclude: ^pandas/__init__\.py$|^pandas/core/api\.py$
+        files: '.pxd$|.py$'
+        types: [file]
 -   repo: https://github.com/asottile/pyupgrade
     rev: v2.7.2
     hooks:
@@ -39,10 +41,11 @@ repos:
     -   id: pip_to_conda
         name: Generate pip dependency from conda
         description: This hook checks if the conda environment.yml and requirements-dev.txt are equal
-        language: system
+        language: python
         entry: python -m scripts.generate_pip_deps_from_conda
         files: ^(environment.yml|requirements-dev.txt)$
         pass_filenames: false
+        additional_dependencies: [pyyaml]
 -   repo: https://github.com/asottile/yesqa
     rev: v1.2.2
     hooks:

ci/deps/travis-37-cov.yaml

@@ -32,15 +32,15 @@ dependencies:
   - google-cloud-bigquery>=1.27.2 # GH 36436
   - psycopg2
   - pyarrow>=0.15.0
-  - pymysql=0.7.11
+  - pymysql<0.10.0 # temporary pin, GH 36465
   - pytables
   - python-snappy
   - python-dateutil
   - pytz
   - s3fs>=0.4.0
   - scikit-learn
   - scipy
-  - sqlalchemy=1.3.0
+  - sqlalchemy
   - statsmodels
   - xarray
   - xlrd

doc/source/development/contributing.rst

@@ -1365,16 +1365,16 @@ environments. If you want to use virtualenv instead, write::
 The ``-E virtualenv`` option should be added to all ``asv`` commands
 that run benchmarks. The default value is defined in ``asv.conf.json``.
 
-Running the full test suite can take up to one hour and use up to 3GB of RAM.
-Usually it is sufficient to paste only a subset of the results into the pull
-request to show that the committed changes do not cause unexpected performance
-regressions.  You can run specific benchmarks using the ``-b`` flag, which
-takes a regular expression.  For example, this will only run tests from a
-``pandas/asv_bench/benchmarks/groupby.py`` file::
+Running the full benchmark suite can be an all-day process, depending on your
+hardware and its resource utilization. However, usually it is sufficient to paste
+only a subset of the results into the pull request to show that the committed changes
+do not cause unexpected performance regressions.  You can run specific benchmarks
+using the ``-b`` flag, which takes a regular expression. For example, this will
+only run benchmarks from a ``pandas/asv_bench/benchmarks/groupby.py`` file::
 
     asv continuous -f 1.1 upstream/master HEAD -b ^groupby
 
-If you want to only run a specific group of tests from a file, you can do it
+If you want to only run a specific group of benchmarks from a file, you can do it
 using ``.`` as a separator. For example::
 
     asv continuous -f 1.1 upstream/master HEAD -b groupby.GroupByMethods

doc/source/ecosystem.rst

@@ -435,6 +435,11 @@ found in NumPy or pandas, which work well with pandas' data containers.
 Cyberpandas provides an extension type for storing arrays of IP Addresses. These
 arrays can be stored inside pandas' Series and DataFrame.
 
+`Pandas-Genomics`_
+~~~~~~~~~~~~~~~~~~
+
+Pandas-Genomics provides extension types and extension arrays for working with genomics data
+
 `Pint-Pandas`_
 ~~~~~~~~~~~~~~
 
@@ -465,6 +470,7 @@ Library         Accessor   Classes                   Description
 .. _cyberpandas: https://cyberpandas.readthedocs.io/en/latest
 .. _pdvega: https://altair-viz.github.io/pdvega/
 .. _Altair: https://altair-viz.github.io/
+.. _pandas-genomics: https://pandas-genomics.readthedocs.io/en/latest/
 .. _pandas_path: https://github.com/drivendataorg/pandas-path/
 .. _pathlib.Path: https://docs.python.org/3/library/pathlib.html
 .. _pint-pandas: https://github.com/hgrecco/pint-pandas

doc/source/getting_started/intro_tutorials/03_subset_data.rst

@@ -27,14 +27,14 @@ This tutorial uses the Titanic data set, stored as CSV. The data
 consists of the following data columns:
 
 -  PassengerId: Id of every passenger.
--  Survived: This feature have value 0 and 1. 0 for not survived and 1
+-  Survived: This feature has value 0 and 1. 0 for not survived and 1
    for survived.
 -  Pclass: There are 3 classes: Class 1, Class 2 and Class 3.
 -  Name: Name of passenger.
 -  Sex: Gender of passenger.
 -  Age: Age of passenger.
--  SibSp: Indication that passenger have siblings and spouse.
--  Parch: Whether a passenger is alone or have family.
+-  SibSp: Indication that passengers have siblings and spouses.
+-  Parch: Whether a passenger is alone or has a family.
 -  Ticket: Ticket number of passenger.
 -  Fare: Indicating the fare.
 -  Cabin: The cabin of passenger.
@@ -199,7 +199,7 @@ selection brackets ``[]``. Only rows for which the value is ``True``
 will be selected.
 
 We know from before that the original Titanic ``DataFrame`` consists of
-891 rows. Let’s have a look at the amount of rows which satisfy the
+891 rows. Let’s have a look at the number of rows which satisfy the
 condition by checking the ``shape`` attribute of the resulting
 ``DataFrame`` ``above_35``:
 
@@ -398,7 +398,7 @@ See the user guide section on :ref:`different choices for indexing <indexing.cho
     <div class="d-flex flex-row gs-torefguide">
         <span class="badge badge-info">To user guide</span>
 
-A full overview about indexing is provided in the user guide pages on :ref:`indexing and selecting data <indexing>`.
+A full overview of indexing is provided in the user guide pages on :ref:`indexing and selecting data <indexing>`.
 
 .. raw:: html
 

doc/source/getting_started/intro_tutorials/04_plotting.rst

@@ -167,7 +167,7 @@ I want each of the columns in a separate subplot.
     @savefig 04_airqual_area_subplot.png
     axs = air_quality.plot.area(figsize=(12, 4), subplots=True)
 
-Separate subplots for each of the data columns is supported by the ``subplots`` argument
+Separate subplots for each of the data columns are supported by the ``subplots`` argument
 of the ``plot`` functions. The builtin options available in each of the pandas plot
 functions that are worthwhile to have a look.
 
@@ -214,7 +214,7 @@ I want to further customize, extend or save the resulting plot.
         </li>
     </ul>
 
-Each of the plot objects created by pandas are a
+Each of the plot objects created by pandas is a
 `matplotlib <https://matplotlib.org/>`__ object. As Matplotlib provides
 plenty of options to customize plots, making the link between pandas and
 Matplotlib explicit enables all the power of matplotlib to the plot.

doc/source/user_guide/indexing.rst

@@ -933,6 +933,24 @@ and :ref:`Advanced Indexing <advanced>` you may select along more than one axis
 
    df2.loc[criterion & (df2['b'] == 'x'), 'b':'c']
 
+.. warning::
+
+   ``iloc`` supports two kinds of boolean indexing. If the indexer is a boolean ``Series``,
+   an error will be raised. For instance, in the following example, ``df.iloc[s.values, 1]`` is ok.
+   The boolean indexer is an array. But ``df.iloc[s, 1]`` would raise ``ValueError``.
+
+   .. ipython:: python
+
+      df = pd.DataFrame([[1, 2], [3, 4], [5, 6]],
+                        index=list('abc'),
+                        columns=['A', 'B'])
+      s = (df['A'] > 2)
+      s
+
+      df.loc[s, 'B']
+
+      df.iloc[s.values, 1]
+
 .. _indexing.basics.indexing_isin:
 
 Indexing with isin