Skip to content

DEPS: drop numpy < 1.12 #23062

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 45 commits into from
Oct 15, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
22dab2d
DEPS: drop numpy < 1.12
h-vetinari Oct 9, 2018
a495db9
Drop bottleneck < 1.2.0
h-vetinari Oct 9, 2018
124d511
Drop matplotlib < 2.0.0
h-vetinari Oct 9, 2018
12b9a79
Drop numexpr < 2.6.2
h-vetinari Oct 9, 2018
c8bfe66
Remove mpl compat code
h-vetinari Oct 9, 2018
b2cffb4
Remove redundant functions
h-vetinari Oct 9, 2018
e9c407e
Reinstante _unpack_cycler
h-vetinari Oct 10, 2018
aac634d
Drop pytables < 3.4.2
h-vetinari Oct 10, 2018
f18880d
Bump numpy requirement in install.py
h-vetinari Oct 10, 2018
f81ad9f
Add output for debugging tests/plotting/test_datetimelike
h-vetinari Oct 10, 2018
c29b478
Drop scipy < 0.18.1
h-vetinari Oct 10, 2018
7ecc5f2
Fix debugging output
h-vetinari Oct 10, 2018
6e533e7
Reduce min version for numexpr/pytables after conda investigation
h-vetinari Oct 10, 2018
2ab7f55
Review (jreback & jbrockmendel)
h-vetinari Oct 10, 2018
7ca1753
Next iteration debugging tests/plotting/test_datetimelike
h-vetinari Oct 10, 2018
9b85061
Spread matplotlib-version in CI for debugging
h-vetinari Oct 10, 2018
5b54612
Re-drop pytables < 3.4.2
h-vetinari Oct 10, 2018
708b2f6
Remove non-existent version
h-vetinari Oct 10, 2018
a41ed9f
Debugggggg
h-vetinari Oct 10, 2018
1e0c553
Fix typos
h-vetinari Oct 10, 2018
317e042
More mpl compat
h-vetinari Oct 10, 2018
31dc4fa
Revert xfails for missing MaskedArrays
h-vetinari Oct 10, 2018
ec93bdb
Clean up diff/switches in test_datetimelike
h-vetinari Oct 11, 2018
d20077a
Fix wording
h-vetinari Oct 11, 2018
925b555
Review (jreback)
h-vetinari Oct 11, 2018
b971bfb
Last round of mpl debugging
h-vetinari Oct 11, 2018
93eabad
Tiny fix
h-vetinari Oct 11, 2018
d72b547
Revert debugging helpers
h-vetinari Oct 11, 2018
e075eff
Partially revert mpl-pins used for debugging
h-vetinari Oct 11, 2018
305f12b
Merge remote-tracking branch 'upstream/master' into bump_numpy
h-vetinari Oct 11, 2018
752b5d7
Rename azure containers
h-vetinari Oct 11, 2018
9dc846a
Bump numexpr check
h-vetinari Oct 12, 2018
e7f5bf2
Merge remote-tracking branch 'upstream/master' into bump_numpy
h-vetinari Oct 12, 2018
169974b
Try retrigger CircleCI
h-vetinari Oct 12, 2018
5b45639
Better error log for failing tests
h-vetinari Oct 12, 2018
308e943
Debugging azure macos build
h-vetinari Oct 12, 2018
9be3d10
Revert "Partially revert mpl-pins used for debugging"
h-vetinari Oct 12, 2018
61763f6
Final debug???
h-vetinari Oct 12, 2018
ededd73
shift debugging pin away from doc build
h-vetinari Oct 12, 2018
079bdaf
MPLMPLMPLMPLMPLMPLMPLMPLMPLMPLMPL
h-vetinari Oct 12, 2018
162458b
Revert "shift debugging pin away from doc build"
h-vetinari Oct 12, 2018
cdd497d
Revert "Revert "Partially revert mpl-pins used for debugging""
h-vetinari Oct 12, 2018
d8587a8
Merge branch 'master' into bump_numpy
h-vetinari Oct 14, 2018
8e37179
Retrigger CI
h-vetinari Oct 14, 2018
d097b43
Merge remote-tracking branch 'upstream/master' into bump_numpy
h-vetinari Oct 15, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ci/azure-macos-35.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ dependencies:
- html5lib
- jinja2
- lxml
- matplotlib
- matplotlib=2.2.0
- nomkl
- numexpr
- numpy=1.10.4
- numpy=1.12.0
- openpyxl=2.5.5
- pytables
- python=3.5*
Expand Down
2 changes: 1 addition & 1 deletion ci/azure-windows-27.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ dependencies:
- html5lib
- jinja2=2.8
- lxml
- matplotlib
- matplotlib=2.0.1
- numexpr
- numpy=1.12*
- openpyxl=2.5.5
Expand Down
2 changes: 1 addition & 1 deletion ci/azure/macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
strategy:
maxParallel: 11
matrix:
py35_np_110:
py35_np_120:
ENV_FILE: ci/azure-macos-35.yaml
CONDA_PY: "35"
CONDA_ENV: pandas
Expand Down
2 changes: 1 addition & 1 deletion ci/azure/windows-py27.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
strategy:
maxParallel: 11
matrix:
py36_np14:
py36_np121:
ENV_FILE: ci/azure-windows-27.yaml
CONDA_PY: "27"
CONDA_ENV: pandas
Expand Down
10 changes: 5 additions & 5 deletions ci/circle-27-compat.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,18 @@ channels:
- defaults
- conda-forge
dependencies:
- bottleneck=1.0.0
- bottleneck=1.2.0
- cython=0.28.2
- jinja2=2.8
- numexpr=2.4.4 # we test that we correctly don't use an unsupported numexpr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you remove code related this this?

Copy link
Contributor Author

@h-vetinari h-vetinari Oct 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure where code related to this would be hiding? There was nothing in compat.__init__.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(bamboo-dev) jreback@dev:~/pandas-dev$ more pandas/core/computation/check.py 
import warnings
from distutils.version import LooseVersion

_NUMEXPR_INSTALLED = False
_MIN_NUMEXPR_VERSION = "2.4.6"

try:
    import numexpr as ne
    ver = LooseVersion(ne.__version__)
    _NUMEXPR_INSTALLED = ver >= LooseVersion(_MIN_NUMEXPR_VERSION)

    if not _NUMEXPR_INSTALLED:
        warnings.warn(
            "The installed version of numexpr {ver} is not supported "
            "in pandas and will be not be used\nThe minimum supported "
            "version is {min_ver}\n".format(
                ver=ver, min_ver=_MIN_NUMEXPR_VERSION), UserWarning)

except ImportError:  # pragma: no cover
    pass

__all__ = ['_NUMEXPR_INSTALLED']

- numpy=1.9.3
- numexpr=2.6.1
- numpy=1.12.0
- openpyxl=2.5.5
- psycopg2
- pytables=3.2.2
- pytables=3.4.2
- python-dateutil=2.5.0
- python=2.7*
- pytz=2013b
- scipy=0.14.0
- scipy=0.18.1
- sqlalchemy=0.7.8
- xlrd=0.9.2
- xlsxwriter=0.5.2
Expand Down
10 changes: 5 additions & 5 deletions ci/requirements-optional-conda.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
beautifulsoup4>=4.2.1
blosc
bottleneck
bottleneck>=1.2.0
fastparquet
feather-format
gcsfs
Expand All @@ -9,17 +9,17 @@ ipython>=5.6.0
ipykernel
jinja2
lxml
matplotlib
matplotlib>=2.0.0
nbsphinx
numexpr
numexpr>=2.6.1
openpyxl=2.5.5
pyarrow
pymysql
pytables
pytables>=3.4.2
pytest-cov
pytest-xdist
s3fs
scipy
scipy>=0.18.1
seaborn
sqlalchemy
statsmodels
Expand Down
8 changes: 4 additions & 4 deletions ci/requirements-optional-pip.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Do not modify directly
beautifulsoup4>=4.2.1
blosc
bottleneck
bottleneck>=1.2.0
fastparquet
feather-format
gcsfs
Expand All @@ -11,17 +11,17 @@ ipython>=5.6.0
ipykernel
jinja2
lxml
matplotlib
matplotlib>=2.0.0
nbsphinx
numexpr
numexpr>=2.6.1
openpyxl==2.5.5
pyarrow
pymysql
tables
pytest-cov
pytest-xdist
s3fs
scipy
scipy>=0.18.1
seaborn
sqlalchemy
statsmodels
Expand Down
6 changes: 3 additions & 3 deletions ci/travis-27-locale.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ channels:
- defaults
- conda-forge
dependencies:
- bottleneck=1.0.0
- bottleneck=1.2.0
- cython=0.28.2
- lxml
- matplotlib=1.4.3
- numpy=1.9.3
- matplotlib=2.0.0
- numpy=1.12.0
- openpyxl=2.4.0
- python-dateutil
- python-blosc
Expand Down
2 changes: 1 addition & 1 deletion ci/travis-27.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ dependencies:
- jemalloc=4.5.0.post
- jinja2=2.8
- lxml
- matplotlib
- matplotlib=2.2.2
- mock
- nomkl
- numexpr
Expand Down
12 changes: 6 additions & 6 deletions doc/source/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ Dependencies
------------

* `setuptools <https://setuptools.readthedocs.io/en/latest/>`__: 24.2.0 or higher
* `NumPy <http://www.numpy.org>`__: 1.9.0 or higher
* `NumPy <http://www.numpy.org>`__: 1.12.0 or higher
* `python-dateutil <https://dateutil.readthedocs.io/en/stable/>`__: 2.5.0 or higher
* `pytz <http://pytz.sourceforge.net/>`__

Expand All @@ -236,11 +236,11 @@ Recommended Dependencies

* `numexpr <https://github.com/pydata/numexpr>`__: for accelerating certain numerical operations.
``numexpr`` uses multiple cores as well as smart chunking and caching to achieve large speedups.
If installed, must be Version 2.4.6 or higher.
If installed, must be Version 2.6.1 or higher.

* `bottleneck <https://github.com/kwgoodman/bottleneck>`__: for accelerating certain types of ``nan``
evaluations. ``bottleneck`` uses specialized cython routines to achieve large speedups. If installed,
must be Version 1.0.0 or higher.
must be Version 1.2.0 or higher.

.. note::

Expand All @@ -255,9 +255,9 @@ Optional Dependencies

* `Cython <http://www.cython.org>`__: Only necessary to build development
version. Version 0.28.2 or higher.
* `SciPy <http://www.scipy.org>`__: miscellaneous statistical functions, Version 0.14.0 or higher
* `SciPy <http://www.scipy.org>`__: miscellaneous statistical functions, Version 0.18.1 or higher
* `xarray <http://xarray.pydata.org>`__: pandas like handling for > 2 dims, needed for converting Panels to xarray objects. Version 0.7.0 or higher is recommended.
* `PyTables <http://www.pytables.org>`__: necessary for HDF5-based storage. Version 3.0.0 or higher required, Version 3.2.1 or higher highly recommended.
* `PyTables <http://www.pytables.org>`__: necessary for HDF5-based storage, Version 3.4.2 or higher
* `Feather Format <https://github.com/wesm/feather>`__: necessary for feather-based storage, version 0.3.1 or higher.
* `Apache Parquet <https://parquet.apache.org/>`__, either `pyarrow <http://arrow.apache.org/docs/python/>`__ (>= 0.4.1) or `fastparquet <https://fastparquet.readthedocs.io/en/latest>`__ (>= 0.0.6) for parquet-based storage. The `snappy <https://pypi.org/project/python-snappy>`__ and `brotli <https://pypi.org/project/brotlipy>`__ are available for compression support.
* `SQLAlchemy <http://www.sqlalchemy.org>`__: for SQL database support. Version 0.8.1 or higher recommended. Besides SQLAlchemy, you also need a database specific driver. You can find an overview of supported drivers for each SQL dialect in the `SQLAlchemy docs <http://docs.sqlalchemy.org/en/latest/dialects/index.html>`__. Some common drivers are:
Expand All @@ -266,7 +266,7 @@ Optional Dependencies
* `pymysql <https://github.com/PyMySQL/PyMySQL>`__: for MySQL.
* `SQLite <https://docs.python.org/3/library/sqlite3.html>`__: for SQLite, this is included in Python's standard library by default.

* `matplotlib <http://matplotlib.org/>`__: for plotting, Version 1.4.3 or higher.
* `matplotlib <http://matplotlib.org/>`__: for plotting, Version 2.0.0 or higher.
* For Excel I/O:

* `xlrd/xlwt <http://www.python-excel.org/>`__: Excel reading (xlrd) and writing (xlwt)
Expand Down
24 changes: 24 additions & 0 deletions doc/source/whatsnew/v0.24.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -206,8 +206,32 @@ Other Enhancements

Backwards incompatible API changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- A newly constructed empty :class:`DataFrame` with integer as the ``dtype`` will now only be cast to ``float64`` if ``index`` is specified (:issue:`22858`)

.. _whatsnew_0240.api_breaking.deps:

Dependencies have increased minimum versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We have updated our minimum supported versions of dependencies (:issue:`21242`).
If installed, we now require:

+-----------------+-----------------+----------+
| Package | Minimum Version | Required |
+=================+=================+==========+
| numpy | 1.12.0 | X |
+-----------------+-----------------+----------+
| bottleneck | 1.2.0 | |
+-----------------+-----------------+----------+
| matplotlib | 2.0.0 | |
+-----------------+-----------------+----------+
| numexpr | 2.6.1 | |
+-----------------+-----------------+----------+
| pytables | 3.4.2 | |
+-----------------+-----------------+----------+
| scipy | 0.18.1 | |
+-----------------+-----------------+----------+

.. _whatsnew_0240.api_breaking.interval_values:

Expand Down
31 changes: 10 additions & 21 deletions pandas/compat/numpy/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,16 @@
# numpy versioning
_np_version = np.__version__
_nlv = LooseVersion(_np_version)
_np_version_under1p10 = _nlv < LooseVersion('1.10')
_np_version_under1p11 = _nlv < LooseVersion('1.11')
_np_version_under1p12 = _nlv < LooseVersion('1.12')
_np_version_under1p13 = _nlv < LooseVersion('1.13')
_np_version_under1p14 = _nlv < LooseVersion('1.14')
_np_version_under1p15 = _nlv < LooseVersion('1.15')


if _nlv < '1.9':
if _nlv < '1.12':
raise ImportError('this version of pandas is incompatible with '
'numpy < 1.9.0\n'
'numpy < 1.12.0\n'
'your numpy version is {0}.\n'
'Please upgrade numpy to >= 1.9.0 to use '
'Please upgrade numpy to >= 1.12.0 to use '
'this pandas version'.format(_np_version))


Expand All @@ -43,9 +40,7 @@ def np_datetime64_compat(s, *args, **kwargs):
tz-changes in 1.11 that make '2015-01-01 09:00:00Z' show a deprecation
warning, when need to pass '2015-01-01 09:00:00'
"""

if not _np_version_under1p11:
s = tz_replacer(s)
s = tz_replacer(s)
return np.datetime64(s, *args, **kwargs)


Expand All @@ -56,23 +51,17 @@ def np_array_datetime64_compat(arr, *args, **kwargs):
tz-changes in 1.11 that make '2015-01-01 09:00:00Z' show a deprecation
warning, when need to pass '2015-01-01 09:00:00'
"""

if not _np_version_under1p11:

# is_list_like
if (hasattr(arr, '__iter__') and
not isinstance(arr, string_and_binary_types)):
arr = [tz_replacer(s) for s in arr]
else:
arr = tz_replacer(arr)
# is_list_like
if (hasattr(arr, '__iter__')
and not isinstance(arr, string_and_binary_types)):
arr = [tz_replacer(s) for s in arr]
else:
arr = tz_replacer(arr)

return np.array(arr, *args, **kwargs)


__all__ = ['np',
'_np_version_under1p10',
'_np_version_under1p11',
'_np_version_under1p12',
'_np_version_under1p13',
'_np_version_under1p14',
'_np_version_under1p15'
Expand Down
19 changes: 2 additions & 17 deletions pandas/core/algorithms.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
ensure_platform_int, ensure_object,
ensure_float64, ensure_uint64,
ensure_int64)
from pandas.compat.numpy import _np_version_under1p10
from pandas.core.dtypes.missing import isna, na_value_for_dtype

from pandas.core import common as com
Expand Down Expand Up @@ -910,26 +909,12 @@ def checked_add_with_arr(arr, b, arr_mask=None, b_mask=None):
------
OverflowError if any x + y exceeds the maximum or minimum int64 value.
"""
def _broadcast(arr_or_scalar, shape):
"""
Helper function to broadcast arrays / scalars to the desired shape.
"""
if _np_version_under1p10:
if is_scalar(arr_or_scalar):
out = np.empty(shape)
out.fill(arr_or_scalar)
else:
out = arr_or_scalar
else:
out = np.broadcast_to(arr_or_scalar, shape)
return out

# For performance reasons, we broadcast 'b' to the new array 'b2'
# so that it has the same size as 'arr'.
b2 = _broadcast(b, arr.shape)
b2 = np.broadcast_to(b, arr.shape)
if b_mask is not None:
# We do the same broadcasting for b_mask as well.
b2_mask = _broadcast(b_mask, arr.shape)
b2_mask = np.broadcast_to(b_mask, arr.shape)
else:
b2_mask = None

Expand Down
2 changes: 1 addition & 1 deletion pandas/core/computation/check.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from distutils.version import LooseVersion

_NUMEXPR_INSTALLED = False
_MIN_NUMEXPR_VERSION = "2.4.6"
_MIN_NUMEXPR_VERSION = "2.6.1"

try:
import numexpr as ne
Expand Down
7 changes: 1 addition & 6 deletions pandas/core/ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -1805,12 +1805,7 @@ def to_series(right):

elif right.shape[0] == left.shape[0] and right.shape[1] == 1:
# Broadcast across columns
try:
right = np.broadcast_to(right, left.shape)
except AttributeError:
# numpy < 1.10.0
right = np.tile(right, (1, left.shape[1]))

right = np.broadcast_to(right, left.shape)
right = left._constructor(right,
index=left.index,
columns=left.columns)
Expand Down
9 changes: 1 addition & 8 deletions pandas/plotting/_compat.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,8 @@ def inner():
return inner


_mpl_ge_1_2_1 = _mpl_version('1.2.1', operator.ge)
_mpl_le_1_2_1 = _mpl_version('1.2.1', operator.le)
_mpl_ge_1_3_1 = _mpl_version('1.3.1', operator.ge)
_mpl_ge_1_4_0 = _mpl_version('1.4.0', operator.ge)
_mpl_ge_1_4_1 = _mpl_version('1.4.1', operator.ge)
_mpl_ge_1_5_0 = _mpl_version('1.5.0', operator.ge)
_mpl_ge_2_0_0 = _mpl_version('2.0.0', operator.ge)
_mpl_le_2_0_0 = _mpl_version('2.0.0', operator.le)
_mpl_ge_2_0_1 = _mpl_version('2.0.1', operator.ge)
_mpl_ge_2_1_0 = _mpl_version('2.1.0', operator.ge)
_mpl_ge_2_2_0 = _mpl_version('2.2.0', operator.ge)
_mpl_ge_2_2_2 = _mpl_version('2.2.2', operator.ge)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don’t add this as it’s confusing
we have a certain pattern stick to it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed _mpl_le_3_0_0 as it was an error anyway (however, there used to be _mpl_le_2_0_0 until this PR)...

_mpl_ge_3_0_0 = _mpl_version('3.0.0', operator.ge)
9 changes: 0 additions & 9 deletions pandas/plotting/_converter.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,6 @@
from pandas.tseries.frequencies import FreqGroup
from pandas.core.indexes.period import Period, PeriodIndex

from pandas.plotting._compat import _mpl_le_2_0_0

# constants
HOURS_PER_DAY = 24.
MIN_PER_HOUR = 60.
Expand Down Expand Up @@ -371,13 +369,6 @@ def __init__(self, locator, tz=None, defaultfmt='%Y-%m-%d'):
if self._tz is dates.UTC:
self._tz._utcoffset = self._tz.utcoffset(None)

# For mpl > 2.0 the format strings are controlled via rcparams
# so do not mess with them. For mpl < 2.0 change the second
# break point and add a musec break point
if _mpl_le_2_0_0():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since mpl has a min version of 2.0.0 don't we still need this check?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was probably being used incorrectly. I don't think it would apply to 2.0.0 but not 2.0.1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made sure to test all available matplotlib versions, and while the removal of the code for 2.0.0 was an oversight, nothing in the test suite failed. As far as I can tell, 2.0.0 wasn't tested in the CI before, which is also the reason that several if-switches before this PR were strictly speaking wrong (when I removed the compat code in a way that just removed pre-2.0 branches, suddenly there were failures).

self.scaled[1. / SEC_PER_DAY] = '%H:%M:%S'
self.scaled[1. / MUSEC_PER_DAY] = '%H:%M:%S.%f'


class PandasAutoDateLocator(dates.AutoDateLocator):

Expand Down
Loading