Remove unused, avoid uses of deprecated api #22071

jbrockmendel · 2018-07-27T00:18:38Z

core.dtypes.common.is_period is confusingly named and never used. This PR gets rid of it.

The deprecated numpy API that causes zillions of warnings includes ndarray.data, so this replaces the old usage with the encouraged usage.

A bunch of tslibs.util isn't actually used in tslibs.util, so the relevant functions are moved up to _libs.util.

Docstrings for most of what's left in tslibs.util.

Simplify util._checknull by removing an unnecessary try/except.

codecov · 2018-07-27T19:10:50Z

Codecov Report

Merging #22071 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #22071   +/-   ##
=======================================
  Coverage   92.07%   92.07%           
=======================================
  Files         170      170           
  Lines       50690    50690           
=======================================
  Hits        46672    46672           
  Misses       4018     4018

Flag	Coverage Δ
#multiple	`90.48% <ø> (ø)`	⬆️
#single	`42.31% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d30c4a0...e6fa730. Read the comment docs.

jreback · 2018-07-28T13:19:35Z

pandas/_libs/src/numpy_helper.h

 void set_array_not_contiguous(PyArrayObject* ao) {
-    ao->flags &= ~(NPY_ARRAY_C_CONTIGUOUS | NPY_ARRAY_F_CONTIGUOUS);
+    // Numpy>=1.8-compliant equivalent to:


we are numpy >= 1.9! can this be simplified?

No. The existing implementation uses the deprecated numpy API.

jreback · 2018-07-28T13:21:24Z

pandas/_libs/tslibs/util.pxd

@@ -1,10 +1,18 @@
-from numpy cimport ndarray


wait, we have a tslibs/util.pxd and a util.pxd?

When util.pxd was moved from src to tslibs, we added _libs/util.pxd as an alias so as to keep existing cimports

jbrockmendel · 2018-07-28T17:11:14Z

we are numpy >= 1.9! can this be simplified?

An update/background on the ever-present numpy deprecation warnings. Based on conversation on the cython tracker it looks like

a) numpy more or less forgot why they did that deprecation, have no plans to follow up on it
b) this could be fixed in cython; I'll keep an eye on it but no one should hold their breath
c) if this is fixed in cython, pandas has a handful of places that uses the old API that will need to be updated (many/most of which this PR does)

AFAICT we get the deprecation warning if we do one or more of:
i) cimport numpy as cnp
ii) from numpy cimport ...
iii) cdef extern from ... any file that has an #include <numpy/...>
iii) b) unless we add to that file the lines:

#ifndef
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#endif

iii) c) But any C file we add this to becomes off-limits for cython files that have numpy cimports (or else we get compile-time errors)

The upshot being: there are a bunch of files for which we can get rid of these warnings.

Many of the files only have numpy cimports because they need to a) call import_array in order to use util.is_integer_object, b) need int64_t, or c) need ndarray.

These can all be worked around if we get numpy cimports out of util (which this PR does)

jreback · 2018-07-29T15:57:02Z

pandas/_libs/tslibs/util.pxd

 from cpython cimport PyTypeObject

+cdef extern from *:


we don't need this in util.pxd? (the higher level one)?

does this change perf?

does this change perf?

No.

we don't need this in util.pxd? (the higher level one)?

a) everything in this file is cimported into the other one, so it is available there. b) No, this is only used by tslibs.period.

jreback · 2018-07-29T15:59:01Z

pandas/core/dtypes/common.py

@@ -304,34 +304,6 @@ def is_offsetlike(arr_or_obj):
    return False


-def is_period(arr):


is this not generally exposed to the user / higher level? I don't remember why we added this (and it has the wrong name)

jreback · 2018-07-29T15:59:39Z

pandas/core/dtypes/common.py

-    Parameters
-    ----------
-    arr : array-like
-        The array-like to check.


unfortunately I think we need to deprecate this (and remove in next version)

isn't this change orthogonal to this PR?

isn't this change orthogonal to this PR?

Sure, can be separated out. This started out as a grab-bag PR.

unfortunately I think we need to deprecate this (and remove in next version)

Darn, OK.

jreback · 2018-07-30T10:28:57Z

the warnings in the log seem more than before on interval.pyx. but could be just me.

jbrockmendel · 2018-07-30T15:44:02Z

the warnings in the log seem more than before on interval.pyx. but could be just me.

A couple PRs back that fixed a bunch of warnings, changes to interval.pyx were reverted because they caused failures on windows. So this file has been cleaned up much less than the others, might seem like more in comparison.

jreback · 2018-07-31T13:24:32Z

thanks!

jbrockmendel added 10 commits July 26, 2018 11:50

remove unused is_period

f9a95e7

move unused utils to _libs.util, docstrings

4e05a07

remove usages of deprecated API

d1cfac8

docstring

cf84856

whitespace, docstring cleanup

abda192

use util.is_array

6d8c335

update import

9360a85

update test

ca05636

remove unused import

1f326e2

avoid need for cnp in util

82228e1

gfyoung added Internals Related to non-user accessible pandas implementation Clean labels Jul 27, 2018

gfyoung requested a review from jreback July 27, 2018 04:38

jbrockmendel added 2 commits July 27, 2018 12:08

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm

1dc6fff

troubleshoot appveyor failure

d318cc8

jbrockmendel added 2 commits July 27, 2018 13:20

try getting NPY_MIN_INT64 directly

d25070d

disentangle util from src, docstring

bed3eed

jbrockmendel force-pushed the rm branch from 10d4d00 to bed3eed Compare July 28, 2018 00:08

dummy commit to allow push

6320a2c

jreback requested changes Jul 28, 2018

View reviewed changes

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm

2650222

jreback requested changes Jul 29, 2018

View reviewed changes

jbrockmendel added 3 commits July 29, 2018 10:21

revert removal of is_period

d952272

Merge branch 'master' of https://github.com/pandas-dev/pandas into rm

559a461

revert is_period changes

e6fa730

jreback added this to the 0.24.0 milestone Jul 30, 2018

jreback approved these changes Jul 31, 2018

View reviewed changes

jreback merged commit 48d4a1c into pandas-dev:master Jul 31, 2018

jbrockmendel deleted the rm branch July 31, 2018 16:00

dberenbaum pushed a commit to dberenbaum/pandas that referenced this pull request Aug 3, 2018

Remove unused, avoid uses of deprecated api (pandas-dev#22071)

e6c7c01

Sup3rGeo pushed a commit to Sup3rGeo/pandas that referenced this pull request Oct 1, 2018

Remove unused, avoid uses of deprecated api (pandas-dev#22071)

40a3c97

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove unused, avoid uses of deprecated api #22071

Remove unused, avoid uses of deprecated api #22071

jbrockmendel commented Jul 27, 2018

codecov bot commented Jul 27, 2018 •

edited

Loading

jreback Jul 28, 2018

jbrockmendel Jul 28, 2018

jreback Jul 28, 2018

jbrockmendel Jul 28, 2018

jbrockmendel commented Jul 28, 2018

jreback Jul 29, 2018

jreback Jul 29, 2018

jbrockmendel Jul 29, 2018

jreback Jul 29, 2018

jreback Jul 29, 2018

jreback Jul 29, 2018

jbrockmendel Jul 29, 2018

jreback commented Jul 30, 2018

jbrockmendel commented Jul 30, 2018

jreback commented Jul 31, 2018

		@@ -304,34 +304,6 @@ def is_offsetlike(arr_or_obj):
		return False


		def is_period(arr):

Remove unused, avoid uses of deprecated api #22071

Remove unused, avoid uses of deprecated api #22071

Conversation

jbrockmendel commented Jul 27, 2018

codecov bot commented Jul 27, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbrockmendel commented Jul 28, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jul 30, 2018

jbrockmendel commented Jul 30, 2018

jreback commented Jul 31, 2018

codecov bot commented Jul 27, 2018 •

edited

Loading