BUG: fix Series.interpolate(method='index') with unsorted index #29943
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jreback
requested changes
Dec 1, 2019
pandas/tests/series/test_missing.py
Outdated
| @@ -1655,3 +1655,10 @@ def test_interpolate_timedelta_index(self, interp_methods_ind): | |||
| pytest.skip( | |||
| "This interpolation method is not supported for Timedelta Index yet." | |||
| ) | |||
|
|
|||
| def test_interpolate_unsorted_index(self): | |||
There was a problem hiding this comment.
can you add all of the examples from the OP.
There was a problem hiding this comment.
you can likely parameterize this test
pandas/tests/series/test_missing.py
Outdated
| # GH 21037 | ||
| ts = pd.Series(data=[10, 9, np.nan, 2, 1], index=[10, 9, 3, 2, 1]) | ||
| result = ts.interpolate(method="index").sort_index(ascending=True) | ||
| expected = ts.sort_index(ascending=True).interpolate(method="index") |
There was a problem hiding this comment.
can you hard-code the expected result (e.g. don't use interpolate)
pandas/core/missing.py
Outdated
| @@ -277,7 +277,10 @@ def interpolate_1d( | |||
| inds = lib.maybe_convert_objects(inds) | |||
| else: | |||
| inds = xvalues | |||
| result[invalid] = np.interp(inds[invalid], inds[valid], yvalues[valid]) | |||
| seq = np.argsort(inds[valid]) | |||
There was a problem hiding this comment.
can you add a comment here on what you are doing
pandas/core/missing.py
Outdated
| @@ -277,7 +277,10 @@ def interpolate_1d( | |||
| inds = lib.maybe_convert_objects(inds) | |||
| else: | |||
| inds = xvalues | |||
| result[invalid] = np.interp(inds[invalid], inds[valid], yvalues[valid]) | |||
| seq = np.argsort(inds[valid]) | |||
doc/source/whatsnew/v1.0.0.rst
Outdated
| @@ -550,6 +550,7 @@ Numeric | |||
| - Bug in :class:`UInt64Index` precision loss while constructing from a list with values in the ``np.uint64`` range (:issue:`29526`) | |||
| - Bug in :class:`NumericIndex` construction that caused indexing to fail when integers in the ``np.uint64`` range were used (:issue:`28023`) | |||
| - Bug in :class:`NumericIndex` construction that caused :class:`UInt64Index` to be casted to :class:`Float64Index` when integers in the ``np.uint64`` range were used to index a :class:`DataFrame` (:issue:`28279`) | |||
| - Bug in :meth:`Series.interpolate` when using method ``index`` with unsorted index. (:issue:`21037`) | |||
There was a problem hiding this comment.
method=`index` with an unsorted index, would previously return incorrect results.
|
Hi @jreback, all done, please have a look. |
jreback
approved these changes
Dec 3, 2019
|
lgtm. @TomAugspurger if any comments |
TomAugspurger
approved these changes
Dec 3, 2019
|
Thanks! |
proost
pushed a commit
to proost/pandas
that referenced
this pull request
Dec 19, 2019
…as-dev#29943) * fix series interpolate bug with unsorted index GH21037
proost
pushed a commit
to proost/pandas
that referenced
this pull request
Dec 19, 2019
…as-dev#29943) * fix series interpolate bug with unsorted index GH21037
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Series.interpolate(method='index')use numpy.interp under the hood, which cannot work with unsorted X values. Here we sort the known data fornumpy.interp.black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff