Skip to content

Implement Akima1DInterpolator #12833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion ci/requirements-3.5_OSX.run
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ openpyxl
xlsxwriter
xlrd
xlwt
scipy
numexpr
pytables
html5lib
Expand Down
1 change: 1 addition & 0 deletions doc/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -292,6 +292,7 @@
'matplotlib': ('http://matplotlib.org/', None),
'python': ('http://docs.python.org/3', None),
'numpy': ('http://docs.scipy.org/doc/numpy', None),
'scipy': ('http://docs.scipy.org/doc/scipy', None),
'py': ('http://pylib.readthedocs.org/en/latest/', None)
}
import glob
Expand Down
11 changes: 8 additions & 3 deletions doc/source/missing_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -392,9 +392,12 @@ The ``method`` argument gives access to fancier interpolation methods.
If you have scipy_ installed, you can set pass the name of a 1-d interpolation routine to ``method``.
You'll want to consult the full scipy interpolation documentation_ and reference guide_ for details.
The appropriate interpolation method will depend on the type of data you are working with.
For example, if you are dealing with a time series that is growing at an increasing rate,
``method='quadratic'`` may be appropriate. If you have values approximating a cumulative
distribution function, then ``method='pchip'`` should work well.

* If you are dealing with a time series that is growing at an increasing rate,
``method='quadratic'`` may be appropriate.
* If you have values approximating a cumulative distribution function,
then ``method='pchip'`` should work well.
* To fill missing values with goal of smooth plotting, use ``method='akima'``.

.. warning::

Expand All @@ -406,6 +409,8 @@ distribution function, then ``method='pchip'`` should work well.

df.interpolate(method='pchip')

df.interpolate(method='akima')

When interpolating via a polynomial or spline approximation, you must also specify
the degree or order of the approximation:

Expand Down
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.18.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ Other Enhancements
- ``pd.read_csv()`` now supports opening ZIP files that contains a single CSV, via extension inference or explict ``compression='zip'`` (:issue:`12175`)
- ``pd.read_csv()`` now supports opening files using xz compression, via extension inference or explicit ``compression='xz'`` is specified; ``xz`` compressions is also supported by ``DataFrame.to_csv`` in the same way (:issue:`11852`)
- ``pd.read_msgpack()`` now always gives writeable ndarrays even when compression is used (:issue:`12359`).
- ``interpolate()`` now supports ``method='akima'`` (:issue:`7588`).
- ``Index.take`` now handles ``allow_fill`` and ``fill_value`` consistently (:issue:`12631`)

.. ipython:: python
Expand Down
8 changes: 6 additions & 2 deletions pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -3451,7 +3451,8 @@ def interpolate(self, method='linear', axis=0, limit=None, inplace=False,
----------
method : {'linear', 'time', 'index', 'values', 'nearest', 'zero',
'slinear', 'quadratic', 'cubic', 'barycentric', 'krogh',
'polynomial', 'spline' 'piecewise_polynomial', 'pchip'}
'polynomial', 'spline' 'piecewise_polynomial', 'pchip',
'akima'}

* 'linear': ignore the index and treat the values as equally
spaced. This is the only method supported on MultiIndexes.
Expand All @@ -3465,13 +3466,16 @@ def interpolate(self, method='linear', axis=0, limit=None, inplace=False,
require that you also specify an `order` (int),
e.g. df.interpolate(method='polynomial', order=4).
These use the actual numerical values of the index.
* 'krogh', 'piecewise_polynomial', 'spline', and 'pchip' are all
* 'krogh', 'piecewise_polynomial', 'spline', 'pchip' and 'akima' are all
wrappers around the scipy interpolation methods of similar
names. These use the actual numerical values of the index. See
the scipy documentation for more on their behavior
`here <http://docs.scipy.org/doc/scipy/reference/interpolate.html#univariate-interpolation>`__ # noqa
`and here <http://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html>`__ # noqa

.. versionadded:: 0.18.1
Added support for the 'akima' method

axis : {0, 1}, default 0
* 0: fill column-by-column
* 1: fill row-by-row
Expand Down
71 changes: 64 additions & 7 deletions pandas/core/missing.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ def clean_interp_method(method, **kwargs):
order = kwargs.get('order')
valid = ['linear', 'time', 'index', 'values', 'nearest', 'zero', 'slinear',
'quadratic', 'cubic', 'barycentric', 'polynomial', 'krogh',
'piecewise_polynomial', 'pchip', 'spline']
'piecewise_polynomial', 'pchip', 'akima', 'spline']
if method in ('spline', 'polynomial') and order is None:
raise ValueError("You must specify the order of the spline or "
"polynomial.")
Expand Down Expand Up @@ -188,7 +188,7 @@ def _interp_limit(invalid, fw_limit, bw_limit):

sp_methods = ['nearest', 'zero', 'slinear', 'quadratic', 'cubic',
'barycentric', 'krogh', 'spline', 'polynomial',
'piecewise_polynomial', 'pchip']
'piecewise_polynomial', 'pchip', 'akima']
if method in sp_methods:
inds = np.asarray(xvalues)
# hack for DatetimeIndex, #1646
Expand Down Expand Up @@ -232,12 +232,19 @@ def _interpolate_scipy_wrapper(x, y, new_x, method, fill_value=None,
# GH 5975, scipy.interp1d can't hande datetime64s
x, new_x = x._values.astype('i8'), new_x.astype('i8')

try:
alt_methods['pchip'] = interpolate.pchip_interpolate
except AttributeError:
if method == 'pchip':
raise ImportError("Your version of scipy does not support "
if method == 'pchip':
try:
alt_methods['pchip'] = interpolate.pchip_interpolate
except AttributeError:
raise ImportError("Your version of Scipy does not support "
"PCHIP interpolation.")
elif method == 'akima':
try:
from scipy.interpolate import Akima1DInterpolator # noqa
alt_methods['akima'] = _akima_interpolate
except ImportError:
raise ImportError("Your version of Scipy does not support "
"Akima interpolation.")

interp1d_methods = ['nearest', 'zero', 'slinear', 'quadratic', 'cubic',
'polynomial']
Expand Down Expand Up @@ -267,6 +274,56 @@ def _interpolate_scipy_wrapper(x, y, new_x, method, fill_value=None,
return new_y


def _akima_interpolate(xi, yi, x, der=0, axis=0):
"""
Convenience function for akima interpolation.
xi and yi are arrays of values used to approximate some function f,
with ``yi = f(xi)``.

See `Akima1DInterpolator` for details.

Parameters
----------
xi : array_like
A sorted list of x-coordinates, of length N.
yi : array_like
A 1-D array of real values. `yi`'s length along the interpolation
axis must be equal to the length of `xi`. If N-D array, use axis
parameter to select correct axis.
x : scalar or array_like
Of length M.
der : int or list, optional
How many derivatives to extract; None for all potentially
nonzero derivatives (that is a number equal to the number
of points), or a list of derivatives to extract. This number
includes the function value as 0th derivative.
axis : int, optional
Axis in the yi array corresponding to the x-coordinate values.

See Also
--------
scipy.interpolate.Akima1DInterpolator

Returns
-------
y : scalar or array_like
The result, of length R or length M or M by R,

"""
from scipy import interpolate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you are asking here ...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move the import of the Akima1DInterpolate to _akima_interpolate.

And change how its handled in the alt_methods above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you still like to see a change around this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is ok

try:
P = interpolate.Akima1DInterpolator(xi, yi, axis=axis)
except TypeError:
# Scipy earlier than 0.17.0 missing axis
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so need a test for this. on the 2.7_LOCALE build we install a really old scipy. We may need to install a somewhat newer one on another build to explicity test for this. When did this Akima1DInterpolate appear in scipy? and seems the axis parm was added in 0.17.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.14.0 (as per scipy/scipy@a0a3b38) I implemented a different check.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok the 3.4 build has scipy==0.14.0 installed so pls verify that the test works corretctly (IOW it is NOT skipped, but passes).

remove scipy from ci/requirements-3.5_OSX. Make sure that you DO see some skips. I don't think we were testing having NO scipy.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok , just remove scipy from ci/requirements-3.5_OSX and I think will be good to go.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from ci/requirements-3.5_OSX.run ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep (we are including a version of scipy on EVERY build, so need to take it from one)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

P = interpolate.Akima1DInterpolator(xi, yi)
if der == 0:
return P(x)
elif interpolate._isscalar(der):
return P(x, der=der)
else:
return [P(x, nu) for nu in der]


def interpolate_2d(values, method='pad', axis=0, limit=None, fill_value=None,
dtype=None):
""" perform an actual interpolation of values, values will be make 2-d if
Expand Down
24 changes: 23 additions & 1 deletion pandas/tests/test_generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,13 @@ def _skip_if_no_pchip():
except ImportError:
raise nose.SkipTest('scipy.interpolate.pchip missing')


def _skip_if_no_akima():
try:
from scipy.interpolate import Akima1DInterpolator # noqa
except ImportError:
raise nose.SkipTest('scipy.interpolate.Akima1DInterpolator missing')

# ----------------------------------------------------------------------
# Generic types test cases

Expand Down Expand Up @@ -734,7 +741,7 @@ def test_interpolate(self):
non_ts[0] = np.NaN
self.assertRaises(ValueError, non_ts.interpolate, method='time')

def test_interp_regression(self):
def test_interpolate_pchip(self):
tm._skip_if_no_scipy()
_skip_if_no_pchip()

Expand All @@ -747,6 +754,21 @@ def test_interp_regression(self):
# does not blow up, GH5977
interp_s[49:51]

def test_interpolate_akima(self):
tm._skip_if_no_scipy()
_skip_if_no_akima()

ser = Series([10, 11, 12, 13])

expected = Series([11.00, 11.25, 11.50, 11.75,
12.00, 12.25, 12.50, 12.75, 13.00],
index=Index([1.0, 1.25, 1.5, 1.75,
2.0, 2.25, 2.5, 2.75, 3.0]))
# interpolate at new_index
new_index = ser.index.union(Index([1.25, 1.5, 1.75, 2.25, 2.5, 2.75]))
interp_s = ser.reindex(new_index).interpolate(method='akima')
assert_series_equal(interp_s[1:3], expected)

def test_interpolate_corners(self):
s = Series([np.nan, np.nan])
assert_series_equal(s.interpolate(), s)
Expand Down