DataFrame.quantile and DataFrame.describe Not Handling NaN

[chris-sheehan]

Code

>>> import numpy as np
>>> import pandas as pd
>>> df = DataFrame([1, 2, 3, np.nan], columns = ['x'])
>>> print df.x.quantile(0.5)

...outputs NaN, where would expect it to handle NaN values and return 2.0

Versions

pandas: 0.18.1
numpy: 1.11.0

In NumPy 1.10.1, the behavior of numpy.median and numpy.percentile are changed to not handle NaN...

Median warns and returns nan when invalid values are encountered... Similar to mean, median and percentile now emits a Runtime warning and returns NaN in slices where a NaN is present. To compute the median or percentile while ignoring invalid values use the new nanmedian or nanpercentile functions.

Since pandas.DataFrame uses numpy.percentile for .describe and .quantile, neither handle NaN values when paired with numpy >= 1.10.0.

[jorisvandenbossche]

@chris-sheehan Thanks for reporting! This has been fixed in the meantime in the development version. See #13098