BUG: mean overflows for integer dtypes (fixes #10155)

[mortada]

closes #10155

[shoyer]

For float32 input, the sum should still be float32.

[mortada]

oh ... you are right. This means we actually have more to fix than the cases for int types. Because dtype_max would be float64 even when the inputs are float32... I'll update shortly.

[mortada]

There are technically two changes now:

old behavior

1. For int dtypes, mean() first sums up values in int and converts to float, therefore it can potential have int overflow
2. For float dtypes, mean() sums up values in float64 and returns the result in float64, even if the input dtype is a lower precision dtype such as float32

new behavior

1. For int dtypes, mean() sums up values in float64 and returns the result as float64
2. For float dtypes, mean() sums up values using the same input float dtype and returns the result in the same input dtype. Namely, float32 input will have both the computation and return value in float32

Both points in the old behavior are not consistent with numpy, and both points in the new behavior are.

[shoyer]

LGTM... just needs a release note

[jreback]

ok - in another issue should audit the rest of the nan funcs for overflow / dtype preserving effects

• lets just put in place some tests

[mortada]

So I was actually quite puzzled by why the unit tests wouldn't pass for some of the virtual environments. And I think I tracked down the problem ... seems like a bug in numpy < 1.9.0. For python 3.4 this is not a problem because numpy >= 1.9.0 is required, but for other environments with older numpy versions this can fail

(python2):~/code/github/pandas$ nosetests -s pandas/tests/test_nanops.py 
............S.S....S....F....SS...
======================================================================
FAIL: test_nanmean_overflow (pandas.tests.test_nanops.TestnanopsDataFrame)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mortada/code/github/pandas/pandas/tests/test_nanops.py", line 336, in test_nanmean_overflow
    self.assertEqual(result, a)
AssertionError: 20150515061816464.0 != 20150515061816532

I verified that np.mean() produces this wrong value for older versions prior to 1.9.0.

[mortada]

Ok I've updated this with a numpy version check, travis should pass now. Also added a release note.

[jreback]

why are you changing the count?

[mortada]

oh that's because for float dtypes count can be higher precision and therefore changing the dtype of the returned result. E.g. for float32 input count is still float64 and therefore the result would be cast to float64 when we do sum / count, and that's not what we want

[jreback]

shouldn't count always be integer type?

[jreback]

hmm, seems we are casting that, don't really remember why. You can try changing get_count to simply return an int64 which I think will always be correct.

[mortada]

@jreback actually changing count to int64 may not help, this is what I'm seeing:

In [1]: import numpy as np

In [2]: (np.float32(100) / np.int64(100)).dtype
Out[2]: dtype('float64')

In [3]: (np.float32(100) / np.int32(100)).dtype
Out[3]: dtype('float64')

In [4]: (np.float32(100) / np.float32(100)).dtype
Out[4]: dtype('float32')

It still gets cast to float64 unless we have the denominator as float32 as well.

[jreback]

hmm, ok.

maybe better to add pass thru the dtype to get_counts then as well (and cast to that inside get_counts), to avoid this specific code all over the place

[mortada]

sure sounds good, will update shortly

[mortada]

@jreback I added an optional dtype parameter to _get_counts() as you suggested

[mortada]

@jreback @shoyer this should be ready for review.

I'll create another issue/PR to audit the rest of the nan funcs for overflow and returned dtypes

[jreback]

what does numpy do in < 1.9.0?

[mortada]

for numpy < 1.9.0: (wrong result)

In [1]: import numpy as np

In [2]: np.__version__
Out[2]: '1.8.2'

In [3]: a = 20150515061816532

In [4]: arr = np.array(np.ones(500) * a, dtype=np.int64)

In [5]: arr.mean()
Out[5]: 20150515061816464.0

numpy >= 1.9.0: (correct result)

In [1]: import numpy as np

In [2]: np.__version__
Out[2]: '1.9.0'

In [3]: a = 20150515061816532

In [4]: arr = np.array(np.ones(500) * a, dtype=np.int64)

In [5]: arr.mean()
Out[5]: 20150515061816532.0

[jreback]

oh, ok

[shoyer]

Indeed, I think this is good to go... I'll wait a little bit and then merge.

[jreback]

yep, lgtm

[shoyer]

thanks @mortada !

[mortada]

cool thanks guys!