COMPAT/TST: fix group_info dtype issues, xref #10981

[jreback]

closes #10981

[jreback]

@behzadnouri it may be less readable, but I didn't have the time to debug. welcome your assistance with that. these win- issues are annoying. if you can spin up a vm would be great (I do this on macosx in fact ,with a win vm).

[behzadnouri]

if u post the traceback i can look into.

[jreback]

I took master and reverted that commit

[pandas] c:\Users\Jeff Reback\pandas>git revert  54f788a1c545c6b028dcd63a776d9f4fd7c9dbae -m 1
[master aa34c62] Revert "Merge pull request #10988 from jreback/winfixes"
 2 files changed, 7 insertions(+), 7 deletions(-)

[pandas] c:\Users\Jeff Reback\pandas>git diff HEAD~1
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
index f34fd6e..8f8ce31 100644
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -1811,13 +1811,13 @@ class BinGrouper(BaseGrouper):
     @cache_readonly
     def group_info(self):
         ngroups = self.ngroups
-        obs_group_ids = np.arange(ngroups, dtype='int64')
+        obs_group_ids = np.arange(ngroups)
         rep = np.diff(np.r_[0, self.bins])

         if ngroups == len(self.bins):
-            comp_ids = np.repeat(np.arange(ngroups, dtype='int64'), rep)
+            comp_ids = np.repeat(np.arange(ngroups), rep)
         else:
-            comp_ids = np.repeat(np.r_[-1, np.arange(ngroups, dtype='int64')], rep)
+            comp_ids = np.repeat(np.r_[-1, np.arange(ngroups)], rep)

         return comp_ids, obs_group_ids, ngroups

@@ -2570,8 +2570,8 @@ class SeriesGroupBy(GroupBy):

         # group boundries are where group ids change
         # unique observations are where sorted values change
-        idx = com._ensure_int64(np.r_[0, 1 + np.nonzero(ids[1:] != ids[:-1])[0]])
-        inc = com._ensure_int64(np.r_[1, val[1:] != val[:-1]])
+        idx = np.r_[0, 1 + np.nonzero(ids[1:] != ids[:-1])[0]]
+        inc = np.r_[1, val[1:] != val[:-1]]

         # 1st item of each group is a new unique observation
         mask = isnull(val)
diff --git a/pandas/tseries/tests/test_resample.py b/pandas/tseries/tests/test_resample.py
index ec03d55..49d3446 100644
--- a/pandas/tseries/tests/test_resample.py
+++ b/pandas/tseries/tests/test_resample.py
@@ -919,7 +919,7 @@ class TestResample(tm.TestCase):
     def test_resample_group_info(self):  # GH10914
         for n, k in product((10000, 100000), (10, 100, 1000)):
             dr = date_range(start='2015-08-27', periods=n // 10, freq='T')
-            ts = Series(np.random.randint(0, n // k, n).astype('int64'),
+            ts = Series(np.random.randint(0, n // k, n),
                         index=np.random.choice(dr, n))

             left = ts.resample('30T', how='nunique')
@@ -1585,7 +1585,7 @@ class TestTimeGrouper(tm.TestCase):
         # check TimeGrouper's aggregation is identical as normal groupby

         n = 20

errors

[pandas] c:\Users\Jeff Reback\pandas>nosetests pandas/tseries/tests/test_resample.py
...E.E.E....E......FE.EFE....EEEE...E..........E.............EE....E...E..............EF.....E
======================================================================
ERROR: test_custom_grouper (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 54, in test_custom_grouper
    g._cython_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1150, in _cython_agg_general
    result, names = self.grouper.aggregate(obj.values, how)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 6743, in pandas.algos.group_add_float64 (pandas\algos.c:125163)
    def group_add_float64(ndarray[float64_t, ndim=2] out,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_how_lambda_functions (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 850, in test_how_lambda_functions
    bar_exp = ts.resample('M', how='std')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 792, in std
    return np.sqrt(self.var(ddof=ddof))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 801, in var
    return self._cython_agg_general('var')
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1150, in _cython_agg_general
    result, names = self.grouper.aggregate(obj.values, how)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 6988, in pandas.algos.group_var_float64 (pandas\algos.c:128550)
    def group_var_float64(ndarray[float64_t, ndim=2] out,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_ohlc_5min (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 504, in test_ohlc_5min
    label='right')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 837, in ohlc
    lambda x: x._cython_agg_general('ohlc'))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2717, in _apply_to_column_groupbys
    return func(self)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 837, in <lambda>
    lambda x: x._cython_agg_general('ohlc'))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1150, in _cython_agg_general
    result, names = self.grouper.aggregate(obj.values, how)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 7195, in pandas.algos.group_ohlc_float64 (pandas\algos.c:131795)
    def group_ohlc_float64(ndarray[float64_t, ndim=2] out,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_resample_axis1 (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 539, in test_resample_axis1
    result = df.resample('M', axis=1)
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2814, in aggregate
    return getattr(self, arg)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1168, in _python_agg_general
    result, counts = self.grouper.agg_series(obj, f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1880, in agg_series
    return grouper.get_result()
  File "pandas\src\reduce.pyx", line 274, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35764)
    raise
  File "pandas\src\reduce.pyx", line 260, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35604)
    res = self.f(cached_typ)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1162, in <lambda>
    f = lambda x: func(x, *args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 763, in <lambda>
    f = lambda x: x.mean(axis=self.axis)
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 4540, in stat_func
    skipna=skipna, numeric_only=numeric_only)
  File "c:\Users\Jeff Reback\pandas\pandas\core\series.py", line 2177, in _reduce
    self._get_axis_number(axis)
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 314, in _get_axis_number
    .format(axis, type(self)))
ValueError: No axis named 1 for object type <class 'pandas.core.series.Series'>

======================================================================
ERROR: test_resample_empty (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 688, in test_resample_empty
    empty_series.resample('d', how)
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1168, in _python_agg_general
    result, counts = self.grouper.agg_series(obj, f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1880, in agg_series
    return grouper.get_result()
  File "pandas\src\reduce.pyx", line 227, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:34986)
    counts[0] = self.bins[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

======================================================================
ERROR: test_resample_frame_basic (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 322, in test_resample_frame_basic
    g._cython_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2739, in _cython_agg_general
    new_items, new_blocks = self._cython_agg_blocks(how, numeric_only=numeric_only)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2775, in _cython_agg_blocks
    result, _ = self.grouper.aggregate(block.values, how, axis=agg_axis)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 6743, in pandas.algos.group_add_float64 (pandas\algos.c:125163)
    def group_add_float64(ndarray[float64_t, ndim=2] out,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_resample_how (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 131, in test_resample_how
    closed='right', label='right')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 792, in std
    return np.sqrt(self.var(ddof=ddof))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 801, in var
    return self._cython_agg_general('var')
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1150, in _cython_agg_general
    result, names = self.grouper.aggregate(obj.values, how)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 6988, in pandas.algos.group_var_float64 (pandas\algos.c:128550)
    def group_var_float64(ndarray[float64_t, ndim=2] out,
ValueError: ("Buffer dtype mismatch, expected 'int64_t' but got 'long'", 'how=std')

======================================================================
ERROR: test_resample_ohlc (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 406, in test_resample_ohlc
    result = s.resample('5Min', how='ohlc')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 837, in ohlc
    lambda x: x._cython_agg_general('ohlc'))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2717, in _apply_to_column_groupbys
    return func(self)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 837, in <lambda>
    lambda x: x._cython_agg_general('ohlc'))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1150, in _cython_agg_general
    result, names = self.grouper.aggregate(obj.values, how)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 7195, in pandas.algos.group_ohlc_float64 (pandas\algos.c:131795)
    def group_ohlc_float64(ndarray[float64_t, ndim=2] out,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_resample_ohlc_dataframe (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 431, in test_resample_ohlc_dataframe
    res = df.resample('H', how='ohlc')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 315, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2814, in aggregate
    return getattr(self, arg)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 837, in ohlc
    lambda x: x._cython_agg_general('ohlc'))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3488, in _apply_to_column_groupbys
    keys=self._selected_obj.columns, axis=1)
  File "c:\Users\Jeff Reback\pandas\pandas\tools\merge.py", line 809, in concat
    copy=copy)
  File "c:\Users\Jeff Reback\pandas\pandas\tools\merge.py", line 839, in __init__
    objs = list(objs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3486, in <genexpr>
    (func(col_groupby) for _, col_groupby
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 837, in <lambda>
    lambda x: x._cython_agg_general('ohlc'))
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1150, in _cython_agg_general
    result, names = self.grouper.aggregate(obj.values, how)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 7195, in pandas.algos.group_ohlc_float64 (pandas\algos.c:131795)
    def group_ohlc_float64(ndarray[float64_t, ndim=2] out,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_resample_panel (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 552, in test_resample_panel
    result = panel.resample('M', axis=1)
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3548, in aggregate
    return getattr(self, arg)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1166, in _python_agg_general
    for name, obj in self._iterate_slices():
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3524, in _iterate_slices
    raise NotImplementedError("axis other than 0 is not supported")
NotImplementedError: axis other than 0 is not supported

======================================================================
ERROR: test_resample_panel_numpy (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 578, in test_resample_panel_numpy
    expected = panel.resample('M', how='mean', axis=1)
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 287, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3548, in aggregate
    return getattr(self, arg)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1166, in _python_agg_general
    for name, obj in self._iterate_slices():
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3524, in _iterate_slices
    raise NotImplementedError("axis other than 0 is not supported")
NotImplementedError: axis other than 0 is not supported

======================================================================
ERROR: test_resample_timegrouper (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 901, in test_resample_timegrouper
    result = df.set_index('A').resample('M', how='count')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 315, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2814, in aggregate
    return getattr(self, arg)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3506, in count
    return self._wrap_agged_blocks(data.items, list(blk))
  File "pandas\lib.pyx", line 1258, in pandas.lib.count_level_2d (pandas\lib.c:22712)
    def count_level_2d(ndarray[uint8_t, ndim=2, cast=True] mask,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_all_values_single_bin (pandas.tseries.tests.test_resample.TestResamplePeriodIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1411, in test_all_values_single_bin
    result = s.resample("A", how='mean')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 92, in resample
    rs = self._resample_periods()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 357, in _resample_periods
    return grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1168, in _python_agg_general
    result, counts = self.grouper.agg_series(obj, f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1880, in agg_series
    return grouper.get_result()
  File "pandas\src\reduce.pyx", line 274, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35764)
    raise
  File "pandas\src\reduce.pyx", line 249, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35283)
    cached_ityp = self.ityp(islider.buf)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 185, in __new__
    ordinal, freq = cls._from_arraylike(data, freq, tz)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 251, in _from_arraylike
    raise ValueError('freq not specified and cannot be '
ValueError: freq not specified and cannot be inferred from first element

======================================================================
ERROR: test_basic_downsample (pandas.tseries.tests.test_resample.TestResamplePeriodIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1063, in test_basic_downsample
    result = ts.resample('a-dec')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 92, in resample
    rs = self._resample_periods()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 357, in _resample_periods
    return grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1168, in _python_agg_general
    result, counts = self.grouper.agg_series(obj, f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1880, in agg_series
    return grouper.get_result()
  File "pandas\src\reduce.pyx", line 274, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35764)
    raise
  File "pandas\src\reduce.pyx", line 249, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35283)
    cached_ityp = self.ityp(islider.buf)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 185, in __new__
    ordinal, freq = cls._from_arraylike(data, freq, tz)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 251, in _from_arraylike
    raise ValueError('freq not specified and cannot be '
ValueError: freq not specified and cannot be inferred from first element

======================================================================
ERROR: test_basic_upsample (pandas.tseries.tests.test_resample.TestResamplePeriodIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1084, in test_basic_upsample
    result = ts.resample('a-dec')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 92, in resample
    rs = self._resample_periods()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 357, in _resample_periods
    return grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1168, in _python_agg_general
    result, counts = self.grouper.agg_series(obj, f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1880, in agg_series
    return grouper.get_result()
  File "pandas\src\reduce.pyx", line 274, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35764)
    raise
  File "pandas\src\reduce.pyx", line 249, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35283)
    cached_ityp = self.ityp(islider.buf)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 185, in __new__
    ordinal, freq = cls._from_arraylike(data, freq, tz)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 251, in _from_arraylike
    raise ValueError('freq not specified and cannot be '
ValueError: freq not specified and cannot be inferred from first element

======================================================================
ERROR: test_evenly_divisible_with_no_extra_bins (pandas.tseries.tests.test_resample.TestResamplePeriodIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1437, in test_evenly_divisible_with_no_extra_bins
    result = df.resample('7D', how='count')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 82, in resample
    rs = self._resample_timestamps()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 315, in _resample_timestamps
    result = grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2814, in aggregate
    return getattr(self, arg)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3506, in count
    return self._wrap_agged_blocks(data.items, list(blk))
  File "pandas\lib.pyx", line 1258, in pandas.lib.count_level_2d (pandas\lib.c:22712)
    def count_level_2d(ndarray[uint8_t, ndim=2, cast=True] mask,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_quarterly_resampling (pandas.tseries.tests.test_resample.TestResamplePeriodIndex)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1344, in test_quarterly_resampling
    result = ts.resample('A')
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 92, in resample
    rs = self._resample_periods()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 357, in _resample_periods
    return grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2343, in aggregate
    return getattr(self, func_or_funcs)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1168, in _python_agg_general
    result, counts = self.grouper.agg_series(obj, f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1880, in agg_series
    return grouper.get_result()
  File "pandas\src\reduce.pyx", line 274, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35764)
    raise
  File "pandas\src\reduce.pyx", line 249, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35283)
    cached_ityp = self.ityp(islider.buf)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 185, in __new__
    ordinal, freq = cls._from_arraylike(data, freq, tz)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\period.py", line 251, in _from_arraylike
    raise ValueError('freq not specified and cannot be '
ValueError: freq not specified and cannot be inferred from first element

======================================================================
ERROR: test_aggregate_normal (pandas.tseries.tests.test_resample.TestTimeGrouper)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1552, in test_aggregate_normal
    dt_result = getattr(dt_grouped, func)()
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 801, in var
    return self._cython_agg_general('var')
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2739, in _cython_agg_general
    new_items, new_blocks = self._cython_agg_blocks(how, numeric_only=numeric_only)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2775, in _cython_agg_blocks
    result, _ = self.grouper.aggregate(block.values, how, axis=agg_axis)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1584, in aggregate
    result = self._aggregate(result, counts, values, agg_func, is_numeric)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1629, in _aggregate
    agg_func(result, counts, values, comp_ids)
  File "pandas\src\generated.pyx", line 6988, in pandas.algos.group_var_float64 (pandas\algos.c:128550)
    def group_var_float64(ndarray[float64_t, ndim=2] out,
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

======================================================================
ERROR: test_panel_aggregation (pandas.tseries.tests.test_resample.TestTimeGrouper)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1512, in test_panel_aggregation
    binagg = bingrouped.mean()
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1166, in _python_agg_general
    for name, obj in self._iterate_slices():
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 3524, in _iterate_slices
    raise NotImplementedError("axis other than 0 is not supported")
NotImplementedError: axis other than 0 is not supported

======================================================================
FAIL: test_resample_dup_index (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 454, in test_resample_dup_index
    result = df.resample('Q',axis=1)
  File "c:\Users\Jeff Reback\pandas\pandas\core\generic.py", line 3517, in resample
    return sampler.resample(self).__finalize__(self)
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 92, in resample
    rs = self._resample_periods()
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\resample.py", line 357, in _resample_periods
    return grouped.aggregate(self._agg_method)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 2814, in aggregate
    return getattr(self, arg)(*args, **kwargs)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 764, in mean
    return self._python_agg_general(f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1168, in _python_agg_general
    result, counts = self.grouper.agg_series(obj, f)
  File "c:\Users\Jeff Reback\pandas\pandas\core\groupby.py", line 1880, in agg_series
    return grouper.get_result()
  File "pandas\src\reduce.pyx", line 238, in pandas.lib.SeriesBinGrouper.get_result (pandas\lib.c:35159)
    vslider = Slider(self.arr, self.dummy_arr)
  File "pandas\src\reduce.pyx", line 434, in pandas.lib.Slider.__init__ (pandas\lib.c:40127)
    assert(values.ndim == 1)
AssertionError

======================================================================
FAIL: test_resample_group_info (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 942, in test_resample_group_info
    assert_series_equal(left, right)
  File "c:\Users\Jeff Reback\pandas\pandas\util\testing.py", line 899, in assert_series_equal
    assert_attr_equal('dtype', left, right)
  File "c:\Users\Jeff Reback\pandas\pandas\util\testing.py", line 701, in assert_attr_equal
    left_attr, right_attr)
  File "c:\Users\Jeff Reback\pandas\pandas\util\testing.py", line 791, in raise_assert_detail
    raise AssertionError(msg)
AssertionError: Attributes are different

Attribute "dtype" are different
[left]:  int32
[right]: int64

======================================================================
FAIL: test_aggregate_with_nat (pandas.tseries.tests.test_resample.TestTimeGrouper)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\Users\Jeff Reback\pandas\pandas\tseries\tests\test_resample.py", line 1607, in test_aggregate_with_nat
    assert_frame_equal(expected, dt_result)
  File "c:\Users\Jeff Reback\pandas\pandas\util\testing.py", line 1021, in assert_frame_equal
    obj='DataFrame.iloc[:, {0}]'.format(i))
  File "c:\Users\Jeff Reback\pandas\pandas\util\testing.py", line 918, in assert_series_equal
    check_less_precise, obj='{0}'.format(obj))
  File "pandas\src\testing.pyx", line 58, in pandas._testing.assert_almost_equal (pandas\src\testing.c:3809)
    cpdef assert_almost_equal(a, b, bint check_less_precise=False,
  File "pandas\src\testing.pyx", line 147, in pandas._testing.assert_almost_equal (pandas\src\testing.c:2685)
    raise_assert_detail(obj, msg, lobj, robj)
  File "c:\Users\Jeff Reback\pandas\pandas\util\testing.py", line 791, in raise_assert_detail
    raise AssertionError(msg)
AssertionError: DataFrame.iloc[:, 0] are different

DataFrame.iloc[:, 0] values are different (20.0 %)
[left]:  [2.49734811743, 1.55654204842, nan, -1.34556660251, -0.346241152977]
[right]: [2.49734811743, 1.55654204842, 0.0, -1.34556660251, -0.346241152977]

----------------------------------------------------------------------
Ran 94 tests in 5.413s

[behzadnouri]

you need int64 in BinGrouper.group_info

but not in GroupBy.nunique

[jreback]

======================================================================
FAIL: test_resample_group_info (pandas.tseries.tests.test_resample.TestResample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff Reback\pandas3\pandas\tseries\tests\test_resample.py", line 942, in test_resample_group_info
    assert_series_equal(left, right)
  File "C:\Users\Jeff Reback\pandas3\pandas\util\testing.py", line 899, in assert_series_equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\Jeff Reback\pandas3\pandas\util\testing.py", line 701, in assert_attr_equal
    left_attr, right_attr)
  File "C:\Users\Jeff Reback\pandas3\pandas\util\testing.py", line 791, in raise_assert_detail
    raise AssertionError(msg)
AssertionError: Attributes are different

Attribute "dtype" are different
[left]:  int32
[right]: int64

----------------------------------------------------------------------
Ran 94 tests in 28.655s

FAILED (failures=1)

[pandas3] C:\Users\Jeff Reback\pandas3>nosetests pandas\tests\test_groupby.py
........................................................................................................................................................................................................
............................F...........................
======================================================================
FAIL: test_series_groupby_nunique (pandas.tests.test_groupby.TestGroupBy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff Reback\pandas3\pandas\tests\test_groupby.py", line 1646, in test_series_groupby_nunique
    check_nunique(frame, ['jim'])
  File "C:\Users\Jeff Reback\pandas3\pandas\tests\test_groupby.py", line 1636, in check_nunique
    assert_series_equal(left, right)
  File "C:\Users\Jeff Reback\pandas3\pandas\util\testing.py", line 899, in assert_series_equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\Jeff Reback\pandas3\pandas\util\testing.py", line 701, in assert_attr_equal
    left_attr, right_attr)
  File "C:\Users\Jeff Reback\pandas3\pandas\util\testing.py", line 791, in raise_assert_detail
    raise AssertionError(msg)
AssertionError: Attributes are different

Attribute "dtype" are different
[left]:  int32
[right]: int64

----------------------------------------------------------------------
Ran 256 tests in 56.689s

FAILED (failures=1)

[pandas3] C:\Users\Jeff Reback\pandas3>git diff HEAD~1
diff --git a/pandas/core/groupby.py b/pandas/core/groupby.py
index f34fd6e..bea27cb 100644
--- a/pandas/core/groupby.py
+++ b/pandas/core/groupby.py
@@ -2570,8 +2570,8 @@ class SeriesGroupBy(GroupBy):

         # group boundries are where group ids change
         # unique observations are where sorted values change
-        idx = com._ensure_int64(np.r_[0, 1 + np.nonzero(ids[1:] != ids[:-1])[0]])
-        inc = com._ensure_int64(np.r_[1, val[1:] != val[:-1]])
+        idx = np.r_[0, 1 + np.nonzero(ids[1:] != ids[:-1])[0]]
+        inc = np.r_[1, val[1:] != val[:-1]]

         # 1st item of each group is a new unique observation
         mask = isnull(val)

[pandas3] C:\Users\Jeff Reback\pandas3>

[behzadnouri]

the type which comes out of numpy calls is correct. nunique values should have the same types as array of indices, which is always NPY_INTP in numpy.

these tests are not failing in cython calls or in the actual values. they are failing only because the constructed expected right frame/series is set to have int64 type. you may fix the expected right value in the tests to have np.intp.

[jreback]

from test_resample_group_info

completely disagree. You HAVE to return int64 for indexers. we don't use platform indexer anywhere. So they must be converted. This is true in all other code, why would would you argue against it here? (esp since this is such a general routine).

its irrelevant what numpy returns. Pandas should always return the same dtype as input except if an upcast is needed. If I pass in int64 then I expect int64 out *regardless of the platform.

(Pdb) p ts.dtype
dtype('int64')
(Pdb) p ts.resample('30T',how='nunique').dtype
dtype('int32')

[behzadnouri]

its irrelevant what numpy returns

that is why there are so many ugly com._ensure_this, com._ensure_that code bloat throughout pandas. numpy is the backend, how can that be irrelevant? and hasn't been enough pain manually chasing type mismatches here and there?

should always return the same dtype as input

nunique is always int type. the input can be anything, it can be string, it can be object. regardless of what the input is nunique returns ints. how can u argue about type of nunique output based on ts.dtype? what if ts.dtype was object or float?

You HAVE to return int64 for indexers.

are you sure? cause multi-index labels are more indexer than nuinque:

>>> MultiIndex.from_arrays([[1, 2], [3, 4]]).labels[0]
FrozenNDArray([0, 1], dtype='int8')

[jreback]

that is why there are so many ugly com._ensure_this, com._ensure_that code bloat throughout pandas. numpy is the backend, how can that be irrelevant? and hasn't been enough pain manually chasing type mismatches here and there?

I agree, but that is a completely separate issue to do that. You cannot simply change it. It STILL has to work on windows

nunique is always int type. the input can be anything, it can be string, it can be object. regardless of what the input is nunique returns ints. how can u argue about type of nunique output based on ts.dtype? what if ts.dtype was object or float?

We ALWAYS return int64 for counting types.

[jreback]

(Pdb) df = DataFrame({'A' : [1,2]})
(Pdb) df.count()
A    2
dtype: int64
(Pdb) df.A.value_counts()
2    1
1    1
Name: A, dtype: int64

[jreback]

certainly welcome work on this: #3033

but as I said, that is a bit non-trivial and out-of-scope for a fix here.

[behzadnouri]

We ALWAYS return int64 for counting types.

well you are doing it wrong. if one place it needs to be forced int64 is in the multi-index/factorize labels, because there it is really needed to avoid overflow in groupby & join methods and cython calls. but, there you are forcing smaller integer types for indexers in a method called _coerce_**indexer**_dtype.

for unique/count i do not see why it has to deviate from numpy.

[jreback]

the point is that it is a long-time consistency to return data to the users in int64 (and float64) when dtype is unspecified. and count and value_counts already do this.

We certainly could change this. But again that is out of scope for this issue, and further that would have to be an announced API change. You are changing the user facing API in a potentially unexpected way. It might be ok, it might not. But I think it would be wise to actually push this into master and have people tests it for a while.

As far as what is used internally, #3033 covers this. I am certainly open to cleaning this up. I think everyone can agree its fairly ugly and when it was done there were other considerations.