Skip to content

BUG: uint64 series resampling #47301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
cengjx09 opened this issue Jun 10, 2022 · 2 comments
Closed
2 of 3 tasks

BUG: uint64 series resampling #47301

cengjx09 opened this issue Jun 10, 2022 · 2 comments
Labels
Bug Closing Candidate May be closeable, needs more eyeballs Resample resample method

Comments

@cengjx09
Copy link

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import datetime

period = '15s'
df = pd.DataFrame([["80000000", 1],
                   ["80101000", 100]],
                  columns=["Time", "Value"])
df['DateTime'] = df['Time'].apply(lambda s: datetime.datetime.strptime(str(s), '%H%M%S%f'))
df.set_index('DateTime', inplace=True)
df["Value"] = df["Value"].astype('uint64')

df_rs = df['Value'].resample(period)
print(df_rs.first())

Issue Description

fail to access the first value of an resampled series with unit64 dtype . If dtype is changed to uint32 or int, the ouput is correct.

errors

File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\resample.py", line 1028, in f
return self._downsample(_method, min_count=min_count)
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\resample.py", line 1190, in _downsample
result = obj.groupby(self.grouper, axis=self.axis).aggregate(how, **kwargs)
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\generic.py", line 265, in aggregate
return getattr(self, func)(*args, **kwargs)
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 2241, in first
return self._agg_general(
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 1506, in _agg_general
result = self._cython_agg_general(
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 1592, in _cython_agg_general
new_mgr = data.grouped_reduce(array_func, ignore_failures=True)
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\internals\base.py", line 199, in grouped_reduce
res = func(arr)
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\groupby.py", line 1578, in array_func
result = self.grouper._cython_operation(
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\ops.py", line 939, in _cython_operation
return cy_op.cython_operation(
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\ops.py", line 626, in cython_operation
return self._cython_op_ndim_compat(
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\ops.py", line 451, in _cython_op_ndim_compat
res = self._call_cython_op(
File "C:\Users\86138\anaconda3\lib\site-packages\pandas\core\groupby\ops.py", line 544, in _call_cython_op
func(result, counts, values, comp_ids, min_count)
File "pandas_libs\groupby.pyx", line 1140, in pandas._libs.groupby.group_nth
RuntimeError: empty group with uint64_t

Expected Behavior

DateTime
1900-01-01 08:00:00 1.0
1900-01-01 08:00:15 NaN
1900-01-01 08:00:30 NaN
1900-01-01 08:00:45 NaN
1900-01-01 08:01:00 100.0
Freq: 15S, Name: Value, dtype: float64

Installed Versions

pandas : 1.4.2
numpy : 1.19.3
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 52.0.0.post20210125
Cython : 0.29.23
pytest : 6.2.3
hypothesis : None
sphinx : 4.0.1
blosc : None
feather : None
xlsxwriter : 1.3.8
lxml.etree : 4.6.3
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.22.0
pandas_datareader: 0.10.0
bs4 : 4.9.3
bottleneck : 1.3.2
brotli :
fastparquet : None
fsspec : 0.9.0
gcsfs : None
markupsafe : 1.1.1
matplotlib : 3.3.4
numba : 0.53.1
numexpr : 2.7.3
odfpy : None
openpyxl : 3.0.7
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.6.2
snappy : None
sqlalchemy : 1.4.7
tables : 3.6.1
tabulate : 0.8.9
xarray : None
xlrd : 2.0.1
xlwt : 1.3.0
zstandard : None
None

@cengjx09 cengjx09 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jun 10, 2022
@phofl phofl added the Resample resample method label Jun 14, 2022
simonjayhawkins added a commit to simonjayhawkins/pandas that referenced this issue Jun 15, 2022
@simonjayhawkins
Copy link
Member

it appears this was fixed in commit: [2d80d92] BUG: avoid RuntimeError in groupby.max (#46408) and test_resample_unsigned_int in pandas/tests/resample/test_datetime_index.py was updated.

so probably ok to close this without needing to add further tests

@simonjayhawkins simonjayhawkins added Closing Candidate May be closeable, needs more eyeballs and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jun 15, 2022
@phofl
Copy link
Member

phofl commented Jun 24, 2022

Yep agreed

@phofl phofl closed this as completed Jun 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Closing Candidate May be closeable, needs more eyeballs Resample resample method
Projects
None yet
Development

No branches or pull requests

3 participants