Skip to content

TST: Series repr with pyarrow date32 type #48258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 37 commits into from

Conversation

mroeschke
Copy link
Member

@mroeschke mroeschke commented Aug 25, 2022

@mroeschke mroeschke added Output-Formatting __repr__ of pandas objects, to_string Dtype Conversions Unexpected or buggy dtype conversions Timezones Timezone data dtype Arrow pyarrow functionality labels Aug 25, 2022
@mroeschke mroeschke added this to the 1.5 milestone Aug 25, 2022
def kind(self) -> str:
return "M"

assert not com.is_datetime64tz_dtype(NotTZDtype())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could also check needs_i8_conversion here

@mroeschke
Copy link
Member Author

@jbrockmendel don't know if you've ever hit this while working on EA.argmin/max/sort. For some reason (seemly) unrelated type's, in this case timedelta, argmin/max/sort fail. I got a similar error when trying to define ArrowExtensionArray._values_for_argsort, but ended up removing it since I just could define ArrowExtensionArray.argmin/max/sort instead.

https://github.com/pandas-dev/pandas/runs/8027023184?check_suite_focus=true

2022-08-26T00:21:38.2857760Z =================================== FAILURES ===================================
2022-08-26T00:21:38.2858430Z _________________ TestIndexReductions.test_minmax_timedelta64 __________________
2022-08-26T00:21:38.2860770Z [gw0] darwin -- Python 3.9.13 /Users/runner/micromamba/envs/test/bin/python
2022-08-26T00:21:38.2861240Z 
2022-08-26T00:21:38.2861500Z obj = array([ 86400000000000, 172800000000000, 259200000000000],
2022-08-26T00:21:38.2862360Z       dtype='timedelta64[ns]')
2022-08-26T00:21:38.2871110Z method = 'argmin', args = (), kwds = {'axis': None, 'out': None}
2022-08-26T00:21:38.2878500Z bound = <built-in method argmin of numpy.ndarray object at 0x178dea3f0>
2022-08-26T00:21:38.2879060Z 
2022-08-26T00:21:38.2879260Z     def _wrapfunc(obj, method, *args, **kwds):
2022-08-26T00:21:38.2892110Z         bound = getattr(obj, method, None)
2022-08-26T00:21:38.2892850Z         if bound is None:
2022-08-26T00:21:38.2893420Z             return _wrapit(obj, method, *args, **kwds)
2022-08-26T00:21:38.2894830Z     
2022-08-26T00:21:38.2896420Z         try:
2022-08-26T00:21:38.2896970Z >           return bound(*args, **kwds)
2022-08-26T00:21:38.2898150Z E           TypeError: Cannot cast array data from dtype('<m8[ns]') to dtype('<m8[us]') according to the rule 'safe'
2022-08-26T00:21:38.2898680Z 
2022-08-26T00:21:38.2899440Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:57: TypeError
2022-08-26T00:21:38.2899890Z 
2022-08-26T00:21:38.2900210Z During handling of the above exception, another exception occurred:
2022-08-26T00:21:38.2900610Z 
2022-08-26T00:21:38.2901140Z self = <pandas.tests.reductions.test_reductions.TestIndexReductions object at 0x15efe5d00>
2022-08-26T00:21:38.2901860Z 
2022-08-26T00:21:38.2902170Z     def test_minmax_timedelta64(self):
2022-08-26T00:21:38.2902670Z     
2022-08-26T00:21:38.2903110Z         # monotonic
2022-08-26T00:21:38.2903720Z         idx1 = TimedeltaIndex(["1 days", "2 days", "3 days"])
2022-08-26T00:21:38.2905060Z         assert idx1.is_monotonic_increasing
2022-08-26T00:21:38.2905630Z     
2022-08-26T00:21:38.2906250Z         # non-monotonic
2022-08-26T00:21:38.2906710Z         idx2 = TimedeltaIndex(["1 days", np.nan, "3 days", "NaT"])
2022-08-26T00:21:38.2907330Z         assert not idx2.is_monotonic_increasing
2022-08-26T00:21:38.2907860Z     
2022-08-26T00:21:38.2909520Z         for idx in [idx1, idx2]:
2022-08-26T00:21:38.2911780Z             assert idx.min() == Timedelta("1 days")
2022-08-26T00:21:38.2912570Z             assert idx.max() == Timedelta("3 days")
2022-08-26T00:21:38.2913080Z >           assert idx.argmin() == 0
2022-08-26T00:21:38.2913940Z 
2022-08-26T00:21:38.2914320Z pandas/tests/reductions/test_reductions.py:266: 
2022-08-26T00:21:38.2915080Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2022-08-26T00:21:38.2915810Z pandas/core/indexes/base.py:7175: in argmin
2022-08-26T00:21:38.2916930Z     return super().argmin(skipna=skipna)
2022-08-26T00:21:38.2917250Z pandas/core/base.py:713: in argmin
2022-08-26T00:21:38.2917510Z     return delegate.argmin()
2022-08-26T00:21:38.2917860Z pandas/core/arrays/_mixins.py:200: in argmin
2022-08-26T00:21:38.2918570Z     return nargminmax(self, "argmin", axis=axis)
2022-08-26T00:21:38.2919230Z pandas/core/sorting.py:480: in nargminmax
2022-08-26T00:21:38.2920470Z     return _nanargminmax(arr_values, mask, func)
2022-08-26T00:21:38.2920970Z pandas/core/sorting.py:491: in _nanargminmax
2022-08-26T00:21:38.2921390Z     return non_nan_idx[func(non_nans)]
2022-08-26T00:21:38.2921820Z <__array_function__ internals>:180: in argmin
2022-08-26T00:21:38.2922200Z     ???
2022-08-26T00:21:38.2923120Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:1312: in argmin
2022-08-26T00:21:38.2923920Z     return _wrapfunc(a, 'argmin', axis=axis, out=out, **kwds)
2022-08-26T00:21:38.2924780Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:66: in _wrapfunc
2022-08-26T00:21:38.2926060Z     return _wrapit(obj, method, *args, **kwds)
2022-08-26T00:21:38.2927600Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2022-08-26T00:21:38.2927880Z 
2022-08-26T00:21:38.2928170Z obj = array([ 86400000000000, 172800000000000, 259200000000000],
2022-08-26T00:21:38.2928850Z       dtype='timedelta64[ns]')
2022-08-26T00:21:38.2929530Z method = 'argmin', args = (), kwds = {'axis': None, 'out': None}
2022-08-26T00:21:38.2931450Z wrap = <built-in method __array_wrap__ of numpy.ndarray object at 0x178dea3f0>
2022-08-26T00:21:38.2935490Z 
2022-08-26T00:21:38.2935730Z     def _wrapit(obj, method, *args, **kwds):
2022-08-26T00:21:38.2936160Z         try:
2022-08-26T00:21:38.2936550Z             wrap = obj.__array_wrap__
2022-08-26T00:21:38.2941120Z         except AttributeError:
2022-08-26T00:21:38.2957840Z             wrap = None
2022-08-26T00:21:38.2958820Z >       result = getattr(asarray(obj), method)(*args, **kwds)
2022-08-26T00:21:38.2962350Z E       TypeError: Cannot cast array data from dtype('<m8[ns]') to dtype('<m8[us]') according to the rule 'safe'
2022-08-26T00:21:38.2962810Z 
2022-08-26T00:21:38.2963520Z ../../../micromamba/envs/test/lib/python3.9/site-packages/numpy/core/fromnumeric.py:43: TypeError

@jbrockmendel
Copy link
Member

i dont think ive seen that, no. is it flaky?

@jbrockmendel
Copy link
Member

[finger slipped on the wrong button]

@mroeschke
Copy link
Member Author

mroeschke commented Sep 8, 2022

i dont think ive seen that, no. is it flaky?

I don't think so since it appears on some specific builds. I've tried installing those environments on my laptop but can't replicating them.

I'll keep poking around. I suspect it has to do with some statefulness.

@mroeschke
Copy link
Member Author

mroeschke commented Sep 14, 2022

Noting if worked on the future: Disabling the pyarrow timedelta types in the tests 21d152c fixed the seemingly unrelated timedelta errors in #48258 (comment), so there may be some odd interaction between then for pyarrow version 5, 6, 7

@mroeschke mroeschke changed the title BUG: is_datetime64tz_dtype shortcut only for DatetimeTZDtype TST: Series repr with pyarrow date32 type Sep 14, 2022
@mroeschke mroeschke marked this pull request as draft September 15, 2022 00:37
@mroeschke mroeschke removed this from the 1.5 milestone Sep 15, 2022
@mroeschke
Copy link
Member Author

May revisit this later, but since the original issue has been address by another PR, closing

@mroeschke mroeschke closed this Sep 20, 2022
@mroeschke mroeschke deleted the bug/is_dttz_dtype branch September 20, 2022 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Arrow pyarrow functionality Dtype Conversions Unexpected or buggy dtype conversions Output-Formatting __repr__ of pandas objects, to_string Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: series created from pyarrow array with date32 type in 1.5rc0 can be created but not displayed
2 participants