Skip to content

BUG: maybe_infer_to_datetimelike incorrect on 2D inputs #23988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jbrockmendel opened this issue Nov 29, 2018 · 5 comments
Closed

BUG: maybe_infer_to_datetimelike incorrect on 2D inputs #23988

jbrockmendel opened this issue Nov 29, 2018 · 5 comments
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions

Comments

@jbrockmendel
Copy link
Member

Moved from discussion in #23730

>>> ts = pd.Timestamp.now().tz_localize('US/Pacific')
>>> data = np.array([[ts, ts, ts], [ts, ts, ts]])
>>> pd.DataFrame(data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/core/frame.py", line 416, in __init__
    copy=copy)
  File "pandas/core/frame.py", line 574, in _init_ndarray
    return create_block_manager_from_blocks([values], [columns, index])
  File "pandas/core/internals/managers.py", line 1665, in create_block_manager_from_blocks
    mgr = BlockManager(blocks, axes)
  File "pandas/core/internals/managers.py", line 118, in __init__
    self=self.ndim))
AssertionError: Number of Block dimensions (1) must equal number of axes (2)

This is because maybe_infer_to_datetimelike returns a DatetimeIndex (and not something 2D)

>>> from pandas.core.frame import maybe_infer_to_datetimelike
>>> maybe_infer_to_datetimelike(data)
DatetimeIndex(['2018-11-28 16:07:44.050944-08:00',
               '2018-11-28 16:07:44.050944-08:00',
               '2018-11-28 16:07:44.050944-08:00',
               '2018-11-28 16:07:44.050944-08:00',
               '2018-11-28 16:07:44.050944-08:00',
               '2018-11-28 16:07:44.050944-08:00'],
              dtype='datetime64[ns, US/Pacific]', freq=None)

cc @h-vetinari since you're already on a roll in this neighborhood

@mroeschke
Copy link
Member

I'm guessing this is specific to the timezone aware case and perpetuated by the fact that the DatetimeTZ block only stores 1D arrays.

@h-vetinari
Copy link
Contributor

Sorry I can't be of help here, haven't made it to maybe_infer_to_datetimelike yet (and I feel I have only the most rudimentary overview of the datetime/timedelta code)...

@mroeschke mroeschke added Bug Datetime Datetime data dtype Timezones Timezone data dtype labels Dec 2, 2018
@mroeschke
Copy link
Member

This looks to be fixed on master; was this fixed by #26825 or your EA refactoring @jbrockmendel?

In [16]: >>> ts = pd.Timestamp.now().tz_localize('US/Pacific')
    ...: >>> data = np.array([[ts, ts, ts], [ts, ts, ts]])
    ...: >>> pd.DataFrame(data)
Out[16]:
                                 0                                1                                2
0 2019-07-04 17:54:16.485999-07:00 2019-07-04 17:54:16.485999-07:00 2019-07-04 17:54:16.485999-07:00
1 2019-07-04 17:54:16.485999-07:00 2019-07-04 17:54:16.485999-07:00 2019-07-04 17:54:16.485999-07:00

In [18]: pd.__version__
Out[18]: '0.25.0rc0'

@jbrockmendel
Copy link
Member Author

26825 fixed the DataFrame constructor error. Haven't looked at if maybe_infer_to_datetimelike needs separate attention

@mroeschke mroeschke added good first issue Needs Tests Unit test(s) needed to prevent regressions and removed Bug Datetime Datetime data dtype Timezones Timezone data dtype labels Jan 26, 2020
@mroeschke
Copy link
Member

I think this issue was sufficiently closed by #26825, closing but happy to reopen if I misunderstood

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
Development

No branches or pull requests

3 participants