Skip to content

BUG: Change of behavior in casting of datetime-like types in MultiIndex #44081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 12 commits into from
Closed

BUG: Change of behavior in casting of datetime-like types in MultiIndex #44081

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
6423920
Follows 1.2.5 behaviour
shubham11941140 Oct 18, 2021
e04cf72
PEP 8 Issues
shubham11941140 Oct 18, 2021
92c6be6
Update test_reset_index.py
shubham11941140 Oct 18, 2021
715b580
Update test_reset_index.py
shubham11941140 Oct 18, 2021
44075c1
PEP8 issues
shubham11941140 Oct 18, 2021
57622f1
imports resolved
shubham11941140 Oct 18, 2021
fb9591a
black
shubham11941140 Oct 18, 2021
0576008
Solved import dependency
shubham11941140 Oct 27, 2021
28fb638
try-except removed
shubham11941140 Oct 27, 2021
92224e7
Solved Index type issue
shubham11941140 Oct 28, 2021
ccecca5
pre-commit removed
shubham11941140 Oct 28, 2021
5f6d795
import issue
shubham11941140 Oct 28, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions pandas/core/indexes/multi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1673,6 +1673,12 @@ def get_level_values(self, level):
"""
level = self._get_level_number(level)
values = self._get_level_values(level)
import pandas as pd
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't need to import pandas here - can just import the to_datetime func?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

try:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation doesn't seem right.. What are you trying to do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to ensure the behavior of DatetimeIndex reverts to 1.2.5

Copy link
Member

@phofl phofl Oct 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this seems to be a regression, please investigate what changed and caused the issue and try to restore the original behavior. Calling try except here should not be necessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed the try except block

values = pd.to_datetime(values)
except ValueError:
pass

return values

@doc(Index.unique)
Expand Down
24 changes: 24 additions & 0 deletions pandas/tests/series/methods/test_reset_index.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,3 +179,27 @@ def test_reset_index_dtypes_on_empty_series_with_multiindex(array, dtype):
{"level_0": np.int64, "level_1": np.float64, "level_2": dtype, 0: object}
)
tm.assert_series_equal(result, expected)


def test_set_index_MultiIndex():
import datetime as dt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imports should be at the top of the file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

df = DataFrame(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since you're testing the DataFrame behavior, this probably goes in the tests/frame/methods file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually in this file, there are a lot of tests regarding setting of indexes, hence I have kept this file.

{'date': [
dt.date(2021, 8, 1), dt.date(2021, 8, 2), dt.date(2021, 8, 3)
], 'ticker': ['aapl', 'goog', 'yhoo'], 'value': [
5.63269, 4.45609, 2.74843
]}
)

df.set_index(
['date', 'ticker'], inplace=True
)
res = df.index.get_level_values(0)
ex = pd.DatetimeIndex(
['2021-08-01', '2021-08-02', '2021-08-03'],
dtype='datetime64[ns]',
name='date',
freq=None
)
for i in range(len(ex)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use this to assert the df are equal:

https://pandas.pydata.org/docs/reference/api/pandas.testing.assert_frame_equal.html

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it, the issue is that if you print the expected and result, they are exactly the same but the assert_frame_equal is failing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tm.assert_index_equal works as I am comparing indexes and not DataFrames.

assert ex[i] == res[i]