Skip to content

BUG: Change of behavior in casting of datetime-like types in MultiIndex #44081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 12 commits into from

Conversation

shubham11941140
Copy link
Contributor

@shubham11941140 shubham11941140 commented Oct 18, 2021

Added try-except block in multi.py to reset the behaviour back to 1.2.5

@pep8speaks
Copy link

pep8speaks commented Oct 18, 2021

Hello @shubham11941140! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-10-28 01:57:38 UTC

Copy link
Member

@alimcmaster1 alimcmaster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR - some comments above



def test_set_index_MultiIndex():
import datetime as dt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imports should be at the top of the file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

name='date',
freq=None
)
for i in range(len(ex)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it, the issue is that if you print the expected and result, they are exactly the same but the assert_frame_equal is failing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tm.assert_index_equal works as I am comparing indexes and not DataFrames.

@@ -1673,6 +1673,12 @@ def get_level_values(self, level):
"""
level = self._get_level_number(level)
values = self._get_level_values(level)
import pandas as pd
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't need to import pandas here - can just import the to_datetime func?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -1673,6 +1673,12 @@ def get_level_values(self, level):
"""
level = self._get_level_number(level)
values = self._get_level_values(level)
import pandas as pd
try:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation doesn't seem right.. What are you trying to do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to ensure the behavior of DatetimeIndex reverts to 1.2.5

Copy link
Member

@phofl phofl Oct 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this seems to be a regression, please investigate what changed and caused the issue and try to restore the original behavior. Calling try except here should not be necessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed the try except block

@alimcmaster1
Copy link
Member

Please see contributing guide for help: https://pandas.pydata.org/pandas-docs/stable/development/contributing.html



def test_set_index_MultiIndex():
df = DataFrame(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since you're testing the DataFrame behavior, this probably goes in the tests/frame/methods file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually in this file, there are a lot of tests regarding setting of indexes, hence I have kept this file.

@@ -1,3 +1,4 @@
import datetime as dt
from datetime import datetime
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import date here instead of importing 'dt'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@shubham11941140 shubham11941140 requested a review from phofl October 27, 2021 17:47
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls look at the 1.3 release. I am pretty sure we change this on purpose. datetime.date is retained which is the point here.

@@ -764,7 +767,11 @@ def _shallow_copy(self: _IndexT, values, name: Hashable = no_default) -> _IndexT
name : Label, defaults to self.name
"""
name = self._name if name is no_default else name
u = self._simple_new(values, name=name)
if all(isinstance(x, date) for x in u):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this check absolutely doesn't belong here

@jbrockmendel
Copy link
Member

pls look at the 1.3 release. I am pretty sure we change this on purpose. datetime.date is retained which is the point here.

@jreback the change described in #43091 was caused by #38552, which was about Index([date]).astype("category").astype(object) round-tripping.

@simonjayhawkins
Copy link
Member

Thanks @shubham11941140 for the PR. These changes do not appear to correspond with the changes in the PR that was identified in the issue as causing the reported change in behavior.

We also need to decide if we are reverting to 1.2.5 behavior.

I am closing for now but feel free to contribute to the discussion in #43091

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Change of behavior in casting of datetime-like types in MultiIndex
7 participants