Skip to content

CLN/BUG: Consolidate Index.astype and fix tz aware bugs #18937

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 27, 2017
Merged

CLN/BUG: Consolidate Index.astype and fix tz aware bugs #18937

merged 2 commits into from
Dec 27, 2017

Conversation

jschendel
Copy link
Member

@jschendel jschendel commented Dec 25, 2017 •
edited
Loading

Only behavioral changes:

@codecov
Copy link

codecov bot commented Dec 25, 2017 •
edited
Loading

Codecov Report

Merging #18937 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18937      +/-   ##
==========================================
- Coverage   91.59%   91.59%   -0.01%     
==========================================
  Files         150      150              
  Lines       48964    48939      -25     
==========================================
- Hits        44850    44826      -24     
+ Misses       4114     4113       -1
Flag Coverage Δ
#multiple 89.95% <100%> (-0.01%) ⬇️
#single 41.16% <51.11%> (+0.02%) ⬆️
Impacted Files Coverage Δ
pandas/core/indexes/datetimes.py 95.57% <100%> (-0.05%) ⬇️
pandas/core/indexes/interval.py 93.81% <100%> (-0.08%) ⬇️
pandas/core/indexes/base.py 96.45% <100%> (+0.01%) ⬆️
pandas/core/indexes/datetimelike.py 97.19% <100%> (+0.06%) ⬆️
pandas/core/indexes/numeric.py 97.28% <100%> (-0.09%) ⬇️
pandas/core/indexes/period.py 93% <100%> (+0.06%) ⬆️
pandas/core/indexes/timedeltas.py 91.08% <100%> (-0.18%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a18d725...d7c7b9d. Read the comment docs.

sinhrks
sinhrks reviewed Dec 25, 2017
View reviewed changes
pandas/core/indexes/datetimes.py Outdated
return Index(self.values.astype('i8', copy=copy), name=self.name,
dtype='i8')
elif is_datetime64_ns_dtype(dtype):
if is_datetime64_ns_dtype(dtype):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can it be merged to datetimelike using is_dtype_equal(self.dtype, dtype)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch. I think it can be merged all the way down to base.

@sinhrks sinhrks added Clean Dtype Conversions Unexpected or buggy dtype conversions labels Dec 25, 2017
@jreback jreback added this to the 0.23.0 milestone Dec 26, 2017
@jreback
Copy link
Contributor

jreback commented Dec 26, 2017

lgtm. ex @sinhrks comment. ping when resolved.

@pep8speaks
Copy link

pep8speaks commented Dec 27, 2017 •
edited
Loading

Hello @jschendel! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on December 27, 2017 at 04:17 Hours UTC

@jschendel jschendel changed the title CLN: Consolidate Index.astype CLN/BUG: Consolidate Index.astype and fix tz aware bugs Dec 27, 2017
@jschendel
Copy link
Member Author

jschendel commented Dec 27, 2017 •
edited
Loading

Made the change suggested by @sinhrks, and fixed #18951 at the same time, since it was closely tied to code I was changing without any other dependencies.

Bringing is_dtype_equal(self.dtype, dtype) check down to base.py appears to have slightly changed the behavior of RangeIndex. Previous RangeIndex.astype('int64') returned an Int64Index, but with the change it now remains a RangeIndex. This seems fine to me, but not 100% sure.

Bringing is_dtype_equal(self.dtype, dtype) check down to base.py should also allow for further cleaning of IntervalIndex.astype, but there are some larger issues there that don't allow it to work right now. Specifically, IntervalDtype(*) == 'interval' returns False, unlike CategoricalDtype, which breaks things when relying on is_dtype_equal and trying .astype('interval'). Casting between subtypes also doesn't work, e.g IntervalDtype('int64') to IntervalDtype('float64'). These issues seem independent from the changes being made here, and big enough to require a separate PR.

@jreback jreback merged commit edfb784 into pandas-dev:master Dec 27, 2017
@jreback
Copy link
Contributor

jreback commented Dec 27, 2017

thanks @jschendel nice patch!

keep em coming!

@jschendel jschendel deleted the cln-index-astype branch December 27, 2017 20:12
hexgnu pushed a commit to hexgnu/pandas that referenced this pull request Dec 28, 2017
@jschendel @hexgnu
CLN/BUG: Consolidate Index.astype and fix tz aware bugs (pandas-dev#1…
e577e2f 
…8937)
@jorisvandenbossche jorisvandenbossche mentioned this pull request Oct 14, 2020
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sinhrks sinhrks sinhrks left review comments

Assignees
No one assigned
Labels
Clean Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Milestone
0.23.0
Development

Successfully merging this pull request may close these issues.

BUG: DatetimeIndex.astype doesn't fully handle tz aware dtypes CLN: consolidate Index.astype
4 participants
@jschendel @jreback @pep8speaks @sinhrks