BUG/API: Indexes on empty frames/series should be RangeIndex #49637
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
|
@pandas-dev/pandas-core any comment? IMO, this simplify the construction logic nicely, so will be good to get in. |
|
I think this is also a sensible change for 2.0.0. |
|
this doesn't break any tests? |
|
@jreback , this break a lot of tests, as empty series/frames are in a lot of tests. So I wanted to know it if others approve of this general idea/change, before really going through the tests. |
|
I would be supportive of this change; IMO it would needs its own section in the 2.0 whatsnew |
|
closes #40077? |
topper-123
force-pushed
the
zero-len_frames_series_rangeindex
branch
from
f224dad to
4237e62
Compare
|
Still need docs + some cleanup, otherwise is ready if passes the CI. |
|
This is ready for review now. |
mroeschke
reviewed
Nov 29, 2022
mroeschke
reviewed
Nov 29, 2022
topper-123
force-pushed
the
zero-len_frames_series_rangeindex
branch
from
9effc51 to
36f3630
Compare
|
Updated. |
mroeschke
reviewed
Nov 29, 2022
mroeschke
reviewed
Nov 29, 2022
|
Updated. |
mroeschke
reviewed
Dec 1, 2022
mroeschke
reviewed
Dec 1, 2022
topper-123
force-pushed
the
zero-len_frames_series_rangeindex
branch
from
3ccd284 to
f6e5f0e
Compare
|
Rebased & updated. |
Co-authored-by: Matthew Roeschke <[email protected]>
topper-123
force-pushed
the
zero-len_frames_series_rangeindex
branch
from
ba9bc75 to
131dfef
Compare
|
Gentle ping. I've rebased (no code changes) just in case. |
|
Thanks @topper-123 |
3 tasks
3 tasks
3 tasks
Closed
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've looked into #49572 and it seems that the fix is reasonable simple, see included code. In short, currently if the user hasn't supplied
indexorcolumnsvalues, then the index/columns is aRangeIndexif the data has lenght > 0, while they'reIndex[object]if the length is 0. After this PR, aRangeIndexwill be used in both cases. This will simplify type inference etc. testing etc.However, this fix requires changes to a lot of tests (>500), of which I've fixed about 100 (not included in this version of the PR), so it's quite a lot of work to get this fixed up. Test fixing so far seems to be only a matter of ensuring that various empty dataframes/Series have
RangeIndexrather than aIndex[object], so relatively simple, but also tedious work.I didn't get much response to #49572 from the core devs, so before I spend more time on this rather big item, could you guys chime in on if you agree that this a good thing to do? IMO having the same index dtypes for series/frame of length 0 and >0 will conceptually simplify pandas and decrease the amount of surprises that users will encounter.
I've included the code that fixes the issue in this PR, as mentioned, and ATM this PR fails, because I haven't fixed the tests, but I will get them fixed if I get positive response. If you can respond either here or in #49572, I'll of course read them both.