Skip to content

BUG: concat(axis=1) sorts by default for DatetimeIndex #54769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
lukemanley opened this issue Aug 26, 2023 · 4 comments · Fixed by #55085
Closed
3 tasks done

BUG: concat(axis=1) sorts by default for DatetimeIndex #54769

lukemanley opened this issue Aug 26, 2023 · 4 comments · Fixed by #55085
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@lukemanley
Copy link
Member

lukemanley commented Aug 26, 2023

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

a = pd.Series(1, index=pd.to_datetime(['2000-01-01', '2000-01-03']))
b = pd.Series(2, index=pd.to_datetime(['2000-01-02', '2000-01-04']))

pd.concat([a, b], axis=1)

Issue Description

concat documentation states concat will not sort by default. I believe the default sort behavior was deprecated some time ago. However, there is still a sort=True behavior for DatetimeIndex:

              0    1
2000-01-01  1.0  NaN
2000-01-02  NaN  2.0
2000-01-03  1.0  NaN
2000-01-04  NaN  2.0

The source of this behavior is a hard-coded special-casing in pandas.core.indexes.api.union_indexes. Making this not sort by default seems to break at least one test (test_concat_datetime_timezone) and possibly more.

Expected Behavior

Per the docs, I would expect no sort by default. But that might surprise users if changed. I suppose this either needs to be documented (if intentional) or deprecated (if not intentional).

Installed Versions

.

@lukemanley lukemanley added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Aug 26, 2023
@kshitiz305
Copy link

Hi I am a first time contributor would like to know if I shall update the docs or shall make changes to sort the bug

@lukemanley
Copy link
Member Author

Hi I am a first time contributor would like to know if I shall update the docs or shall make changes to sort the bug

Hey @kshitiz305, I think this needs a bit more discussion before determining which path to take.

@lukemanley
Copy link
Member Author

@phofl - just noticed you had opened #51683 for this issue. Any particular reason for closing it?

@phofl
Copy link
Member

phofl commented Sep 7, 2023

That kind of fell off my radar back then. You can continue if you are interested, the fix itself should be good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants