Skip to content

BUG: transpose inferring dtype for dt in object column #51565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 7, 2023

Conversation

phofl
Copy link
Member

@phofl phofl commented Feb 22, 2023

cc @jbrockmendel thoughts about doing it for 2.0 or 2.1?

@@ -63,7 +65,7 @@ def test_transpose_tzaware_2col_mixed_tz(self):
df4 = DataFrame({"A": dti, "B": dti2})
assert (df4.dtypes == [dti.dtype, dti2.dtype]).all()
assert (df4.T.dtypes == object).all()
tm.assert_frame_equal(df4.T.T, df4)
tm.assert_frame_equal(df4.T.T, df4, check_dtype=False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you cast the expected here to keep the test strict

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -464,7 +464,7 @@ def test_groupby_quantile_dt64tz_period():

# Check that we match the group-by-group result
exp = {i: df.iloc[i::5].quantile(0.5) for i in range(5)}
expected = DataFrame(exp).T
expected = DataFrame([x.tolist() for x in exp.values()], index=exp.keys())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could do .infer_objects()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea

@jbrockmendel
Copy link
Member

thoughts about doing it for 2.0 or 2.1?

no strong preference. i think we're doing more still-merging-to-2.0 in the RC this time around than we usually do, not sure how much is too much

@phofl
Copy link
Member Author

phofl commented Feb 22, 2023

Then lets do for 2.1, no strong preference either on my side

@phofl phofl added Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Feb 22, 2023
@phofl phofl added this to the 2.1 milestone Feb 22, 2023
@@ -1919,6 +1919,7 @@ def using_copy_on_write() -> bool:
"""
Fixture to check if Copy-on-Write is enabled.
"""
pd.options.mode.copy_on_write = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this intentional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope thx, already removed. Snuck in

result = self._constructor(new_arr, index=self.columns, columns=self.index)
result = self._constructor(
new_arr, index=self.columns, columns=self.index, dtype=new_arr.dtype
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to pass copy=False here (possibly just to be future-safe)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I’ll add it in the copy=True pr.

actually, it probably makes sense to go through all usages of constructor and pass copy=False for arrays. Will take a look how many there are

Copy link
Member

@jbrockmendel jbrockmendel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jbrockmendel jbrockmendel merged commit 27f7365 into pandas-dev:main Mar 7, 2023
@jbrockmendel
Copy link
Member

thanks @phofl

@phofl phofl deleted the transpose branch March 7, 2023 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Transpose changes dtype from object to timedelta/datetime
2 participants