BUG: transpose casts mixed dtypes to object #43340
Conversation
debnathshoham
commented
Sep 1, 2021
debnathshoham
commented
- closes BUG: transpose unnecessarily returning object #43337
- tests added / passed
- Ensure all linting tests pass, see here for how to run them
- whatsnew entry
|
All the failures are from the below two, not sure why .These are passing in some cases (also in my local). |
Based on a quick look, initial guess might be different |
Best guess is something weird is going on with find_common_type between dt64tz[US/Eastern] and dt64tz[US/Pacific]. I think find_common_type should return object, but in some places we pretend that the common type is dt64tz[UTC] |
|
ok if you can merge master will look again |
| @@ -57,6 +58,7 @@ def test_transpose_object_to_tzaware_mixed_tz(self): | |||
| df2 = DataFrame([dti, dti2]) | |||
| assert (df2.dtypes == object).all() | |||
| res2 = df2.T | |||
| print("\n", res2.dtypes, [dti.dtype, dti2.dtype]) | |||
There was a problem hiding this comment.
removed the prints
these are the only two tests failing (but not on my local, was trying to figure why)
| @@ -501,6 +501,7 @@ Reshaping | |||
| - Improved error message when creating a :class:`DataFrame` column from a multi-dimensional :class:`numpy.ndarray` (:issue:`42463`) | |||
| - :func:`concat` creating :class:`MultiIndex` with duplicate level entries when concatenating a :class:`DataFrame` with duplicates in :class:`Index` and multiple keys (:issue:`42651`) | |||
| - Bug in :meth:`pandas.cut` on :class:`Series` with duplicate indices (:issue:`42185`) and non-exact :meth:`pandas.CategoricalIndex` (:issue:`42425`) | |||
| - Bug in :meth:`DataFrame.transpose`, where mixed dtypes were cast to ``object`` (:issue:`43337`) | |||
There was a problem hiding this comment.
this is saying the opposite of what you are doing. make this more clear to a reader
| new_arr = new_arr.copy() | ||
| result = self._constructor(new_arr, index=self.columns, columns=self.index) | ||
| common_dtype = find_common_type(dtypes) if len(dtypes) > 0 else None | ||
| result = self._constructor( |
There was a problem hiding this comment.
pass copy to the constructor (and are you testing this)?
| @@ -103,3 +103,10 @@ def test_transpose_get_view_dt64tzget_view(self): | |||
|
|
|||
| rtrip = result._mgr.blocks[0].values | |||
| assert np.shares_memory(arr._data, rtrip._data) | |||
|
|
|||
| def test_transpose_mixed_dtypes(self): | |||
There was a problem hiding this comment.
test the copy keyword. test an example which does keep object (e.g. (1, 1.5) and ( 'a', 1)) for (a, b) and test with simple dtypes as well (e.g. int64)
|
this very likley changes a number of tests you will have to edit those |
|
This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this. |
|
status of this? |
|
sorry, was not able to give time into this.. |
|
Thanks @debnathshoham but appears this PR has gone stale. When you have the time feel free to merge master and we can reopen. Closing for now. |
