REF: Simplify Index.union #41773
REF: Simplify Index.union #41773
Conversation
pandas/core/dtypes/cast.py
Outdated
| """ | ||
| Find a common data type among the given dtypes. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| types : list of dtypes | ||
| strict_uint64 : if True, object dtype is returned if uint64 and signed int present. |
There was a problem hiding this comment.
why do we not always do this?
There was a problem hiding this comment.
FWIW ive determined that this is the fix for #41775, though the place for the fix is in Index._find_common_type_compat
jreback
added
Dtype Conversions
pandas/core/indexes/base.py
Outdated
| # Now it's: | ||
| # * float | [u]int -> float | ||
| # * uint64 | signed int -> object | ||
| # We may change union(float | [u]int) to go to object. |
There was a problem hiding this comment.
i agree this logic doesn't belong here, but suggest putting it in Index._find_common_type_compat instead of higher up, to keep the Index casting logic in one place
|
I’ve updated to find_common_type_compat. |
|
looks fine can you rebase |
| if self.dtype == "uint64" or target_dtype == "uint64": | ||
| if is_signed_integer_dtype(self.dtype) or is_signed_integer_dtype( | ||
| target_dtype | ||
| ): |
There was a problem hiding this comment.
in principle we could check signed_one.min() and if its non-negative could avoid the object-dtype cast. not for this PR.
|
@topper-123 ping when ready here (then @jbrockmendel can rebase other) |
|
I’ve rebased. |
|
thanks @topper-123 |
This is the
Index.unionpart of #41153. This helps simplify that PR.The special casing in
Index.unionis currently active if both are numeric. After #41153 it should only be special cased if one dtype is uint64 and other a signed int. So after this and #41153:The first and second case is handled correctly by
find_common_type, but the others aren't currently.This PR changes no functionality itself, but prepares for the changes in #41153, where we want e.g.
NumericIndex[int8] .union(NumericIndex[uint32])to giveNumericIndex[int64]and notIndex[object].