-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
Support downcasting of nullable arrays #33435
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
1582a7b
3063398
2c26aed
b506521
5501a99
159f96d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -143,9 +143,7 @@ def maybe_downcast_to_dtype(result, dtype): | |
|
||
else: | ||
dtype = "object" | ||
|
||
dtype = np.dtype(dtype) | ||
|
||
converted = maybe_downcast_numeric(result, dtype, do_round) | ||
if converted is not result: | ||
return converted | ||
|
@@ -180,7 +178,8 @@ def maybe_downcast_to_dtype(result, dtype): | |
|
||
def maybe_downcast_numeric(result, dtype, do_round: bool = False): | ||
""" | ||
Subset of maybe_downcast_to_dtype restricted to numeric dtypes. | ||
Subset of maybe_downcast_to_dtype restricted to numeric and | ||
nullable dtypes. | ||
|
||
Parameters | ||
---------- | ||
|
@@ -210,9 +209,7 @@ def trans(x): | |
# don't allow upcasts here (except if empty) | ||
if result.dtype.itemsize <= dtype.itemsize and result.size: | ||
return result | ||
|
||
if is_bool_dtype(dtype) or is_integer_dtype(dtype): | ||
|
||
if not result.size: | ||
# if we don't have any elements, just astype it | ||
return trans(result).astype(dtype) | ||
|
@@ -239,7 +236,9 @@ def trans(x): | |
if (new_result == result).all(): | ||
return new_result | ||
else: | ||
if np.allclose(new_result, result, rtol=0): | ||
# np.allclose raises TypeError on extension arrays | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm not sure about this. I think allclose should still work and maybe that is the root cause instead, but let's see what others think There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i agree , we might need to enable soething on EAs to make this work, but that is the root cause There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you elaborate on enabling something on EAs? Would this be a new feature to it, or a current feature? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what happens when this is on an EA (e.g. show an exampel with this line specifically) |
||
nd_result = np.array(result).astype(result[0].dtype) | ||
if np.allclose(new_result, nd_result, rtol=0): | ||
return new_result | ||
|
||
elif ( | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -649,3 +649,14 @@ def test_failure_to_convert_uint64_string_to_NaN(): | |
ser = Series([32, 64, np.nan]) | ||
result = to_numeric(pd.Series(["32", "64", "uint64"]), errors="coerce") | ||
tm.assert_series_equal(result, ser) | ||
|
||
|
||
def test_support_downcast_of_nullable_dtypes(): | ||
# GH 33013 | ||
try: | ||
pd.to_numeric(pd.Series([1, 2, 3], dtype="Int32"), downcast="integer") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you parametrize these instead? |
||
pd.to_numeric(pd.Series([1, 2, 3], dtype="Int64"), downcast="integer") | ||
pd.to_numeric(pd.Series([1, 2], dtype="Int32"), downcast="signed") | ||
pd.to_numeric(pd.Series([1, 2, 3], dtype="Int32"), downcast="float") | ||
except TypeError: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Rather than the try...except you would ideally do |
||
pytest.fail("TypeError raised.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you add the issue number
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
move to 1.2