Skip to content

CI/TST: Don't require length for construct_1d_arraylike_from_scalar cast to float64 #47393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 22, 2022
Merged

CI/TST: Don't require length for construct_1d_arraylike_from_scalar cast to float64 #47393

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ceffe6d
CI/TST: Don't require length for construct_1d_arraylike_from_scalar c…
mroeschke Jun 16, 2022
d324332
Just short circuit
mroeschke Jun 16, 2022
cdcf033
Add errstate
mroeschke Jun 17, 2022
1a82a00
Move errstate
mroeschke Jun 17, 2022
e63d61e
Merge remote-tracking branch 'upstream/main' into ci/fix/numpy-dev
mroeschke Jun 18, 2022
536786f
Add errstate to merge
mroeschke Jun 18, 2022
106449d
Fix typo
mroeschke Jun 18, 2022
2131095
Merge remote-tracking branch 'upstream/main' into ci/fix/numpy-dev
mroeschke Jun 18, 2022
df267f9
Merge remote-tracking branch 'upstream/main' into ci/fix/numpy-dev
mroeschke Jun 21, 2022
ffd1171
Move length check
mroeschke Jun 21, 2022
57d43c2
Add back length check
mroeschke Jun 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion pandas/core/construction.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -556,7 +556,10 @@ def sanitize_array(
if dtype is not None and is_float_dtype(data.dtype) and is_integer_dtype(dtype):
# possibility of nan -> garbage
try:
subarr = _try_cast(data, dtype, copy, True)
# GH 47391 numpy > 1.24 will raise a RuntimeError for nan -> int
# casting aligning with IntCastingNaNError below
with np.errstate(invalid="ignore"):
subarr = _try_cast(data, dtype, copy, True)
except IntCastingNaNError:
warnings.warn(
"In a future version, passing float-dtype values containing NaN "
Expand Down
6 changes: 4 additions & 2 deletions pandas/core/dtypes/cast.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1696,7 +1696,7 @@ def construct_1d_arraylike_from_scalar(

else:

if length and is_integer_dtype(dtype) and isna(value):
if is_integer_dtype(dtype) and isna(value):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think still need the length here as this part of the code is this logic to determine the the dtype

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i.e. revert to original.

# coerce if we have nan for an integer dtype
dtype = np.dtype("float64")
elif isinstance(dtype, np.dtype) and dtype.kind in ("U", "S"):
Expand All @@ -1709,7 +1709,9 @@ def construct_1d_arraylike_from_scalar(
value = _maybe_unbox_datetimelike_tz_deprecation(value, dtype)

subarr = np.empty(length, dtype=dtype)
subarr.fill(value)
if length:
# GH 47391: numpy > 1.24 will raise filling np.nan into int dtypes
subarr.fill(value)

return subarr

Expand Down
32 changes: 18 additions & 14 deletions pandas/core/reshape/merge.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1200,23 +1200,27 @@ def _maybe_coerce_merge_keys(self) -> None:

# check whether ints and floats
elif is_integer_dtype(rk.dtype) and is_float_dtype(lk.dtype):
if not (lk == lk.astype(rk.dtype))[~np.isnan(lk)].all():
warnings.warn(
"You are merging on int and float "
"columns where the float values "
"are not equal to their int representation.",
UserWarning,
)
# GH 47391 numpy > 1.24 will raise a RuntimeError for nan -> int
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for 1.5 we ought to actually remove the nans first

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in 1.5. add a deprecation noting that nans will be dropped?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no i mean i think u can remove the nans before comparing to avoid the warning (this is all internal anyhow)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah gotcha. Yeah can clean this for 1.5 in a separate PR

with np.errstate(invalid="ignore"):
if not (lk == lk.astype(rk.dtype))[~np.isnan(lk)].all():
warnings.warn(
"You are merging on int and float "
"columns where the float values "
"are not equal to their int representation.",
UserWarning,
)
continue

elif is_float_dtype(rk.dtype) and is_integer_dtype(lk.dtype):
if not (rk == rk.astype(lk.dtype))[~np.isnan(rk)].all():
warnings.warn(
"You are merging on int and float "
"columns where the float values "
"are not equal to their int representation.",
UserWarning,
)
# GH 47391 numpy > 1.24 will raise a RuntimeError for nan -> int
with np.errstate(invalid="ignore"):
if not (rk == rk.astype(lk.dtype))[~np.isnan(rk)].all():
warnings.warn(
"You are merging on int and float "
"columns where the float values "
"are not equal to their int representation.",
UserWarning,
)
continue

# let's infer and see if we are ok
Expand Down