Skip to content

BUG: maybe_convert_numeric fails with uint64 #14982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
gfyoung opened this issue Dec 25, 2016 · 1 comment
Closed

BUG: maybe_convert_numeric fails with uint64 #14982

gfyoung opened this issue Dec 25, 2016 · 1 comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions

Comments

@gfyoung
Copy link
Member

gfyoung commented Dec 25, 2016 •
edited
Loading

master at aba7d2:

>>> from pandas import lib, np
>>> arr = np.array([2**63], dtype=object)
>>> lib.maybe_convert_numeric(arr, set())
...
OverflowError: Python int too large to convert to C long

At first, I thought the patch would be similar to #14916, but there are several complications here:

  • Handling an array with negative and uint64 values: in maybe_convert_objects, we would just return object to avoid data truncation. However, this function currently only returns numeric arrays. And if we do decide to return non-numeric, should we then raise if coerce_numeric=True then?
  • uint64 has no convenient na_value, so what happens if a uint64 is in na_values?
@gfyoung gfyoung mentioned this issue Dec 25, 2016
Closed
@jreback
Copy link
Contributor

jreback commented Dec 26, 2016

your two suggestions are fine

  • a uint64 array with negative values (or in na_values) must be converted to object.

so unfortunately maybe_convert_numeric gets more complicated :>

@jreback jreback added Bug Dtype Conversions Unexpected or buggy dtype conversions Difficulty Intermediate labels Dec 26, 2016
@jreback jreback added this to the Next Major Release milestone Dec 26, 2016
gfyoung added a commit to forking-repos/pandas that referenced this issue Dec 28, 2016
@gfyoung
BUG: Convert uint64 in maybe_convert_numeric
9cde6a6 
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the origin input be returned if
coerce_numeric is False. Otherwise, an
Exception is raisd.

Closes pandas-devgh-14982.
gfyoung added a commit to forking-repos/pandas that referenced this issue Dec 28, 2016
@gfyoung
BUG: Convert uint64 in maybe_convert_numeric
a83b51b 
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes pandas-devgh-14982.
Partial fix for pandas-devgh-14983.
@gfyoung gfyoung mentioned this issue Dec 28, 2016
Closed
gfyoung added a commit to forking-repos/pandas that referenced this issue Dec 28, 2016
@gfyoung
BUG: Convert uint64 in maybe_convert_numeric
e15620f 
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes pandas-devgh-14982.
Partial fix for pandas-devgh-14983.
gfyoung added a commit to forking-repos/pandas that referenced this issue Dec 29, 2016
@gfyoung
BUG: Convert uint64 in maybe_convert_numeric
c10449c 
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes pandas-devgh-14982.
Partial fix for pandas-devgh-14983.
gfyoung added a commit to forking-repos/pandas that referenced this issue Dec 29, 2016
@gfyoung
BUG: Convert uint64 in maybe_convert_numeric
c3bd28a 
Add handling for uint64 elements in an array
with the follow behavior specifications:

1) If uint64 and NaN are both detected, the
original input will be returned if coerce_numeric
is False. Otherwise, an Exception is raised.

2) If uint64 and negative numbers are both
detected, the original input be returned if
coerce_numeric is False. Otherwise, an
Exception is raised.

Closes pandas-devgh-14982.
Partial fix for pandas-devgh-14983.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

BUG: Convert uint64 in maybe_convert_numeric forking-repos/pandas
2 participants
@jreback @gfyoung