Skip to content

BUG: infer_dtype with decimal/complex #37176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 18, 2020

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx
  • tests added / passed
  • passes black pandas
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

Does this need a whatsnew?

@@ -2546,8 +2565,6 @@ def fast_multiget(dict mapping, ndarray keys, default=np.nan):
# kludge, for Series
return np.empty(0, dtype='f8')

keys = getattr(keys, 'values', keys)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unrelated cleanup

@@ -1414,10 +1414,12 @@ def infer_dtype(value: object, skipna: bool = True) -> str:
return "time"

elif is_decimal(val):
return "decimal"
if all(is_decimal(x) for x in values):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you create is_decimal_array to have the same style

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think you actually want to use the validator e.g as a raw NaN would not work here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you expand on this? do you expect something other than Decimal objects to work here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NaNs are valid as well here; but more important is that every validator has the same pattern. a is_* function e.g. is_decimal_array, which should call here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to use the is_* pattern.

since decimal has Decimal("nan"), i wouldnt expect to accept np.nan here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since decimal has Decimal("nan"), i wouldnt expect to accept np.nan here

yeah ok, though i think we could actually accept np.nan as well and infer a decimal if that is the case (as we do this for all other dtypes). can you open an issue.

@jreback jreback added the Dtype Conversions Unexpected or buggy dtype conversions label Oct 16, 2020
@jreback jreback added this to the 1.2 milestone Oct 18, 2020
@jreback jreback merged commit 61693b5 into pandas-dev:master Oct 18, 2020
@jreback
Copy link
Contributor

jreback commented Oct 18, 2020

thanks

@jbrockmendel jbrockmendel deleted the bug-infer-decimal branch October 18, 2020 17:39
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Oct 26, 2020
kesmit13 pushed a commit to kesmit13/pandas that referenced this pull request Nov 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants