Skip to content

OPS: Remove mask_cmp_op fallback behavior #28601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 26, 2019

Conversation

jbrockmendel
Copy link
Member

xref #28050.

To make the Series vs DataFrame behavior consistent, the two main options are a) change the DataFrame behavior (this PR) or b) change the Series behavior. The latter is complicated by the fact that for object dtypes the Series comparisons go through comp_method_OBJECT_ARRAY instead of the numpy op, so we would still have to change the complex-case test changed here.

@jreback jreback added this to the 1.0 milestone Sep 25, 2019
@jreback
Copy link
Contributor

jreback commented Sep 25, 2019

lgtm. I think this needs a note (you can just put in bug fixes would be fine); as something that 'worked' before now will raise (correctly)

@jbrockmendel
Copy link
Member Author

note added

@@ -199,7 +199,7 @@ Timezones
Numeric
^^^^^^^
- Bug in :meth:`DataFrame.quantile` with zero-column :class:`DataFrame` incorrectly raising (:issue:`23925`)
-
- :class:`DataFrame` inequality comparisons with object-dtype and ``complex`` entries failing to raise ``TypeError`` like their :class:`Series` counterparts (:issue:`28079`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

entries -> dtypes (can do in later PR)

@jreback jreback merged commit 4a0f74f into pandas-dev:master Sep 26, 2019
@jreback
Copy link
Contributor

jreback commented Sep 26, 2019

thanks minor comment for followup

@jbrockmendel jbrockmendel deleted the cmpflex branch September 26, 2019 14:48
@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Sep 26, 2019

@jbrockmendel you mentioned that you changed the behaviour of the flex method for Series or DataFrame (the example in the issue also shows one without complex dtype). Shouldn't that be reflected in the whatsnew note?

@jorisvandenbossche
Copy link
Member

Updated question: this was only for the comparison methods and not the comparison operation, correct? If so, I would be clearer on that in the whatsnew note.

@jbrockmendel
Copy link
Member Author

you mentioned that you changed the behaviour of the flex method for Series or DataFrame

DataFrame's lt, le, gt, ge methods are affected. AFAICT the only case where it really makes a difference is with complex-dtypes that used to get cast to object-dtype for masking fallback, because numpy (for some reason) does complex-dtype comparisons differently from python.

Which is a long way of saying I don't understand the question. Do you have a suggested edit for the whatsnew note?

this was only for the comparison methods and not the comparison operation, correct? I

I don't understand the distinction. Can you give an example?

@jorisvandenbossche
Copy link
Member

DataFrame's lt, le, gt, ge methods are affected

As you say, that are methods. And with the comparison operators I mean >, <, .. An essential difference in this case, and if you just say "comparison" most people will think about the second IMO

@jbrockmendel
Copy link
Member Author

How would you word the note? Adding the word "flex" is a start.

proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
bongolegend pushed a commit to bongolegend/pandas that referenced this pull request Jan 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

API: flex comparisons DataFrame vs Series inconsistency
3 participants