Skip to content

QST: Dataframe filter a column if it is in an interval data type #52616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 tasks done
davetapley opened this issue Apr 12, 2023 · 4 comments
Closed
2 tasks done

QST: Dataframe filter a column if it is in an interval data type #52616

davetapley opened this issue Apr 12, 2023 · 4 comments
Labels
Needs Triage Issue that has not been reviewed by a pandas team member Usage Question

Comments

@davetapley
Copy link
Contributor

Research

  • I have searched the [pandas] tag on StackOverflow for similar questions.

  • I have asked my usage related question on StackOverflow.

Link to question on StackOverflow

https://stackoverflow.com/questions/55832639/dataframe-filter-a-column-if-it-is-in-an-interval-data-type

Question about pandas

Top voted SO answer gives:

data.loc[(data.A>=interval.left) & (data.A<interval.right)]

But this doesn't account for interval.closed.
Obviously that snippet could be expanded but check, but I feel like this should be supported out of the box?


IntervalIndex feels like it's close, but this restriction makes it a no-go for my case:

intervals that are closed on the same side.


Related: #21998

@davetapley davetapley added Needs Triage Issue that has not been reviewed by a pandas team member Usage Question labels Apr 12, 2023
@jbrockmendel
Copy link
Member

df = pd.DataFrame({'A': range(1, 10)})
obj = pd.Interval(1,4)

ser = pd.Series(obj, index=df.index)
arr = ser.array

>>> arr.contains(df["A"])
0    False
1     True
2     True
3     True
4    False
5    False
6    False
7    False
8    False
Name: A, dtype: bool

@phofl
Copy link
Member

phofl commented Apr 22, 2023

Closing, please ping if this does not answer your question

@phofl phofl closed this as completed Apr 22, 2023
@davetapley
Copy link
Contributor Author

Thanks @jbrockmendel, @phofl, that works great, but the typing is broken in VSCode, I get:

Cannot access member "contains" for type "ExtensionArray"
  Member "contains" is unknownPylance[reportGeneralTypeIssues](https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportGeneralTypeIssues)
(function) contains: Unknown

Worth a separate bug report?

@phofl
Copy link
Member

phofl commented Apr 26, 2023

This should go into the pandas-stubs repo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Triage Issue that has not been reviewed by a pandas team member Usage Question
Projects
None yet
Development

No branches or pull requests

3 participants