Skip to content

merge_ordered() doesn't validate fill_method #55884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wjandrea opened this issue Nov 8, 2023 · 2 comments · Fixed by #55927
Closed

merge_ordered() doesn't validate fill_method #55884

wjandrea opened this issue Nov 8, 2023 · 2 comments · Fixed by #55927
Labels
Error Reporting Incorrect or improved errors from pandas Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@wjandrea
Copy link
Contributor

wjandrea commented Nov 8, 2023

The docs say fill_method is the "Interpolation method for data", but I just tried pd.merge_ordered(..., fill_method='linear') and it didn't interpolate.

Looking closer at the docs, fill_method can only be either 'ffill' or None, but apparently that's not actually checked; for example, fill_method='carrot' also works.

fill_method : {'ffill', None}, default None
Interpolation method for data.

Looks like this code is responsible:

if self.fill_method == "ffill":
if left_indexer is None:
raise TypeError("left_indexer cannot be None")
left_indexer = cast("npt.NDArray[np.intp]", left_indexer)
right_indexer = cast("npt.NDArray[np.intp]", right_indexer)
left_join_indexer = libjoin.ffill_indexer(left_indexer)
right_join_indexer = libjoin.ffill_indexer(right_indexer)
else:
left_join_indexer = left_indexer
right_join_indexer = right_indexer

It should probably be changed to something like:

if self.fill_method == "ffill": 
    if left_indexer is None:
    ...
elif self.fill_method is None: 
    left_join_indexer = left_indexer
    right_join_indexer = right_indexer
else:
    raise ValueError(f"fill_method must be one of ['ffill']. Got '{self.fill_method}' instead.")

(ValueError text based on equivalent df.interpolate error)

@tshehadeh9
Copy link

Hi, new contributor here -- can I work on this task?

@lithomas1
Copy link
Member

PRs would be welcome for this.

@lithomas1 lithomas1 added Reshaping Concat, Merge/Join, Stack/Unstack, Explode Error Reporting Incorrect or improved errors from pandas labels Nov 11, 2023
ziadk pushed a commit to ziadk/pandas that referenced this issue Nov 12, 2023
ziadk pushed a commit to ziadk/pandas that referenced this issue Nov 12, 2023
mroeschke added a commit that referenced this issue Nov 14, 2023
* Fixes #55884- Added else for invalid fill_method

* Fixes #55884- Added else for invalid fill_method

* Update pandas/tests/reshape/merge/test_merge_ordered.py

Co-authored-by: Matthew Roeschke <[email protected]>

* Update pandas/tests/reshape/merge/test_merge_ordered.py

Co-authored-by: Matthew Roeschke <[email protected]>

* Update pandas/core/reshape/merge.py

Co-authored-by: Matthew Roeschke <[email protected]>

* Fixed mergeordered ValueError Message

* Use pytest fixtures & parametrize for tests

---------

Co-authored-by: Matthew Roeschke <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Error Reporting Incorrect or improved errors from pandas Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants