Skip to content

merge_asof(left_index=True, right_on=...) overwrites left index with right index #33463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gtg944q opened this issue Apr 10, 2020 · 4 comments · Fixed by #34484
Closed

merge_asof(left_index=True, right_on=...) overwrites left index with right index #33463

gtg944q opened this issue Apr 10, 2020 · 4 comments · Fixed by #34484
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@gtg944q
Copy link

gtg944q commented Apr 10, 2020

Building from an example in the docs...

left = pd.DataFrame({'left_val': ['a', 'b', 'c'], 'left_time': [1, 5, 10]}, index=[1, 5, 10])
right = pd.DataFrame({'right_val': [1, 2, 3, 6, 7]}, index=[1, 2, 3, 6, 7])
>>> pd.merge_asof(left, right, left_index=True, right_index=True)
   left_val  left_time  right_val
1         a          1          1
5         b          5          3
10        c         10          7

Returned index is the index of left.

>>> pd.merge_asof(left, right, left_on='left_time', right_index=True)
   left_val  left_time  right_val
1         a          1          1
5         b          5          3
10        c         10          7

Returned index is the index of left.

>>> pd.merge_asof(left, right, left_on='left_time', right_on='right_val')
  left_val  left_time  right_val
0        a          1          1
1        b          5          3
2        c         10          7

Return value has index reset (I assume expected?)

>>> pd.merge_asof(left, right, left_index=True, right_on='right_val')
  left_val  left_time  right_val
1        a          1          1
3        b          5          3
7        c         10          7

Returned index is the index of right.

Perhaps I'm misunderstanding something about how this is supposed to work, but it seems to me that the behavior in the last example is incorrect.

@gtg944q
Copy link
Author

gtg944q commented Apr 10, 2020

NB: proving to myself that the indexes returned in examples 2 and 4 are indeed the indexes, not the values:

>>> left = pd.DataFrame({'left_val': ['a', 'b', 'c'], 'left_time': [1, 4, 10]}, index=[1, 5, 10])
>>> pd.merge_asof(left, right, left_on='left_time', right_index=True)
   left_val  left_time  right_val
1         a          1          1
5         b          4          3
10        c         10          7
>>> right = pd.DataFrame({'right_val': [1, 2, 3, 6, 8]}, index=[1, 2, 3, 6, 7])
>>> pd.merge_asof(left, right, left_index=True, right_on='right_val')
  left_val  left_time  right_val
1        a          1          1
3        b          4          3
7        c         10          8

@ShaharNaveh
Copy link
Member

@gtg944q Thank you for the report!


Do you mind paste the URL of the examples you found in the docs? we fixed a lot of documentation related problems recently, maybe this is already solved.

@gtg944q
Copy link
Author

gtg944q commented Apr 10, 2020

It's the second block of examples at https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.merge_asof.html, but I'm really just building on an example in there, so it's not a documentation problem.

@mroeschke
Copy link
Member

Since this is advertised as "left-join" like, the last 2 examples you provide are surprising, and I'd expect to propagate the values in the left index (and not reset the index or use the right index). I'll mark as a bug, but this may break some existing tests.

@mroeschke mroeschke added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Apr 10, 2020
@jreback jreback added this to the 1.2 milestone Sep 5, 2020
@jreback jreback modified the milestones: 1.2, Contributions Welcome Nov 19, 2020
@jreback jreback modified the milestones: Contributions Welcome, 1.3 Feb 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants