Skip to content

plot with unsorted indices misses data points #11471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
den-run-ai opened this issue Oct 29, 2015 · 9 comments · Fixed by #16600
Closed

plot with unsorted indices misses data points #11471

den-run-ai opened this issue Oct 29, 2015 · 9 comments · Fixed by #16600
Labels
Bug Visualization plotting
Milestone
0.21.0

Comments

@den-run-ai
Copy link

Compare these 2 plots below, this is not expected behavior:

import pandas as pd
from numpy.random import rand
testarr1=rand(100)
testarr2=rand(100)
pd.DataFrame(testarr2,index=testarr1).plot(style='.')
pd.DataFrame(testarr2,index=testarr1).sort().plot(style='.')
@sinhrks
Copy link
Member

sinhrks commented Oct 29, 2015

Are you reporting a different issue from stackoverflow?

The difference seems to be caused by xlim adjustment logic. A workaround is:

ax = pd.DataFrame(testarr2,index=testarr1).plot(style='.')
ax.set_xlim(0, 1)

PR is welcome.

@den-run-ai
Copy link
Author

I removed SO question, it is completely unrelated after more detailed reading.
Setting xlim is a better workaround than sort.

@yoavram
Copy link

yoavram commented Mar 27, 2017 •
edited
Loading

Another example with real data, gist contains a short notebook and the data.

@sinhrks can you point me towards where xlim adjustment logic is?

@jreback
Copy link
Contributor

jreback commented Mar 27, 2017

@TomAugspurger ?

@TomAugspurger
Copy link
Contributor

@yoavram I believe the issue is somewhere around here

pandas/pandas/tools/plotting.py

Line 3569 in c112252

x = l.get_xdata(orig=False)

That seems to assume x[0] is the left-most /smallest point, and x[-1] is the largest. Presumably, you could call np.min and max on those rather than assuming that it's already sorted. Are you able to make a PR with a fix and test?

@yoavram
Copy link

yoavram commented Apr 7, 2017

I'll have a go at it, but I wonder how to test it - get the data from the aces and check that xlim contains the data range ?

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Apr 7, 2017 via email

nmartensen added a commit to nmartensen/pandas that referenced this issue Jun 5, 2017
@nmartensen
BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)
bfdfbe0 
 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.
@nmartensen nmartensen mentioned this issue Jun 5, 2017
Merged
4 tasks
nmartensen added a commit to nmartensen/pandas that referenced this issue Jun 10, 2017
@nmartensen
BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)
6b35277 
 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.
nmartensen added a commit to nmartensen/pandas that referenced this issue Jun 10, 2017
@nmartensen
BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)
3dac45f 
 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.
nmartensen added a commit to nmartensen/pandas that referenced this issue Jun 10, 2017
@nmartensen
BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)
f17904a 
 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.
nmartensen added a commit to nmartensen/pandas that referenced this issue Jul 19, 2017
@nmartensen
BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)
b49c35b 
 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.
nmartensen added a commit to nmartensen/pandas that referenced this issue Sep 15, 2017
@nmartensen
BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)
603b5f1 
 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.
TomAugspurger pushed a commit that referenced this issue Sep 19, 2017
@nmartensen @TomAugspurger
Fix automatic xlims in line plots (#16600)
3795272 
* BUG: set correct xlims for lines (#11471, #11310)

 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.

* BUG: Let new MPL automatically determine xlims (#15495)

 * Avoid setting xlims since recent matplotlib already does it correctly
 * and we should let it apply its default styles where possible

* TST: plotting: update expected results for matplotlib 2

Matplotlib 2.0 uses new defaults that cause some of our tests to fail.
This adds appropriate new sets of expected results to the following
tests in tests/plotting/test_datetimelike.py:

test_finder_daily
test_finder_quarterly
test_finder_annual
test_finder_hourly
test_finder_minutely
test_finder_monthly
test_format_timedelta_ticks_narrow
test_format_timedelta_ticks_wide

* TST: plotting: Relax some tests to work with matplotlib 2.0

Matplotlib 2.0 by default now adds some padding between the boundaries
of the data and the boundaries of the plot. This causes some of our
tests to fail if we don't relax them slightly.

  modified:   pandas/tests/plotting/test_datetimelike.py
test_irregular_ts_shared_ax_xlim
test_mixed_freq_regular_first
test_mixed_freq_regular_first_df
test_secondary_y_irregular_ts_xlim
test_secondary_y_non_ts_xlim
test_secondary_y_regular_ts_xlim

  modified:   pandas/tests/plotting/test_frame.py
test_area_lim
test_line_lim

  modified:   pandas/tests/plotting/test_series.py
test_ts_area_lim
test_ts_line_lim

* TST: Add lineplot tests with unsorted x data

Two new tests check interaction of non-monotonic x data and xlims:
test_frame / test_unsorted_index_lims
test_series / test_unsorted_index_xlim

* DOC: lineplot/xlims whatsnew entry for v0.21.0
@jorisvandenbossche
Copy link
Member

Closed by #16600

@jorisvandenbossche jorisvandenbossche added this to the 0.21.0 milestone Sep 24, 2017
@fuglede fuglede mentioned this issue Oct 23, 2017
Closed
alanbato pushed a commit to alanbato/pandas that referenced this issue Nov 10, 2017
@nmartensen @alanbato
Fix automatic xlims in line plots (pandas-dev#16600)
788ac49 
* BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)

 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.

* BUG: Let new MPL automatically determine xlims (pandas-dev#15495)

 * Avoid setting xlims since recent matplotlib already does it correctly
 * and we should let it apply its default styles where possible

* TST: plotting: update expected results for matplotlib 2

Matplotlib 2.0 uses new defaults that cause some of our tests to fail.
This adds appropriate new sets of expected results to the following
tests in tests/plotting/test_datetimelike.py:

test_finder_daily
test_finder_quarterly
test_finder_annual
test_finder_hourly
test_finder_minutely
test_finder_monthly
test_format_timedelta_ticks_narrow
test_format_timedelta_ticks_wide

* TST: plotting: Relax some tests to work with matplotlib 2.0

Matplotlib 2.0 by default now adds some padding between the boundaries
of the data and the boundaries of the plot. This causes some of our
tests to fail if we don't relax them slightly.

  modified:   pandas/tests/plotting/test_datetimelike.py
test_irregular_ts_shared_ax_xlim
test_mixed_freq_regular_first
test_mixed_freq_regular_first_df
test_secondary_y_irregular_ts_xlim
test_secondary_y_non_ts_xlim
test_secondary_y_regular_ts_xlim

  modified:   pandas/tests/plotting/test_frame.py
test_area_lim
test_line_lim

  modified:   pandas/tests/plotting/test_series.py
test_ts_area_lim
test_ts_line_lim

* TST: Add lineplot tests with unsorted x data

Two new tests check interaction of non-monotonic x data and xlims:
test_frame / test_unsorted_index_lims
test_series / test_unsorted_index_xlim

* DOC: lineplot/xlims whatsnew entry for v0.21.0
No-Stream pushed a commit to No-Stream/pandas that referenced this issue Nov 28, 2017
@nmartensen @No-Stream
Fix automatic xlims in line plots (pandas-dev#16600)
061897d 
* BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)

 * Do not assume that xdata is sorted.
 * Use numpy.nanmin() and numpy.nanmax() instead.

* BUG: Let new MPL automatically determine xlims (pandas-dev#15495)

 * Avoid setting xlims since recent matplotlib already does it correctly
 * and we should let it apply its default styles where possible

* TST: plotting: update expected results for matplotlib 2

Matplotlib 2.0 uses new defaults that cause some of our tests to fail.
This adds appropriate new sets of expected results to the following
tests in tests/plotting/test_datetimelike.py:

test_finder_daily
test_finder_quarterly
test_finder_annual
test_finder_hourly
test_finder_minutely
test_finder_monthly
test_format_timedelta_ticks_narrow
test_format_timedelta_ticks_wide

* TST: plotting: Relax some tests to work with matplotlib 2.0

Matplotlib 2.0 by default now adds some padding between the boundaries
of the data and the boundaries of the plot. This causes some of our
tests to fail if we don't relax them slightly.

  modified:   pandas/tests/plotting/test_datetimelike.py
test_irregular_ts_shared_ax_xlim
test_mixed_freq_regular_first
test_mixed_freq_regular_first_df
test_secondary_y_irregular_ts_xlim
test_secondary_y_non_ts_xlim
test_secondary_y_regular_ts_xlim

  modified:   pandas/tests/plotting/test_frame.py
test_area_lim
test_line_lim

  modified:   pandas/tests/plotting/test_series.py
test_ts_area_lim
test_ts_line_lim

* TST: Add lineplot tests with unsorted x data

Two new tests check interaction of non-monotonic x data and xlims:
test_frame / test_unsorted_index_lims
test_series / test_unsorted_index_xlim

* DOC: lineplot/xlims whatsnew entry for v0.21.0
@yoavram
Copy link

yoavram commented Apr 9, 2018

Just want to confirm that the problem in above cited gist is fixed with pandas 0.22.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Visualization plotting
Projects
None yet
Milestone
0.21.0
Development

Successfully merging a pull request may close this issue.

Fix automatic xlims in line plots nmartensen/pandas
6 participants
@jreback @jorisvandenbossche @yoavram @TomAugspurger @sinhrks @den-run-ai