Fix automatic xlims in line plots #16600

nmartensen · 2017-06-05T12:39:46Z

closes Pandas.plot() fails to calculate the right xlims #11310, closes plot with unsorted indices misses data points #11471, closes Update plotting functions to use the new Matplotlib default styles #15495
tests added / passed
passes git diff upstream/master --name-only -- '*.py' | flake8 --diff
whatsnew entry

nmartensen · 2017-06-05T19:05:29Z

So this breaks 18 Tests:

test_finder_annual - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_finder_daily - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_finder_hourly - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_finder_minutely - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_finder_monthly - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_finder_quarterly - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_format_timedelta_ticks_narrow - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_format_timedelta_ticks_wide - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_irregular_ts_shared_ax_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_mixed_freq_regular_first - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_mixed_freq_regular_first_df - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_secondary_y_irregular_ts_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_secondary_y_non_ts_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_secondary_y_regular_ts_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
test_area_lim - pandas.tests.plotting.test_frame.TestDataFramePlots
test_line_lim - pandas.tests.plotting.test_frame.TestDataFramePlots
test_ts_area_lim - pandas.tests.plotting.test_series.TestSeriesPlots
test_ts_line_lim - pandas.tests.plotting.test_series.TestSeriesPlots

These tests hardcode expectations from previous MPL defaults. It is only the second commit (which allows MPL to apply its new defaults) that breaks the tests. Do they need a second set of expected results depending on which MPL version is used? Or can we simply update the expected results to what current MPL produces?

The existing tests do not yet cover the case of unsorted x axis (the subject of the first commit). This will need new tests. Is a single test sufficient or do we need different instances for datetimelike/DataFrame/Series?

TomAugspurger · 2017-06-06T16:25:54Z

Thanks for this. Your changes look correct.

Do they need a second set of expected results depending on which MPL version is used? Or can we simply update the expected results to what current MPL produces?

I think we test on an old matplotlib (1.3.1) on python 2.7, and the most recent matplotlib otherwise. So yes, ideally we would cover both. If it's excessively difficult, you can raise skiptests for older matplotlibs where nescessary.

Is a single test sufficient or do we need different instances for datetimelike/DataFrame/Series?

I think just a test for line plots covering a few cases

Series.plot with unsorted index
DataFrame.plot with unsorted index
DataFrame.plot(x='x', y='y') with unsorted x

should be sufficient.

We'll also need a release note. If you rebase on master, there should be a docs/source/whatsnew/v0.20.3.txt you can add this to.

TomAugspurger · 2017-06-06T19:52:53Z

Actually, it's probably best to do this for 0.21.0, not 0.20.3, since the xlim will be now following matplotlib 2's.

codecov · 2017-06-10T19:28:24Z

Codecov Report

Merging #16600 into master will increase coverage by 0.01%.
The diff coverage is 16.66%.

@@            Coverage Diff             @@
##           master   #16600      +/-   ##
==========================================
+ Coverage   90.93%   90.95%   +0.01%     
==========================================
  Files         161      161              
  Lines       49266    49267       +1     
==========================================
+ Hits        44801    44810       +9     
+ Misses       4465     4457       -8

Flag	Coverage Δ
#multiple	`88.71% <16.66%> (+0.01%)`	⬆️
#single	`40.22% <0%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/plotting/_tools.py	`72.92% <0%> (-6.08%)`	⬇️
pandas/plotting/_core.py	`82.38% <25%> (+0.46%)`	⬆️
pandas/plotting/_converter.py	`65.2% <0%> (+1.96%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ceaf852...ce6bcb1. Read the comment docs.

codecov · 2017-06-10T19:28:34Z

Codecov Report

Merging #16600 into master will decrease coverage by 0.01%.
The diff coverage is 16.66%.

@@            Coverage Diff             @@
##           master   #16600      +/-   ##
==========================================
- Coverage   91.22%    91.2%   -0.02%     
==========================================
  Files         163      163              
  Lines       49624    49625       +1     
==========================================
- Hits        45269    45260       -9     
- Misses       4355     4365      +10

Flag	Coverage Δ
#multiple	`88.99% <16.66%> (-0.01%)`	⬇️
#single	`40.19% <0%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/plotting/_tools.py	`72.92% <0%> (-6.08%)`	⬇️
pandas/plotting/_core.py	`82.52% <25%> (-0.21%)`	⬇️
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.77% <0%> (-0.1%)`	⬇️
pandas/plotting/_converter.py	`65.2% <0%> (+1.96%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cbb090f...15eae82. Read the comment docs.

nmartensen · 2017-07-03T19:58:19Z

So the codecov decrease in plotting/_tool.py is actually the point of one of the patches – xlims should only be adjusted with older matplotlib.

If you prefer this PR to be split into two (one for wrong xlims and one for MPL xlim defaults), or if you have other comments, let me know.

TomAugspurger · 2017-07-05T13:08:03Z

So the codecov decrease...

That may be inaccurate. I don't recall which versions of matplotlib are installed on the runs that report code coverage. It should still be being tested on other runs.

jreback · 2017-07-19T10:33:35Z

can you rebase / update according to comments

TomAugspurger · 2017-07-19T11:54:33Z

I think you'll just need a rebase. IIRC, things looked pretty good last time I looked.

nmartensen · 2017-07-19T18:35:34Z

rebased

jreback · 2017-09-10T14:50:32Z

can you rebase / update once again

Two new tests check interaction of non-monotonic x data and xlims: test_frame / test_unsorted_index_lims test_series / test_unsorted_index_xlim

* Do not assume that xdata is sorted. * Use numpy.nanmin() and numpy.nanmax() instead.

* Avoid setting xlims since recent matplotlib already does it correctly * and we should let it apply its default styles where possible

Matplotlib 2.0 uses new defaults that cause some of our tests to fail. This adds appropriate new sets of expected results to the following tests in tests/plotting/test_datetimelike.py: test_finder_daily test_finder_quarterly test_finder_annual test_finder_hourly test_finder_minutely test_finder_monthly test_format_timedelta_ticks_narrow test_format_timedelta_ticks_wide

Matplotlib 2.0 by default now adds some padding between the boundaries of the data and the boundaries of the plot. This causes some of our tests to fail if we don't relax them slightly. modified: pandas/tests/plotting/test_datetimelike.py test_irregular_ts_shared_ax_xlim test_mixed_freq_regular_first test_mixed_freq_regular_first_df test_secondary_y_irregular_ts_xlim test_secondary_y_non_ts_xlim test_secondary_y_regular_ts_xlim modified: pandas/tests/plotting/test_frame.py test_area_lim test_line_lim modified: pandas/tests/plotting/test_series.py test_ts_area_lim test_ts_line_lim

nmartensen · 2017-09-16T07:39:45Z

rebased

TomAugspurger · 2017-09-19T18:52:17Z

Thanks @nmartensen! (sorry about the extra rebase)

* BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310) * Do not assume that xdata is sorted. * Use numpy.nanmin() and numpy.nanmax() instead. * BUG: Let new MPL automatically determine xlims (pandas-dev#15495) * Avoid setting xlims since recent matplotlib already does it correctly * and we should let it apply its default styles where possible * TST: plotting: update expected results for matplotlib 2 Matplotlib 2.0 uses new defaults that cause some of our tests to fail. This adds appropriate new sets of expected results to the following tests in tests/plotting/test_datetimelike.py: test_finder_daily test_finder_quarterly test_finder_annual test_finder_hourly test_finder_minutely test_finder_monthly test_format_timedelta_ticks_narrow test_format_timedelta_ticks_wide * TST: plotting: Relax some tests to work with matplotlib 2.0 Matplotlib 2.0 by default now adds some padding between the boundaries of the data and the boundaries of the plot. This causes some of our tests to fail if we don't relax them slightly. modified: pandas/tests/plotting/test_datetimelike.py test_irregular_ts_shared_ax_xlim test_mixed_freq_regular_first test_mixed_freq_regular_first_df test_secondary_y_irregular_ts_xlim test_secondary_y_non_ts_xlim test_secondary_y_regular_ts_xlim modified: pandas/tests/plotting/test_frame.py test_area_lim test_line_lim modified: pandas/tests/plotting/test_series.py test_ts_area_lim test_ts_line_lim * TST: Add lineplot tests with unsorted x data Two new tests check interaction of non-monotonic x data and xlims: test_frame / test_unsorted_index_lims test_series / test_unsorted_index_xlim * DOC: lineplot/xlims whatsnew entry for v0.21.0

TomAugspurger added the Visualization plotting label Jun 6, 2017

TomAugspurger added this to the 0.20.3 milestone Jun 6, 2017

TomAugspurger modified the milestones: 0.21.0, 0.20.3 Jun 6, 2017

nmartensen force-pushed the master branch 3 times, most recently from e4a7700 to ce6bcb1 Compare June 10, 2017 19:27

nmartensen force-pushed the master branch from ce6bcb1 to ac5a3ba Compare July 19, 2017 16:59

nmartensen added 6 commits September 16, 2017 00:36

TST: Add lineplot tests with unsorted x data

6a9eb51

Two new tests check interaction of non-monotonic x data and xlims: test_frame / test_unsorted_index_lims test_series / test_unsorted_index_xlim

BUG: set correct xlims for lines (pandas-dev#11471, pandas-dev#11310)

603b5f1

* Do not assume that xdata is sorted. * Use numpy.nanmin() and numpy.nanmax() instead.

BUG: Let new MPL automatically determine xlims (pandas-dev#15495)

0a7bbbe

* Avoid setting xlims since recent matplotlib already does it correctly * and we should let it apply its default styles where possible

DOC: lineplot/xlims whatsnew entry for v0.21.0

a468598

nmartensen force-pushed the master branch from ac5a3ba to a468598 Compare September 15, 2017 22:41

Merge branch 'master' into master

15eae82

TomAugspurger merged commit 3795272 into pandas-dev:master Sep 19, 2017

This was referenced Sep 24, 2017

Update plotting functions to use the new Matplotlib default styles #17231

Closed

plot with unsorted indices misses data points #11471

Closed

Update plotting functions to use the new Matplotlib default styles #15495

Closed

bobhaffner mentioned this pull request Oct 21, 2017

Unsorted Float64Index gives clipped plots #17938

Closed

This was referenced Nov 9, 2017

automatic inversion of x axis by pandas.plot(...) #10118

Closed

Line plot not autoscaled #17684

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix automatic xlims in line plots #16600

Fix automatic xlims in line plots #16600

nmartensen commented Jun 5, 2017 •

edited by jorisvandenbossche

Loading

nmartensen commented Jun 5, 2017

TomAugspurger commented Jun 6, 2017

TomAugspurger commented Jun 6, 2017

codecov bot commented Jun 10, 2017

codecov bot commented Jun 10, 2017 •

edited

Loading

nmartensen commented Jul 3, 2017

TomAugspurger commented Jul 5, 2017 •

edited

Loading

jreback commented Jul 19, 2017

TomAugspurger commented Jul 19, 2017

nmartensen commented Jul 19, 2017

jreback commented Sep 10, 2017

nmartensen commented Sep 16, 2017

TomAugspurger commented Sep 19, 2017

Fix automatic xlims in line plots #16600

Fix automatic xlims in line plots #16600

Conversation

nmartensen commented Jun 5, 2017 • edited by jorisvandenbossche Loading

nmartensen commented Jun 5, 2017

TomAugspurger commented Jun 6, 2017

TomAugspurger commented Jun 6, 2017

codecov bot commented Jun 10, 2017

Codecov Report

codecov bot commented Jun 10, 2017 • edited Loading

Codecov Report

nmartensen commented Jul 3, 2017

TomAugspurger commented Jul 5, 2017 • edited Loading

jreback commented Jul 19, 2017

TomAugspurger commented Jul 19, 2017

nmartensen commented Jul 19, 2017

jreback commented Sep 10, 2017

nmartensen commented Sep 16, 2017

TomAugspurger commented Sep 19, 2017

nmartensen commented Jun 5, 2017 •

edited by jorisvandenbossche

Loading

codecov bot commented Jun 10, 2017 •

edited

Loading

TomAugspurger commented Jul 5, 2017 •

edited

Loading