PLOT: Add option to specify the plotting backend #26753

datapythonista · 2019-06-09T11:33:34Z

closes API: engine kw to .plot to enable selectable backends #14130
tets added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Adding option to specify the plotting backend. This does not change the plotting API, so for now backends are expected to implement the existing methods in the matplotlib API. This API is expected to change as a result of the discussions in #26747.

datapythonista · 2019-06-09T11:37:24Z

Just realised with @TomAugspurger previous branch on the same work that I can simply do cf.get_option(key). So I guess the question is simply:

Do we want to import the backend module when it is set, or when the user first plots?

pandas/core/config_init.py

TomAugspurger · 2019-06-09T11:54:47Z

pandas/plotting/_core.py

+                             'A pandas plotting backend must be a module that '
+                             'can be imported'.format(backend_str))
+
+        required_objs = ['LinePlot', 'BarPlot', 'BarhPlot', 'HistPlot',


This seems like we’re being too opinionated about the backends implementation. IMO, we should provide a series and frame .plot accessor that dispatches to the right backend. We can provide an ABC for backends to subclass with the expected user API, but beyond that I don’t think we care.

I don't have a strong opinion about this at this point. I opened #26747 to have a discussion on what we expect from backends, this can surely be simplified.

At this stage I just tried to be conservative, and force the backends to raise NotImplementedError or whatever they consider if they don't implement something. My idea here is that if the user calls something like Series.plot() or any other functionality, we don't find with a weird error that doesn't provide relevant information for the user.

But happy raise exceptions from our side for some of those when the backend is not the default one, if that's the preferred option.

codecov · 2019-06-09T12:12:13Z

Codecov Report

Merging #26753 into master will decrease coverage by 0.02%.
The diff coverage is 40.9%.

@@            Coverage Diff             @@
##           master   #26753      +/-   ##
==========================================
- Coverage    91.7%   91.67%   -0.03%     
==========================================
  Files         179      179              
  Lines       50767    50784      +17     
==========================================
+ Hits        46555    46558       +3     
- Misses       4212     4226      +14

Flag	Coverage Δ
#multiple	`90.27% <40.9%> (-0.02%)`	⬇️
#single	`41.21% <18.18%> (-0.08%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/config_init.py	`93.07% <100%> (+0.16%)`	⬆️
pandas/plotting/_core.py	`84.1% <31.57%> (-4.85%)`	⬇️
pandas/io/gbq.py	`78.94% <0%> (-10.53%)`	⬇️
pandas/core/frame.py	`96.88% <0%> (-0.12%)`	⬇️
pandas/util/testing.py	`90.73% <0%> (+0.1%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7748ca...776e82d. Read the comment docs.

codecov · 2019-06-09T12:12:14Z

Codecov Report

Merging #26753 into master will decrease coverage by <.01%.
The diff coverage is 91.3%.

@@            Coverage Diff             @@
##           master   #26753      +/-   ##
==========================================
- Coverage   91.98%   91.97%   -0.01%     
==========================================
  Files         180      180              
  Lines       50760    50772      +12     
==========================================
+ Hits        46690    46700      +10     
- Misses       4070     4072       +2

Flag	Coverage Δ
#multiple	`90.57% <91.3%> (ø)`	⬆️
#single	`41.82% <34.78%> (-0.11%)`	⬇️

Impacted Files	Coverage Δ
pandas/plotting/_misc.py	`64.86% <100%> (+0.3%)`	⬆️
pandas/plotting/_core.py	`89.38% <100%> (+1.18%)`	⬆️
pandas/core/config_init.py	`95.8% <87.5%> (-1.05%)`	⬇️
pandas/io/gbq.py	`88.88% <0%> (-11.12%)`	⬇️
pandas/core/frame.py	`96.89% <0%> (-0.12%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c275dbf...1f4e1c0. Read the comment docs.

pandas/core/config_init.py

datapythonista · 2019-06-20T11:20:54Z

@jreback this should be ready now, let me know if there is any other change needed.

TomAugspurger · 2019-06-20T11:49:30Z

doc/source/whatsnew/v0.25.0.rst

@@ -132,6 +132,7 @@ Other Enhancements
 - :class:`DatetimeIndex` and :class:`TimedeltaIndex` now have a ``mean`` method (:issue:`24757`)
 - :meth:`DataFrame.describe` now formats integer percentiles without decimal point (:issue:`26660`)
 - Added support for reading SPSS .sav files using :func:`read_spss` (:issue:`26537`)
+- Added new option ``plotting.backend`` to be able to select a plotting backend different that the existing ``matplotlib`` one. Use ``pandas.set_option('plotting.backend', '<backend-module>')`` where ``<backend-module`` is a library implementing the pandas plotting API (:issue:`14130`)


"that" -> "than"

We don't have any alternative engines to list here yet, right?

Not at the moment, but that's a good point. Next week once this is merged I'm planning to work with few people to adapt hvplot. So we can see that everything is working well, and we can fix anything before 0.25. It may make sense to update this and use hvplot as an example when it's ready.

Great. I think this should be a prominent new feature if we are able to get either or both of pdvega ready to use it in time for the release.

TomAugspurger · 2019-06-20T11:55:22Z

pandas/core/config_init.py

@@ -460,6 +462,40 @@ def use_inf_as_na_cb(key):
 # Plotting
 # ---------

+plotting_backend_doc = """


One question: If I wanted to use the altar backend, I would be more like to use .set_option('plotting.backend', 'altair') than ..., 'pdvega'). @jakevdp what name would you prefer?

I think hvplot will just be hvplot, so that's fine.

Anyway, we might consider adding a dict here like plotting_backend_alias that maps the user-facing name like altair to the backend name like pdvega. When the backend library registers themselves, they can also register their aliases.

I see your point, and I think it'd add value to the users, but not sure if I'm in favor of adding the extra complexity it's needed to manage aliases in a dynamic way.

I like the simplicity of the parameter being the name of the module. I guess in some cases will look nicer than others. May be hvplot will use hvplot.pandas, since hvplot contains other things besides our plugin, and the module to use may be hvplot.pandas.

In practice I guess backends will register themselves, and users will rarely switch backends manually. But I guess if they do, it'll be better if they know they need to use the name of the module:

import pandas import hvplot.pandas df.plot() pandas.set_option('backend.plotting', 'matplotlib') df.plot() pandas.set_option('backend.plotting', 'hvplot.pandas') df.plot()

I don't have a strong opinion, but I'd say let's start with the simplest option, and add aliases or something else if we think it's useful once we start using this.

Is it especially complex? I was thinking something like

_plotting_aliases = {} # or define somewhere else def register_plotting_backend_cb(key): backend_str = cf.get_option(key) backend_str = _plotting_aliases.get(backend_str, backend_str) ...

Indeed, I think this simplifies things already, since we can use 'matplotlib' as pandas.plotting._matplotlib. Though we may continue to special case matplotlib to provide a nice error message.

What I wouldn't do is to have the aliases in pandas itself. May be I'm being too strict, but if feels wrong.

But you're right, it's probably not as complex as I was thinking anyway. An simple option plotting.aliases with a dictionary may not be ideal, but would allow backends create an alias by simply:

pandas.set_option('plotting.aliases', dict(pandas.get_option('plotting.aliases'), hvplot='hvplot.pandas'))

but better in a follow up PR I think, so we can focus there on the exact syntax and approach

Perfectly fine doing as a followup.

And my thinking may have been a bit muddled here. I was thinking that the backend library would have already been imported, and so would have a chance to register their own aliases. But as you say, it would be pandas managing them, which doesn't feel quite right.

another simple option is that backends add an optional attribute alias = 'hvplot', and we simply do:

if hasattr(backend_mod, 'alias'): plotting_aliases[alias] = backend_mod.__name__

Yes good idea. But still leaving this as a followup?

Yes, I prefer to keep the focus, the smaller the PRs, the better the content :)

pandas/plotting/_core.py

jreback · 2019-06-21T02:09:06Z

@datapythonista amazing this didn't need to be rebase, but can you merge master so the tests run again after @TomAugspurger convert patch. merge on green.

…o select_backend

WIP/PLOT: Add option to specify the plotting backend

776e82d

datapythonista added the Visualization plotting label Jun 9, 2019

jreback requested changes Jun 9, 2019

View reviewed changes

pandas/core/config_init.py Show resolved Hide resolved

TomAugspurger reviewed Jun 9, 2019

View reviewed changes

datapythonista mentioned this pull request Jun 10, 2019

API: Define API for pandas plotting backends #26747

Open

Marc Garcia added 3 commits June 10, 2019 12:01

Merge remote-tracking branch 'upstream/master' into select_backend

7d89c5e

Moving the validation of the backend to when the backend is selected

fd36e1a

Adding tests, doc and whatsnew

3ae0662

datapythonista changed the title ~~WIP/PLOT: Add option to specify the plotting backend~~ PLOT: Add option to specify the plotting backend Jun 10, 2019

Marc Garcia added 2 commits June 10, 2019 17:04

Restoring plotting backend after tests, to not affect other tests

57f0119

avoid failing tests when matplotlib is not installed

06e829c

jreback requested changes Jun 11, 2019

View reviewed changes

pandas/core/config_init.py Show resolved Hide resolved

Marc Garcia added 5 commits June 11, 2019 11:54

Merging from master

f5233f3

Removing checks to see if plotting backens implement the API

f7c6e33

Removing tests related to previous commit

1095344

Merging from master

e832985

Fixing failing test, and unifying get_plot_backend code

b13a74b

TomAugspurger reviewed Jun 20, 2019

View reviewed changes

Marc Garcia added 2 commits June 20, 2019 13:57

Fixing typo in whatsnew

001c57b

Merge remote-tracking branch 'upstream/master' into select_backend

231094e

TomAugspurger approved these changes Jun 20, 2019

View reviewed changes

jreback added this to the 0.25.0 milestone Jun 21, 2019

jreback approved these changes Jun 21, 2019

View reviewed changes

datapythonista added 2 commits June 21, 2019 08:15

Merge remote-tracking branch 'upstream/master' into select_backend

aa27d34

Merge branch 'select_backend' of github.com:datapythonista/pandas int…

f31fc67

…o select_backend

Adding import lost in the merge

1f4e1c0

datapythonista merged commit 2243629 into pandas-dev:master Jun 21, 2019

DougBurke mentioned this pull request Jun 26, 2019

Supporting different backends in Sherpa sherpa/sherpa#635

Open

lordsutch mentioned this pull request Jul 23, 2019

No longer compatible with pandas >= 0.25.0 PatrikHlobil/Pandas-Bokeh#34

Closed

jbrockmendel mentioned this pull request Jul 25, 2019

Flaky matplotlib tests #27143

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PLOT: Add option to specify the plotting backend #26753

PLOT: Add option to specify the plotting backend #26753

datapythonista commented Jun 9, 2019 •

edited

Loading

datapythonista commented Jun 9, 2019

TomAugspurger Jun 9, 2019

datapythonista Jun 9, 2019

codecov bot commented Jun 9, 2019

codecov bot commented Jun 9, 2019 •

edited

Loading

datapythonista commented Jun 20, 2019

TomAugspurger Jun 20, 2019

TomAugspurger Jun 20, 2019

datapythonista Jun 20, 2019

TomAugspurger Jun 20, 2019

TomAugspurger Jun 20, 2019

datapythonista Jun 20, 2019

TomAugspurger Jun 20, 2019 •

edited

Loading

datapythonista Jun 20, 2019 •

edited

Loading

datapythonista Jun 20, 2019

TomAugspurger Jun 20, 2019

datapythonista Jun 20, 2019

TomAugspurger Jun 20, 2019

datapythonista Jun 20, 2019

jreback commented Jun 21, 2019

PLOT: Add option to specify the plotting backend #26753

PLOT: Add option to specify the plotting backend #26753

Conversation

datapythonista commented Jun 9, 2019 • edited Loading

datapythonista commented Jun 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 9, 2019

Codecov Report

codecov bot commented Jun 9, 2019 • edited Loading

Codecov Report

datapythonista commented Jun 20, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger Jun 20, 2019 • edited Loading

Choose a reason for hiding this comment

datapythonista Jun 20, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jun 21, 2019

datapythonista commented Jun 9, 2019 •

edited

Loading

codecov bot commented Jun 9, 2019 •

edited

Loading

TomAugspurger Jun 20, 2019 •

edited

Loading

datapythonista Jun 20, 2019 •

edited

Loading