Jupyterlab 3.x.x support

[bdecato]

With a fresh install of Jupyterlab version 3.0.0, installing plotly with the command jupyter labextension install [email protected] yields an error:

An error occured.
ValueError: 
"[email protected]" is not compatible with the current JupyterLab
Conflicting Dependencies:
JupyterLab                        Extension      Package
>=3.0.0 <3.1.0                    >=1.3.0 <2.0.0||>=2.0.0 <3.0.0@jupyterlab/rendermime-interfaces

Obviously this is a bleeding edge issue and very much a feature request, not a bug, I just wanted to make sure it was documented. Thank you!

[ptyshevs]

It will be also nice to have an option to disable version check during install. Who knows, maybe it doesn't have any breaking changes.

[nabetse00]

I've tested locally by cloning plotly.py repo and modifying the package.json line 34 to:

"@jupyterlab/rendermime-interfaces": "^1.3.0 || ^2.0.0  || ^3.0.0",

So far so good, no issues.

So steps to try that workaround:

• install jupyterlab v3 (i recommend conda to do this) and plotly

conda install -c conda-forge jupyterlab

• install nodejs (i used my local version v12 but >v12 should work too).
• run jupyter labextension install [email protected] you will get the same error as @bdecato said:

An error occured.
ValueError: 
"[email protected]" is not compatible with the current JupyterLab
Conflicting Dependencies:
JupyterLab                        Extension      Package
>=3.0.0 <3.1.0                    >=1.3.0 <2.0.0||>=2.0.0 <3.0.0@jupyterlab/rendermime-interfaces

• clone plotly.py repo in another folder (TMP, EXT whatever ... ) and open the correct folder for the extension

mkdir EXT
cd EXT
git clone https://github.com/plotly/plotly.py.git
cd .\plotly.py\packages\javascript\jupyterlab-plotly\

• modify package.json line 34 to

"@jupyterlab/rendermime-interfaces": "^1.3.0 || ^2.0.0  || ^3.0.0",

• Build the extension and install; Inside this folder run:

npm instal
npm run build 
jupyter labextension install

• once it finishes, check the extension list:

jupyter labextension list

returns something like:

JupyterLab v3.0.0
Other labextensions (built into JupyterLab)
   app dir: C:\XXXXXXX\YYYYYY\jupyter\lab
        @jupyter-widgets/jupyterlab-manager v3.0.0 enabled ok
        jupyter-matplotlib v0.7.4 enabled ok
        jupyterlab-plotly v4.14.1 enabled ok*


   local extensions:
        jupyterlab-plotly: C:\XXX\EXT\plotly.py\packages\javascript\jupyterlab-plotly

Ready to test v3 compat !

[cvanelteren]

Although the workaround from @bdecato installs the plugin. FigureWidget is broken as of jlab v3 embedding inside ipywidgets V/Hboxes. Running them standalone works fine.

[nabetse00]

@cvanelteren did you install Jupyter widgets extension:

jupyter labextension install @jupyter-widgets/jupyterlab-manager [email protected]

Official doc say it's optional but maybe needed for widgets in ipywidgets.

[cvanelteren]

Yes I have. For the time being I downgraded jupyerlab back down.

On Sun, 3 Jan 2021 at 10:10, nabetse ***@***.***> wrote: @cvanelteren <https://github.com/cvanelteren> did you install *Jupyter widgets extension*: jupyter labextension install @jupyter-widgets/jupyterlab-manager ***@***.*** Official doc say it's optional but maybe needed for widgets in ipywidgets. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2994 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEUVDV74GKQVOMHKRHVMYLLSYAX7ZANCNFSM4VJKLKEA> .

-- - Checkout my website <https://cvanelteren.github.io/> for contact info and current projects!

[jasongrout]

FYI, see the ipywidgets changelog for instructions for maintainers to upgrade widgets packages: https://ipywidgets.readthedocs.io/en/stable/changelog.html#updates-for-widget-maintainers

[jasongrout]

CC @nicolaskruchten - see my comment above for tips to upgrade for JupyterLab 3. This hopefully should be a pretty straightforward upgrade.

[cvanelteren]

Hi @jasongrout , thanks for the tip. Unfortunately I am running the latest ipywidgets already. There is a current PR that should fix the issue.

[jasongrout]

There is a current PR that should fix the issue.

Nice! What PR is open? I couldn't find one.

[nicolaskruchten]

Thanks @jasongrout! We'll look at getting JupyterLab 3.x support in the next week or so.

[barathvb]

Would older versions of plotly (such as 4.8.x) get JupyterLab 3.x support?

[nicolaskruchten]

In general, newer versions of the extension play nicely with older versions of the Python module, so tentatively "yes".

[nicolaskruchten]

@jasongrout thanks for the pointer re Widgets! I found this migration guide for non-widget extensions in general https://jupyterlab.readthedocs.io/en/latest/extension/extension_migration.html but if I read it correctly, following it will mean my extensions are JLab 3 compatible but not compatible with JLab 2, is that right?

[jasongrout]

The extension upgrade script mentioned in that guide makes a prebuilt extension, which is only supported by jlab 3.

[jasongrout]

However, you should be able to publish the npm package in a prebuilt extension package to get a jlab2-compatible extension, if you make sure the dependencies still are compatible.

[nicolaskruchten]

OK, I think I'd missed that this guide was intended to make a 2-compatible source extension into a 3-compatible prebuilt extension, thanks! BTW I've opened up a thread here https://discourse.jupyter.org/t/making-extensions-work-with-versions-2-and-3/7473 which might be a more-public place to discuss this :)

[nicolaskruchten]

Basically at this point I want to make a minimal set of changes to our extensions to get JLab3 support (without dropping support for JLab2), but I'm not quite ready to document/deal with all the support overhead of the prebuilt extension at the same time.

[jasongrout]

For you, I would probably try just changing

plotly.py/packages/javascript/jupyterlab-plotly/package.json

Line 36 in 45ddf75

"@jupyterlab/rendermime-interfaces": "^1.3.0 || ^2.0.0",

to

    "@jupyterlab/rendermime-interfaces": "^1.3.0 || ^2.0.0 || ^3",

and then follow the guidelines at https://ipywidgets.readthedocs.io/en/stable/changelog.html#updates-for-widget-maintainers for updating https://github.com/plotly/plotly.py/blob/master/packages/javascript/plotlywidget/package.json

[nicolaskruchten]

Thanks! For jupyterlab-plotly that's working for me, and I'm currently messing around with the widget.

I'm sure looking forward to a future a year from now where most folks are using prebuilt extensions :)

[nicolaskruchten]

OK, looks like this is really straightforward! Thanks again for the help @jasongrout :)

[nicolaskruchten]

(I'll close this issue again once the extension is actually generally available :)

[andreamoro]

Hopefully that simple change will be introduced by the team anytime soon, though I appreciate this might be related to the fact not all the graphs seem to work.

[nicolaskruchten]

OK, the new extensions are on NPM. Simply uninstalling the old extensions and installing the new ones. The new version number is 4.14.2. The documentation site is being updated right now and should be live in less than 20 minutes.

[jasongrout]

@nicolaskruchten - see #3016 (comment)

[mungojam]

As I understand it jupyterlab 3 allows for extensions to be published as pypi packages that don't require node to be installed.

That would be really useful for some of our users because they only work with python so don't have node set up by default.

Any idea if it would be possible for the plotly extension?

[nicolaskruchten]

It is, and we'll likely do it at some point, but ideally we'd find a way to merge the two extensions first, otherwise users would end up loading the large plotly.js bundle twice, as my understanding is that the pure-python extensions don't get deduplicated the same way as the source-only extensions.

[jasongrout]

pure-python extensions don't get deduplicated the same way as the source-only extensions.

Not exactly the same way, but there is deduplication. Getting deduplication for separate bundles of javascript was the main feature enabling the new "prebuilt" extensions.

[jasongrout]

See https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html#deduplication-of-dependencies

[nicolaskruchten]

Ah, thanks @jasongrout, I hadn't quite gotten that far down this path, I was just glad to get JLab 3 support out the door quickly :)

So yes, we definitely want to go this route, and we'd welcome any help from the community that anyone wants to provide in terms of how to package all this stuff up such that I can easily just turn the crank on the release process every time there's a version bump!

The JS source for the existing extensions is here: https://github.com/plotly/plotly.py/tree/master/packages/javascript and in general we've kept the source trees for Python packages in the sibling here https://github.com/plotly/plotly.py/tree/master/packages/python so I guess we'd need some kind of script I can run during each release process that takes over after I've upgraded the JS bundles and builds and pushes the Python bundles up to PyPI? The release process I manually follow is described here: https://github.com/plotly/plotly.py/blob/master/release.md

[jasongrout]

Installing an extension in JupyterLab no longer requires the package be published to npm (the js file bundle is included in the pip package). Of course, you'd still want to publish to npm if anyone is using something from the package, or if you are using it outside of jupyterlab.

So one question is: do you still need or want to publish one or both packages to npm?

[nicolaskruchten]

I think we'll want to publish both types of extensions for a while, just to give people options and not change things too fast.

[nicolaskruchten]

(Unless there's a big downside to this that I don't see atm)

[jasongrout]

No big downside - just thinking of how the release process may be simplified.

I think probably a good first step might be to look at the widget-cookiecutter, which has been updated to be a prebuilt extension for jlab 3.

Another design consideration - it seems the plotly python package is pretty heavy. You might make a new python package with just the jlab prebuilt extension, so people can install it separately if they wish, and make plotly depend on it. Bonus is that then you'd probably be able to use the widget cookiecutter to do almost all of your work, and upgrades could probably track the cookiecutter pretty closely.

We did something similar with ipywidgets itself. ipywidgets depends on widgetsnbextension (the classic notebook extension) and jupyterlab_widgets (the jlab prebuilt extension), so you can install either extension individually, but they also come automatically with ipywidgets.

[nicolaskruchten]

Thanks! I need to carve out some time to look at all these options and think about them... Right now on the JupyterLab side we have two extensions: one for the FigureWidget (the plotlywidget extension) and one for the normal Figure using the mimetype display system (the jupyterlab-plotly extension). But the plotlywidget extension code also comes built-in to the plotly package for use in Notebooks (outside of Lab).

Is there some obvious way to clean up all of this? We haven't merged our two extensions together, primarily because of the way we bundle the plotlywidget bit for Notebooks I believe. (tagging @jonmmease who knows more about this than I do ;)

[nicolaskruchten]

I'll add that I'm pretty conservative when it comes to these things, and that I like to try to find a way to do and document things that are likely to work with various older versions, so in the short run I think we'll want to keep publishing the source-only extensions and documenting how to install them (as those instructions are still valid for JupyterLab 2, and possibly even 1, given that no one has complained about 1) and then if there's a new/nicer thing for JupyterLab 3 I'd start publishing the requisite extensions to PyPI and/or conda or whatever, and document that as "an alternative for JupyterLab 3 users with the following benefits", and then once 3 is more widely adopted I'd switch the docs to "prebuilt is the way, unless you're on 2" etc :)

[jasongrout]

The most conservative thing to do is to use the widget-cookiecutter to transform the widget extension, and the jlab extension cookiecutter to transform the renderer extension into prebuilt extensions. This would create two new python packages, each installing one of the js packages into jlab. The widget cookie cutter also generates an nbextension, etc.

[nicolaskruchten]

The most conservative thing to do...

(in addition to the current publishing to NPM for JupyterLab 2 support, right? or does this approach somehow work with JLab 2 also?)

The widget cookie cutter also generates an nbextension, etc.

So we wouldn't need to bake plotlywidget into plotly for non-Lab Notebook support any more?

[jasongrout]

(in addition to the current publishing to NPM for JupyterLab 2 support, right? or does this approach somehow work with JLab 2 also?)

The widget and rendermime should be published to npm and continue to work for jlab 2 (test it, of course :).

[jasongrout]

So we wouldn't need to bake plotlywidget into plotly for non-Lab Notebook support any more?

I'm not sure exactly what you are asking here. The widget cookiecutter would keep your current behavior of having a python package that generates an nbextension, etc.

[nicolaskruchten]

I'm not sure exactly what you are asking here

It's not entirely clear in my mind either TBH, I'm mostly doing light updates and releases with code that @jonmmease wrote a couple of years ago and that doesn't break very often ;) I'll regroup with Jon and come up with a plan.

I've created #2994 for further conversation about the prebuilt extensions, and I'll lock this issue now to prevent too many "me too" and "it's not working in my particular setup" type comments.

If folks are having problems using versions of plotly more recent than 4.14.3 in JupyterLab 3, and you've gone through the troubleshooting guide, please feel free to open a new issue :)

[nicolaskruchten]

OK, if someone wants to kick the tires on it, pip install plotly==5.0.0rc1 should "just work" and install the prebuilt extension and it'll all just work in JLab3 regardless if you have our other extensions already installed.