2.0.0 Wishlist

[theengineear]

@chriddyp @Kully @cldougl

I haven't had many cycles to play around with the python api library lately, but recently got in a little deep for #640, where we've now got fully-unified api modules.

I think enough shifted (mostly additions, but the error handling is definitely not backwards compat) that we should open up a discussion on non-minor changes that we've been looking to add to the plotly module. Here's a list of topics:

• API request/response unification ([2.0.0] Unify API #651)
• Fix or drop support for matplotlib
• Add numpy as an official dep. and use it to speed up the library. (FF requires numpy #667)
• Related to above, clean up how we handle optional imports (Cleaner import checking #317)
• Stop auto-loading graph reference and just use packaged graph reference (Load graph reference from pkg copy of plot schema #614)**
• Add transforms to graph objects (Add support for transform graph objects #587) holding off on this until the transforms spec settles in plotly.js.
• Add frames for graph objects (Better Integrate frames into Figure #604)
• Add a sign_in validation plotly.sign_in should check authentication settings #525 (Validate signin #668)
• Fix up streaming? (this might be more work than it's worth at this point Revamp streaming #253) No time for this atm.
• Some module re-organizing. Most importantly, we should move FigureFactory into it's own sub-package and each Figure-generator should be it's own file. The tools.py module is enormous and is a constant point of pain for circular imports ([2.0.0] Relocate figure factory #652).

**In fact, if we do do this, we can just add some meta programming to auto-create a module full of all the objects from graph-reference. This way we don't have those dump "Scatter is not defined" errors, etc.

Anything else?

[theengineear]

I'm still going to be working mostly on frontend code, but I'd definitely be interested in making some non-trivial improvements here. Thoughts?

[chriddyp]

Add numpy as an official dep. and use it to speed up the library.

I'm OK with using this in all of our FigureFactory methods but I'd still like to keep this as a protected import. I think there will still be new users who might have trouble installing it on their machine with their default python distribution.

[chriddyp]

For 2.0.0, I think that we only need to consider backwards incompatible changes if we're going to version as according to SemVer. In other words, it doesn't need to be a big release.

Here's an interesting discussion on this: https://gist.github.com/jashkenas/cbd2b088e20279ae2c8e

So, from that list above, it seems like:

• Add transforms to graph objects
• Add frames to graph objects

could be done in either 1.x or later in 2.x

[theengineear]

Ok, I sorta thought we were trying to strive for SemVer. I feel pretty confident that I don't want to change how errors work in a non-major version change. I'd be pretty annoyed if I upgraded a minor version and all of a sudden all my exceptions were being caught like they were before!

The point about the 1% breakage is interesting, but it assumes that you can actually know what users are doing with your code.

Perhaps were just too stuck on the idea of having a major, romantic 2.0.0 release. I'm not suggesting that this needs to be monumental, but there are a couple breaking api changes that we'd like to make and we want to make those clear to users.

So, from that list above, it seems like...

👍 , those can be in 1.x.

[theengineear]

Now that I think about it a bit more, I don't think that the no-auto-load-graph-references needs to be in 2.x either, right?

@chriddyp ^^

[chriddyp]

Ok, I sorta thought we were trying to strive for SemVer.

Yeah sorry for the confusion, I think that we're striving for SemVer too. I just posted that link to mention the alternative where we put "big features" in major version releases (which we don't need to do if we're doing SemVer).

[chriddyp]

I don't think that the no-auto-load-graph-references needs to be in 2.x either, right?

👍 , yeah. The auto-updating makes it a little bit weird. If Version 1.20 removes graph-reference, then someone on Version 1.19 (with auto-updating graph reference) could end up with a newer graph reference than the user in Version 1.20. That being said, we only guarantee the graph reference that is actually shipped in the package.

[Kully]

Add numpy as an official dep. and use it to speed up the library.

This goes hand in hand with the trisurf optimization stuff which I still have to put back in (@choldgraf 's improvements).

[theengineear]

^^ @Kully , I'm going to fix up our optional imports and then we're going to just liberally use numpy scipy pandas in the FF stuff.

[choldgraf]

+1 for supporting the scientific python stack (numpy, scipy, maybe pandas). I think your target audience (at least the ones that want to use the python API) will already be using it.

[theengineear]

Getting there with frames support. It's tricky because it's a top-level attr on figure, which we already make up in the plotly python api (figure is not in plot-schema).

[theengineear]

Wow, everything done up in the top comment but matplotlib. @chriddyp any thoughts on that? I feel weird about keeping it around if no one has time to maintain it. It's sorta destined to be more and more broken...

[choldgraf]

what's the main drawback about supporting matplotlib? Their API is pretty stable IIRC. No idea if it's something that users actually want though. Although I can say that I recently went to a talk about Altair (another new plotting library in python) and one of the first questions was whether they could convert between altair plots and matplotlib. Though I think the plotly niche may be different enough...

[theengineear]

The main drawback is maintenance. It's just a matter of someone needing to go back and fix our current support. I'd love to support it, but I know I definitely don't have free cycles to patch it up. I remember that matplotlib 4.x broke our support and so only matplotlib 3.x really works for us (iirc).

I'm mainly just looking to open up this conversation since I think the current state of this isn't super user-friendly.

[choldgraf]

maybe it's something to deprecate for now unless you get folks clamoring to keep it. My intuition is that a person would use matplotlib for a different use-case than they'd use plotly. The only challenge is that matplotlib is by far the most widely-used plotting library in python, so I guess having a connection might make it easier for some people to play around with plotly.

[theengineear]

The only challenge is that matplotlib is by far the most widely-used plotting library in python

Yup. And this is why it would be really nice to have better support for conversions.

Hmm, this conversation has convinced me that we don't really want to 🔪 support for it now (maybe never). If that's the case, I'd rather not even deprecate it and just try to get someone to find time to improve it.

[choldgraf]

this is one of those situations where it'd be really nice to have a way to find out which functions / sub-modules people are calling from within open-source packages :P

[theengineear]

oh, wait, you didn't know that import plotly automatically starts a logger than sends all of your python programs to the US government for safe keeping?

[choldgraf]

you joke but a friend of mine is actually working on something like that right now ;)

https://github.com/arokem/popylar

Basically it just sends a cookie (as if you were visiting a website) to google analytics every time an import statement is made (or something like this). Kinda neat but the big challenge right now is to figure out how to prevent it from being abused etc.

[theengineear]

lol popylar, nice. yah, that's definitely a hard line to toe. really interesting though!

[chriddyp]

yeah, I think we should keep it, albeit in a fragile state. we still have a lot of docs about it (/plot.ly/matplotlib) that are neat for people to see