Suboptimal performance when rendering large amount of traces

[hy9be]

Here is the testing script

I kept the total data points fixed, and tested different combinations of amount of traces and amount of data points per trace:

You could see that 5000 traces needs 17 seconds to render even it only renders 5000 markers in total. But for the case of only 1 trace, scattergl could use that 17 seconds to render over 10 million markers!

And by looking into the code I found that for each trace we created a Scene2D and a Shader, which get the shader reference and rebuild program. Is there anything we can optimize here?

[hy9be]

Another question is since we are using Shader, why it looks all the points are plotted sequentially? Correct me if I am wrong, one of the biggest advantages of using Shader is to leverage the parallel computing paradigm of GPU.

[hy9be]

A timeline profile screenshot:

[etpinard]

Thanks for the very detailed report!

We could indeed optimize scattergl so that it groups traces into the least number of gl-vis objects (see gl-scatter2d and gl-scatter2d-sdf) possible, but since that can already (easily) be done by the users pre-Plotly.plot this is not high on our priority list at the moment.

Performance for graphs with numerous traces is probably plotly.js' Achilles heel at the moment (not just WebGL based traces).

[hy9be]

@etpinard Thanks for the reply! I will try to group the traces first.

And I looked into a non-glscatter example. It looks the time consuming phases are:

• generate uid for each trace in cleanData
• supplyDataDefaults for each trace, for converting strings into objects, and processing color strings
• sort scatter layers
• apply style for each trace

May I ask the specific reason for that "Achilles heel", like the piece of leaf? Thanks!

[hy9be]

@etpinard I did some optimization to group and stacked bars when there are large amount of traces. Could any of you guys take a look? Thanks!

#1544

[hy9be]

@etpinard I opened another PR to optimize the performance of scatter plot, when there are large amount of traces. All tests passed. FYI: #1555

[etpinard]

Addressed in #1555

This is of course not the end of our multi-trace performance problems ...