add_hline and add_vline (+add_vrect/add_hrect)

[nicolaskruchten]

We should add these convenience methods, as the ref=x/ref=paper trick is pretty hard for people to understand.

We'd want to be able to add these to all subplots, one subplot, or possibly a single row or a single column. Moved to #2140

We'd also want add_vspan and add_hspan (or vrect/hrect, not set on the names)

[emmanuelle]

Also requested in https://community.plot.ly/t/speeding-up-fig-add-shape-calls-in-python/35880/2

[nicolaskruchten]

Related to #2140

[jvschoen]

A quick solution would be:

fig.layout.update(shapes=[{'type': 'line','y0':y_intercept,'y1': y_intercept,
                           'x0':str(df.x_value.min()), 'x1':str(df.x_value.max()),
                           'xref':'x' + str(i + 1), 'yref':'y' + str(i + 1),
                           'line': {'color': 'black', 'width': 1, 'dash': 'dot'}} 
                          for i, member in enumerate(pd.unique(df.x_value))])

[jvschoen]

Though an issue arrises if you want to add both an x and y line, the second update will overwrite the former and you only see the last added line. As a workaround for that :

list(np.array([ 
        [
        # Horizontal Line
        {'type': 'line',
         'y0': df.y_val.min(),'y1': df_faces.y_val.max(),
         'x0': df.x_val.median(), 'x1': df.x_val.median(),
         'xref':'x' + str(i + 1),
         'yref':'y' + str(i + 1),
         'line': {'color': 'black','width': 1, 'dash': 'dot'}},
        # Vertical Line
        {'type': 'line',
         'y0': df.y_val.median(),'y1': df..median(),
         'x0':str(df.x_val.min()), 'x1':str(df.x_val.max()),
         'xref':'x' + str(i+1),
         'yref':'y' + str(i + 1),
         'line': {'color': 'black','width': 1, 'dash': 'dot'}}
        ]
        for i , member in enumerate(pd.unique(df.facets_val))]).ravel())

So now I see why you referred to this current usage as a bit clunky

[nicolaskruchten]

Well we have fig.add_shape() already that doesn't overwrite :)

But what I'm hoping for is a way to have a single function call fig.add_hline(5) which will add a line at y=5 to all subplots, and then some options like fig.add_hline(5, row=1, col=2) for individual subplots :)

[MrPaulAR]

I would also ask that you consider the ability to add text to the upcoming "add_hline" and "add_vline" items.

[anuraghuram]

There should also be an option to resize v/h lines to fit the plot's min and max y/x axis respectively. I notice that when I add a vline over an existing plot, it runs off the plot (as I presume is happening in the screenshots provided in this thread plotly/plotly_express#143)

[nicolaskruchten]

@nicholas-esterer let's start with this issue for now, without tackling the "all subplots" or "one row of subplots" just yet. Let's do the easy case to get going. cc @emmanuelle

[nicolaskruchten]

Let's do the "hard version" with the multi-subplot targeting as part of #2140

[nicolaskruchten]

@MrPaulAR we'll see what we can do about adding text. The underlying shapes mechanism doesn't support text today, so this may have to be done as a separate step with add_annotation although we could consider adding a text_annotation flag to these methods. Might get tricky with the various positioning arguments though.

[ghost]

while most discussion circles around a sugar api call, can we please have a feature to have a TRUE h/v line? timeseries &co dynamic data shifts around. especially with plotly callbacks i cant jugg the whole figure up/down the server just to extendTrace data. Please add a true horizontal / vertical line property with only one coordinate y and x respectively, since the other is infinitely extending.
Ugly workaround: oversize the x0 x1 on a hline and set the xaxis range tot he actual data so if it moves around hline shape covers due data within x0 x1 interval. BUT please do TRUE infinite x in this case. ?

[nicolaskruchten]

This new API does indeed implement true one-coordinate infinite h/v lines that cannot be panned or zoomed away.

[nicolaskruchten]

This has now been released as part of Plotly.py 4.12 ! https://community.plotly.com/t/announcing-plotly-py-4-12-horizontal-and-vertical-lines-and-rectangles/46783

[nathansegers]

@nicolaskruchten is it possible to use the add_vline with X-coordinates as X-axis data points?

For example, my x-axis data is class-based like [12345, 54321, 99999] and I want to plot a line only on the 54321.
I used to use this code:

fig.update_layout(
    shapes=[
      dict(
        type="line",
        xref='x', yref='y',
        x0=x_data[5], x1=x_data[5],
        y0=0, y1=125,
      )
    ]
)

But apparently, this gives different results in Plotly as it does in Dash.

Plotly

Dash

(The small part on the left is the zoomed-out result, because the vertical-line was plotted at 12.000.000 (which is the class of the X_data, but Dash seems to regard is as an integer)

This only happened in the latest updates (I used to work with plotly=4.8.0 before this error)

I wanted to give the add_vlines() a try, but it only stays on the "absolute" position, and does not scroll, which is what I need.

I don't know if I should make a different Issue or not.

[nicolaskruchten]

Hi @nathansegers, yes, please do make a separate issue for this :)

[jenna-jordan]

@nicolaskruchten The add_vline and add_hline methods are great, and I would really like to use them, but I have encountered a weird issue, where the line has a length of one. fig.add_hline(y=10) results in a line at 10 on the y axis but only extends from 0 to 1 on the x axis. fig.add_vline(x=1000000) results in a line at 1 million on the x axis but only extends from 0 to 1 on the y axis. Since my plot has max x values in the hundreds of billions, these lines are basically invisible (at least, until I change the axes to log scale, at which point the plot just looks strange. Screenshot of the log-scale mode below)

[nicolaskruchten]

@jenna-jordan if you're in JupyterLab you'll need to update your lab extension to v4.12 to get this feature to work as intended, and if you're in VSCode, you'll need to wait for the VSCode extension to be updated to the latest version of the Plotly.js rendering engine, unfortunately.

[jenna-jordan]

@nicolaskruchten I am rendering it with streamlit, actually, so I'm not sure how that effects things.

[nicolaskruchten]

@jenna-jordan ah, that explains it. Streamlit bundles plotly.js directly and they'll need to update their version in order to suppor this new feature.

[ievgennaida]

@nicolaskruchten thanks for those features. Is it possible to draw a legend for those lines?

[nicolaskruchten]

No legend entries for shapes at the moment, only for traces.

[ievgennaida]

@nicolaskruchten Why it was decided to have completely different processing and behavior of the shapes? I guess would be just easier to have the same logic and renderers as you have for traces, but it's just a guess... Now it's lacking legend, turn on/off functions.

I am trying to use a fake trace to solve this problem.
Is it possible to control whether a specific legend is clickable or not?
Another option would be to write a script to hide shapes when the trace is clicked.
Is it possible to control shapes collection visibility without removing them? Only the transparent color?

Thanks in advance!

[nicolaskruchten]

I wasn't involved in Plotly development when the original design decisions were made around shapes, but generally speaking, we consider traces to be "data" and shapes to be "annotations" and hence conceptually different things. The legend is itself a form of data-driven annotation so shapes don't contribute to it.

For the rest of your questions, a better place to ask them would be our forum at community.plotly.com as in Github we mostly try to keep track of bugs and development-related activities :)

[ievgennaida]

@nicolaskruchten thanks for the answer.

[Sank-WoT]

Performance issues fig.add_vrect

code

for index, row in dfUpload.iterrows():
  print(f’Step {index} addition {time.time() - start_time}’)
  fig.add_vrect(x0=row[‘BSTART’], x1=row[‘BEND’],
  annotation_text=row[‘nam’], annotation_position=“top right”,
  fillcolor=row[‘color’], opacity=0.3, line_width=0)
  print(f’Step {index} added {time.time() - start_time}’)

Step 0 addition 6.687209367752075
Step 0 added 6.690212249755859
Step 1 addition 6.690212249755859
Step 1 added 6.695227861404419
Step 2 addition 6.695227861404419
Step 2 added 6.70222806930542
Step 3 addition 6.70222806930542
Step 3 added 6.710227966308594
Step 4 addition 6.710227966308594
Step 4 added 6.7192277908325195
Step 5 addition 6.720227956771851
Step 5 added 6.73122763633728
…
Step 18 addition 7.010226488113403
Step 18 added 7.04421329498291
Step 19 addition 7.04421329498291
Step 19 added 7.080225944519043
Step 20 addition 7.080225944519043
Step 20 added 7.118224143981934
Step 21 addition 7.118224143981934
Step 21 added 7.158225774765015
Step 22 addition 7.158225774765015
Step 22 added 7.200225353240967
Step 23 addition 7.200225353240967
Step 23 added 7.243214845657349
Step 24 addition 7.244225025177002
Step 24 added 7.288224935531616
Step 25 addition 7.288224935531616
…
Step 40 addition 8.189055442810059
Step 40 added 8.263055086135864
Step 41 addition 8.263055086135864
Step 41 added 8.338054895401001
Step 42 addition 8.339054822921753
Step 42 added 8.41605281829834
Step 43 addition 8.41605281829834
Step 43 added 8.495052576065063
Step 44 addition 8.49605417251587
Step 44 added 8.576043367385864
Step 45 addition 8.576043367385864
Step 45 added 8.659053087234497
Step 46 addition 8.659053087234497
Step 46 added 8.744052648544312
Step 47 addition 8.745052576065063
Step 47 added 8.834052085876465
Step 48 addition 8.834052085876465
Step 48 added 8.92204999923706
Step 49 addition 8.923098087310791
Step 49 added 9.013051509857178
Step 50 addition 9.013051509857178
Step 50 added 9.105050802230835
…
Step 80 addition 12.666525840759277
Step 80 added 12.817530393600464
Step 81 addition 12.817530393600464
Step 81 added 12.97252607345581

As you can see from the log, performance down.
step 1
6.690212249755859 - 6.695227861404419 ~ 0.005
step 81
12.97252607345581 - 12.817530393600464 ~ 0.16

the difference is 32 times!!!

I am outputting 82 rectangles in 7 seconds !!!

Please, fix the problem.

[nicholas-esterer]

fig.add_vrect is basically a fig.add_shape call with some extra styling added on the new shape(s) (multiple shapes if you add to multiple facets). Do you see a similar performance hit when doing the same test but adding a shape with fig.add_shape, e.g., fig.add_shape(x0=1,x1=2,y0=3,y1=4)? Just trying to see if the problem is with the vrect specifically.

[Ben-Mathews]

@nicholas-esterer I have a similar issue where the time for individual calls grows as the number of shapes/vrects grows. It occurs with both add_shape() and add_vrect(), although add_shape() is about 3x faster. I see the same performance with Plotly 4.14.3 and 5.2.2.

The big issue here is that the cumulative time for all the function calls can get quite large for a large number of objects.

Thanks for your time and attention with this issue!

import time
import numpy as np
import matplotlib.pyplot as plt

t_add_shape = []
t_add_vrect = []

fig = go.Figure()
x = 0
for idx in range(0, 100):
    x0 = x + np.random.randn()
    x1 = x0 + np.random.randn()
    x = x1 + np.random.randn()
    t_start = time.time()
    fig.add_shape(x0=x0, x1=x1, y0=-1000, y1=1000)
    t_add_shape.append(time.time()-t_start)

fig = go.Figure()
x = 0
for idx in range(0, 100):
    x0 = x + np.random.randn()
    x1 = x0 + np.random.randn()
    x = x1 + np.random.randn()
    t_start = time.time()
    fig.add_vrect(x0=x0, x1=x1)
    t_add_vrect.append(time.time()-t_start)

plt.figure()
plt.plot(1e3 * np.asarray(t_add_shape), label='add_shape() times (ms)')
plt.plot(1e3 * np.asarray(t_add_vrect), label='add_vrect() times (ms)')
plt.grid()
plt.xlabel('Iteration Number')
plt.ylabel('Runtime (ms)')
plt.title('add_shape() vs add_vrect()')
plt.show()

[nicholas-esterer]

Thank you for your code. I used it to run some profiling (but I excluded the matplotlib plotting part at the end).

# add_shapes_profiling.py
from plotly import graph_objects as go
import time
import numpy as np

t_add_shape = []
t_add_vrect = []

fig = go.Figure()
x = 0
for idx in range(0, 100):
    x0 = x + np.random.randn()
    x1 = x0 + np.random.randn()
    x = x1 + np.random.randn()
    t_start = time.time()
    fig.add_shape(x0=x0, x1=x1, y0=-1000, y1=1000)
    t_add_shape.append(time.time()-t_start)

fig = go.Figure()
x = 0
for idx in range(0, 100):
    x0 = x + np.random.randn()
    x1 = x0 + np.random.randn()
    x = x1 + np.random.randn()
    t_start = time.time()
    fig.add_vrect(x0=x0, x1=x1)
    t_add_vrect.append(time.time()-t_start)

Then profile with

python3 -m cProfile -o /tmp/add_shapes_profile /tmp/add_shapes_profiling.py

I used this script to examine functions that were called more than 100 times, and sorted them by their time per call, and printed the number of calls as the last item.

# print_profile.py
import pstats
from pstats import SortKey
p = pstats.Stats('/tmp/add_shapes_profile')
timepercall_allcalls=sorted([
    (fname,tt/nc,nc) for fname, (cc, nc, tt, ct, callers) in p.stats.items()
],key=lambda t: t[1],reverse=True)

timepercall_allcalls=filter(lambda tup: tup[2] > 100,timepercall_allcalls)
for fname, tt_nc, nc in timepercall_allcalls:
    print(fname,'%g' % (tt_nc,), nc)

Searching for stuff in the plotly module, it shows where it might be good to optimize. It looks like plotly/graph_objs/layout/_shape.py:__init__ is called a lot and takes some time per call.

It would be interesting to do @Ben-Mathews chart but plot the number of calls of these "hot" functions.

[radek555]

Hi,
I am experiencing the same slowdown in orders of magnitude for just ~50 calls for add_vrect(). I tried with add_shape(type='rect') and it is also affected by similar slowdown.