percy tests

[emmanuelle]

No description provided.

[emmanuelle]

This PR is ready for review. The percy snapshots are on https://percy.io/plotly/plotly.py/builds/2538608. The 3d and mapbox snapshots don't look good, I still have to investigate this.

[nicolaskruchten]

Nice! So the 3d and mapbox stuff basically won't work at all because Percy snapshots the DOM and the 3d stuff is in a canvas3d/webgl context whereas the mapbox stuff makes external requests for images which probably aren't done or aren't available at snapshot-time. This is already a big step forward in terms of locking down regressions though!

In order to move forward I think we should add Percy to the Github checks and ensure that Jon has access to Percy (heads-up @bpostlethwaite!)

[emmanuelle]

OK thanks, I'll remove the 3d and mapbox stuff for now then.

[nicolaskruchten]

Oh actually I would leave them in! It proves that they don't error out, and we do get some benefit from knowing that the legends etc render as expected IMO

[emmanuelle]

I think adding Percy to the Github checks needs to be done by someone with admin rights on Percy, cc @bpostlethwaite

[nicolaskruchten]

@emmanuelle can you rebase onto master please? I think I've wired up percy correctly

[emmanuelle]

Interestingly, in the case of animations the snapshot seems to be at a random position, hence the visual diff here.

[emmanuelle]

Should we avoid animations in the visual test suite? Or is there a way to freeze the animation at a given frame?

[nicolaskruchten]

Hehe yeah. Are the animations in auto-play mode? that won't work, as Percy isn't quite that consistent about timing :) either that or we need to use animations with a single frame or something

[nicolaskruchten]

but in general: hurray, it's working!

[emmanuelle]

yes, it is auto-play. It's pretty cool that we could detect this with percy :-)

[emmanuelle]

I was thinking maybe of having a very large duration between frames so that always the same frame is captured...

[emmanuelle]

so fig.write_html can set auto_play to False, this is probably the solution

[nicolaskruchten]

💃