Skip to content

Major refactor of the image test scripts #656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 20, 2016
Merged

Major refactor of the image test scripts #656

merged 6 commits into from
Jun 20, 2016

Conversation

etpinard
Copy link
Contributor

@etpinard etpinard commented Jun 17, 2016

What this PR adds

  • The ability to run the image pixel comparison tests in queue (i.e. with no concurrency). This is necessary step for bringing CircleCI image test support for our new mapbox plot type. More info in Introducing plotly.js + Mapbox GL #626. It may also be convenient for devs working on weak hardware.
  • Add a glob handler for the first argument to npm run test-image, npm run basesline and npm run test-export allowing devs to more easily target test mocks.

What this PR cleans up

  • Test folder in build/test_images/ and build_test_images_diff are now spawn during npm run pretest
  • Common logic steps in test-image, baseline and test-export are now placed in test/image/assets/.

Examples

# Run all tests in batch:
$ npm run test-image

# Run the 'contour_nolines' test:
$ npm run test-image -- contour_nolines

# Run all gl3d image test in queue:
$ npm run test-image -- gl3d_* --queue

TODO

  • update contributing guide and image test readme

etpinard added 3 commits June 17, 2016 12:49
- init test/image/assets/
- move get_image_request_option to assets/
- move image test folders init to pretest.js
- factor out get image path logic in script into asset file
- add + centralize glob pattern matching
- add --queue option for compare pixel script
- run baseline generation in queue (instead of in batch)
var BATCH_SIZE = 5;

// wait time between each test in test queue
var QUEUE_WAIT = 10;
Copy link
Contributor Author

@etpinard etpinard Jun 17, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking about making changing these to:

var QUEUE_WAIT = process.env.PLOTLYJS_IMAGE_TEST_QUEUE_WAIT || 10;

to better handle machine-to-machine differences

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either env variables or a .gitignore'd config file works for me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok great. I'll do this in a future PR.

@etpinard
Copy link
Contributor Author

cc @mdtusz the diff is a little hard to read, I suggest opening the files separately:

* machine-to-machine; skip over them for now.
*
*/
function untestableFilter(mockName) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be useful/convenient to have these as a config variable array and just check indexOf over all of them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be useful for sure. But ideally, this filter wouldn't be there in the first place. So I'd vote for the solution of least-surface-area i.e. this one ⏫ .

@mdtusz
Copy link
Contributor

mdtusz commented Jun 20, 2016

💃 merge away!

@etpinard etpinard merged commit d16393b into master Jun 20, 2016
@etpinard etpinard deleted the image-test-2.0 branch June 20, 2016 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants