Skip to content

Find way to make flaky tests less flaky on CI #3587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
etpinard opened this issue Feb 27, 2019 · 0 comments
Closed

Find way to make flaky tests less flaky on CI #3587

etpinard opened this issue Feb 27, 2019 · 0 comments
Assignees

Comments

@etpinard
Copy link
Contributor

etpinard commented Feb 27, 2019

Since about a week ago, the test-jasmine CI test container has been failing roughly (my estimate) 1 out of two runs. Of those failure most are due to

retry npm run test-jasmine -- --tags=flaky --skip-tags=noCI

We should try to find a solution.

Potential solutions

  1. split off that
retry npm run test-jasmine -- --tags=flaky --skip-tags=noCI

into a new test container (named test-jasmine3)

  1. use shard_jasmine_tests.js util as in our @gl tests:

SHARDS=($(node $ROOT/tasks/shard_jasmine_tests.js --tag=gl))
for s in ${SHARDS[@]}; do
retry npm run test-jasmine -- "$s" --tags=gl --skip-tags=noCI
done

to split off the @flaky tests into multiple runs.

  1. combine 1) and 2)

@antoinerg @archmoj wanna give this a shot?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants