Skip to content

Fix Flaky Tests #759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rmorshea opened this issue Jun 13, 2022 · 1 comment
Closed

Fix Flaky Tests #759

rmorshea opened this issue Jun 13, 2022 · 1 comment
Labels
flag-good-first-issue A well defined and self-contained task. priority-2-moderate Should be resolved on a reasonable timeline.

Comments

@rmorshea
Copy link
Collaborator

rmorshea commented Jun 13, 2022

Current Situation

There are a number of tests in the suite that are very flaky (usually requiring 2-3 runs to pass). For new contributors this is likely to be confusing since they'll be unsure if those failures are related to their changes. Unfortunately the problems are hard to reproduce since running the local tests rarely, if ever, fails at the rate seen in CI/CD. I've also noticed that there are times when the CI/CD will pass on the first go, while on others it will simply refuse to pass.

Proposed Actions

It's likely that these flaky tests are missing some sort of locking mechanism to ensure that actions occur in the correct order (or at least that's been my experience in the past).

@rmorshea rmorshea added the priority-2-moderate Should be resolved on a reasonable timeline. label Jun 13, 2022
@rmorshea rmorshea added this to the 2.0 milestone Jun 13, 2022
@rmorshea rmorshea removed this from the Luxury milestone Feb 21, 2023
@Archmonger Archmonger added the flag-good-first-issue A well defined and self-contained task. label Feb 23, 2024
@Archmonger
Copy link
Contributor

Resolved in #1251

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flag-good-first-issue A well defined and self-contained task. priority-2-moderate Should be resolved on a reasonable timeline.
Projects
None yet
Development

No branches or pull requests

2 participants