Skip to content

Separate pytest workflow into separate workflows for tests and coverage #10542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from

Conversation

tianyizheng02
Copy link
Contributor

@tianyizheng02 tianyizheng02 commented Oct 15, 2023

Describe your change:

Following up from #10469, I've separated the pytest tests and coverage into two separate workflows. The goal is to cut down on the unneeded clutter in the logs and make it easier for contributors (and ourselves) to read.

  • Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • Documentation change?
  • GitHub workflows

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms include at least one URL that points to Wikipedia or another similar explanation.
  • If this pull request resolves one or more open issues then the description above includes the issue number(s) with a closing keyword: "Fixes #ISSUE-NUMBER".

@tianyizheng02
Copy link
Contributor Author

tianyizheng02 commented Oct 15, 2023

No matter what I do to omit these two files, pytest keeps including them in the coverage step and it's causing the step to fail. @cclauss, any ideas?

=========================== short test summary info ============================
ERROR quantum/q_fourier_transform.py - ModuleNotFoundError: No module named 'qiskit'
ERROR scripts/validate_solutions.py - KeyError: 'GITHUB_TOKEN'
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
======================== 2 warnings, 2 errors in 18.43s ========================
Error: Process completed with exit code 2.

@cclauss
Copy link
Member

cclauss commented Oct 15, 2023

mv quantum/q_fourier_transform.py quantum/q_fourier_transform.py.DISABLED

tianyizheng02 and others added 10 commits October 15, 2023 13:14
Separate existing pytest workflow into a workflow for running tests and a workflow for calculating test coverage
…eports

q_fourier_transform.py and validate_solutions.py are already left out of testing, so they should be left out of coverage reports as well in order to not cause errors
Change [tool.coverage.report] to [tool.coverage.run]. The former omits files from only reporting, while the latter omits files from both measuring and reporting.
Co-authored-by: Christian Clauss <[email protected]>
@tianyizheng02
Copy link
Contributor Author

I've been experimenting with pytest and pytest-cov for a while now and it seems like there isn't a straightforward way to separate the tests from the coverage report. Since pytest-cov is just a plugin for pytest, running the latter requires running the former, and I believe that must include the progress bar and warnings. Plus, this would require us to basically run pytest twice per build. Perhaps we could output the coverage report to a temporary file, but that's not ideal. I don't have any better solution at this time.

@tianyizheng02 tianyizheng02 deleted the test-cov-gh-action branch October 15, 2023 21:24
@cclauss
Copy link
Member

cclauss commented Oct 15, 2023

I was thinking that the coverage report could be piped into a file and then cat filename.txt could happen in a separate GHA step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants