Run test suite on Python 3.10

[ricardoV94]

We should run our tests in Python 3.10 to see if anything is broken or dependencies are missing

[5hv5hvnk]

Hi, do we have to run tests from pymc/tests
pytest --cov=pymc pymc/tests/<name of test>.py
with python3.10 and updated version of requirements?

[ricardoV94]

Hi, do we have to run tests from pymc/tests
pytest --cov=pymc pymc/tests/<name of test>.py
with python3.10 and updated version of requirements?

You would need to replace the py37 environments here and create a new ones with py310

https://github.com/pymc-devs/pymc/tree/main/conda-envs

Then in the workflows bump the conda environment files by 1. For instance here it would become py38:

pymc/.github/workflows/pytest.yml

Lines 138 to 145 in 8b063f9

	activate-environment: pymc-test-py37
	channel-priority: strict
	environment-file: conda-envs/environment-test-py37.yml
	use-mamba: true
	use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
	- name: Install-pymc
	run: |
	conda activate pymc-test-py37

here py39:

pymc/.github/workflows/pytest.yml

Lines 223 to 230 in 8b063f9

	activate-environment: pymc-test-py38
	channel-priority: strict
	environment-file: conda-envs/windows-environment-test-py38.yml
	use-mamba: true
	use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
	- name: Install-pymc
	run: |
	conda activate pymc-test-py38

and here py310:

pymc/.github/workflows/arviz_compat.yml

Lines 71 to 78 in 8b063f9

	activate-environment: pymc-test-py39
	channel-priority: strict
	environment-file: conda-envs/environment-test-py39.yml
	use-mamba: true
	use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly!
	- name: Install pymc
	run: |
	conda activate pymc-test-py39

There might be other places in the worflows that need updating, and probably in this file: https://github.com/pymc-devs/pymc/blob/main/setup.py

@michaelosthege might be able to spot other things I am missing

[michaelosthege]

Are Aesara and aeppl already testing on 3.10?
If not that should be the first step.

Also to not increase our CI resource use by a third, I think we should first combine the workflows and reorganize the test suite. Then we can more easily split the load across Python versions.

[ricardoV94]

Are Aesara and aeppl already testing on 3.10?
If not that should be the first step.

I think they are. Aesara for sure, Aeppl doesn't matter as it relies on Aesara alone

Also to not increase our CI resource use by a third, I think we should first combine the workflows and reorganize the test suite. Then we can more easily split the load across Python versions.

Why increase by a third? I would just bump everything one version up and drop the 3.7, which is no longer supported by numpy anyway.

[ricardoV94]

Correction, Aesara seems to test only 3.7-3.9

[ricardoV94]

For reorganizing the test suite, there's a somewhat related discussion here: #5579

[5hv5hvnk]

@ricardoV94 I created a new environment for python3.10, did the expected changes in pytest.yml file and executed the script to run tests, the output was very similar to comment

Should I try rename the tests in a manner as mentioned in #5579 ?

[michaelosthege]

Should I try rename the tests in a manner as mentioned in #5579 ?

No, that will need a lot of refactoring. To set realistic expectations: This will take months; we can only do one at a time. Splitting test_distribution.py and test_distribution_moments.py has the highest priority.

Can you try to run some of the tests locally in a Python 3.10 environment? pytest -v pymc\tests\test_sampling.py
The Aesara CI only tests with Python 3.8.. But conda-forge has it listed for 3.10 so maybe it could actually work.

[5hv5hvnk]

it completes with 81 passed, 5 skipped, 6 xfailed, 71 warnings

[michaelosthege]

Sounds promising. Do you want to open a PR?

Note that we don't want to increase our use of CI resources here. So just swapping the version of an existing job should be enough

[5hv5hvnk]

I have opened a PR, please have a look

Note that we don't want to increase our use of CI resources here. So just swapping the version of an existing job should be enough

what does CI resources imply I have read this in a lot of your comments and don't know what it means exactly

[michaelosthege]

The cumulative runtime of our CI test pipelines was 6.5 hours and we just reduced it to 3.75.
This is still a lot of compute on quite powerful machines and they eat a lot of electricity doing that.
We don't have to pay for it, but we all live on this planet together and should save energy wherever we can.

Also there's an upper limit of CI minutes per month and if we hit that the jobs start queuing up.

[ricardoV94]

In case that was the source of confusion, CI stands for continuous integration, which is another obscure name for "running python tests on Github servers"