Search-and-replace a bunch of strings to port to v4.

[twiecki]

Here is the script I ran:

#!/bin/bash

find ./ -name "*.ipynb" -type f -exec sed -i -e 's/az.from_pymc3/pm.to_inference_data/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/arviz.from_pymc3/pm.to_inference_data/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/pymc3/pymc/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/PyMC3/PyMC/g' {} \;
find ./ -name "*.py" -type f -exec sed -i -e 's/pymc3/pymc/g' {} \;
find ./ -name "*.py" -type f -exec sed -i -e 's/PyMC3/PyMC/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/from theano import tensor as tt/import aesara.tensor as at/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/import theano\.tensor as tt/import aesara.tensor as at/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/tt\./at./g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/aet/at/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/studenat/studentt/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/theano/aesara/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/Theano/Aesara/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/pm\.sample()/pm.sample(return_inferencedata=False)/g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/, return_inferencedata\=True//g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/return_inferencedata\=True, //g' {} \;
find ./ -name "*.ipynb" -type f -exec sed -i -e 's/return_inferencedata\=True//g' {} \;
pre-commit run --all-files

We probably need a new branch for v4 here.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[twiecki]

Does arviz.from_pymc() exist now?

[OriolAbril]

Does arviz.from_pymc() exist now?

No, the converter was moved to PyMC codebase, it is now pymc.to_inference_data

---

@michaelosthege can you work on this PR to see which run and first merge those? Then we can take a look at the rest in a bit more detail. I don't think it helps anybody to have code published in the website that doesn't reflect the outputs and might not even run. Given that we are trying to automate running the notebooks I think if we do that we can skip the "wait for users to report this is not working" step and just update those that do work.

[twiecki]

@OriolAbril Alright, I changed to use the new function and updated.

[michaelosthege]

Okay, starting tomorrow.
Learning from last years experience I'll automate the git commit and git push parts too.

I think I'll create a Python script that takes CLI args to be parametrized for re-running one notebook, committing on success & pushing to a branch.
Then I can contribute that script to a place in pymc-examples so other people can automate re-runs too.
Something like:

python scripts/rerun.py \
    --notebook=path_to_a_notebook \
    --to_branch=branchname

Given that magic script, which strategy do you prefer for the re-runs this week?

All-in-one

1. Check out main OR v4_search_replace
2. Attempt re-runs for all notebooks & commit the successful ones to a rerun-all branch.

And then we can PR rerun-all back into v4_search_replace OR main.

One PR per notebook

Assuming that we merge v4_search_replace first:

1. Check out main
2. For each notebook:
   1. Attempt re-run
   2. On success: commit to rerun-notebook-xyz, push, open a PR "Re-run notebook xyz"

[OriolAbril]

If the rerun all branch only has notebooks that were executed successfully, then a single PR is fine. I think we should then merge it into main.

I am strongly opposed towards merging notebooks with modified code that have not been executed. I think this is net confusion gain from all the angles I try to look at it. And users are already confused about notebooks using v3 explicitly not running on v4 directly, I can only imagine this getting much worse if it even has v4 code even though it doesn't run and we'll know it doesn't run.

[twiecki]

@OriolAbril If we merge into main, where will v3 notebooks live?

But in general agree, if we can check before merging that's better, no use in merging broken NBs.

[michaelosthege]

I am strongly opposed towards merging notebooks with modified code that have not been executed.

I agree. Shall I then make the bash script @twiecki wrote a part of this weeks re-run automation?

1. git checkout main && git checkout -b v4_search_replace2
2. For each notebook
   3. Run Thomas' API migration script
   4. Run the notebook
   5. On success: git add notebook-xyz && git commit -m "Re-run notebook xyz for v4" && git push origin v4_search_replace2

[OriolAbril]

@OriolAbril If we merge into main, where will v3 notebooks live?

I'll make a snapshot right before merging that will be available from the version switcher in readthedocs. But the example gallery will from now on be unversioned. Links to it should point to latest unless you are writing a book or citing a notebook and want to ensure what you link to doesn't get updated.

Moreover, v3 docs already have the notebooks embedded in them (from the submodule) time, so even if we have a kind of v3 last snapshot, 99% of users won't get to it. v3 notebooks have already been frozen for a while.

---

sounds good @michaelosthege. and that script or part of it could then be adapted to run notebooks on ci twice a year or so (even if only 50% get autoupdated and we have to wait for the rest to be updated from time to time manually).

[twiecki]

As we're going the rerun route I'm closing this.