Skip to content

Updated splines notebook to v4 #274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 7, 2022
Merged

Updated splines notebook to v4 #274

merged 5 commits into from
May 7, 2022

Conversation

fonnesbeck
Copy link
Member

Modest updates to convert to v4.

  • replaced PyMC3 with PyMC througout
  • now uses RNG seeding via Model instantiation
  • few corrections and updates to text

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@review-notebook-app
Copy link

review-notebook-app bot commented Jan 31, 2022

View / edit / reply to this conversation on ReviewNB

OriolAbril commented on 2022-01-31T00:15:37Z
----------------------------------------------------------------

Add a myst target with a notebook specific keyword so we can use it to link to this notebook from other pages of the documentation. https://docs.pymc.io/en/latest/contributing/jupyter_style.html#first-cell


@review-notebook-app
Copy link

review-notebook-app bot commented Jan 31, 2022

View / edit / reply to this conversation on ReviewNB

OriolAbril commented on 2022-01-31T00:15:37Z
----------------------------------------------------------------

the obs dimension here is only annotating, not setting the shape, so I believe that now with the improved dims/coords support it can be removed from coords, and using dims="obs" will still be valid.

Moreover, I think there was also a way of setting dim lengths only, so that instead of creating an arange, something like dim_lenghts={"splines": B.shape[1]} sets the lenght of the dimension for cases like this one where we don't care about coordinate values. @michaelosthege can you confirm this?


michaelosthege commented on 2022-02-06T14:25:17Z
----------------------------------------------------------------

Yes, the Model(coords=...) are only np.arange and those would be auto-generated.

You can write this:

w = pm.Normal("w", mu=0, sd=3, size=B.shape[1], dims="splines")

to automatically create a lenght-3 dimension tied to that (B.shape[1],)-sized RV.

@review-notebook-app
Copy link

review-notebook-app bot commented Jan 31, 2022

View / edit / reply to this conversation on ReviewNB

OriolAbril commented on 2022-01-31T00:15:39Z
----------------------------------------------------------------

with spline_model:
    idata = pm.sample_prior_predictive()
    idata.extend(pm.sample(
        draws=1000,
        tune=1000,
        random_seed=RANDOM_SEED,
        chains=4,
        return_inferencedata=True,
    ))
    pm.sample_posterior_predictive(idata, extend_inferencedata=True)

I think something like the above will work. arviz.from_pymc3 is for pymc3 only. the conversion to inferencedata in v4 is part of the pymc codebase and integrated in the sampling methods. If for some reason you still want to perform a conversion manually , you have to use in v4pymc.to_inference_data


@review-notebook-app
Copy link

review-notebook-app bot commented Jan 31, 2022

View / edit / reply to this conversation on ReviewNB

OriolAbril commented on 2022-01-31T00:15:39Z
----------------------------------------------------------------

Line #1.    wp = trace.posterior["w"].mean(("chain", "draw")).values

instead, we should avoid axis indexes when working with xarray. I don't have time to look into converting the cell into xarray and don't want to block the PR, but this is very easy fix


@review-notebook-app
Copy link

review-notebook-app bot commented Jan 31, 2022

View / edit / reply to this conversation on ReviewNB

OriolAbril commented on 2022-01-31T00:15:40Z
----------------------------------------------------------------

Add yourself here too. https://docs.pymc.io/en/latest/contributing/jupyter_style.html#authorship-and-attribution


Copy link
Member

Yes, the Model(coords=...) are only np.arange and those would be auto-generated.

You can write this:

w = pm.Normal("w", mu=0, sd=3, size=B.shape[1], dims="splines")

to automatically create a lenght-3 dimension tied to that (B.shape[1],)-sized RV.


View entire conversation on ReviewNB

@review-notebook-app
Copy link

review-notebook-app bot commented Feb 6, 2022

View / edit / reply to this conversation on ReviewNB

michaelosthege commented on 2022-02-06T14:25:49Z
----------------------------------------------------------------

Does this render correctly? Could be safer to go with a standard inline code formatting.


OriolAbril commented on 2022-02-06T14:31:11Z
----------------------------------------------------------------

it does and it's standard code formatting. The strange view is all because of reviewnb. The actual change is swapping apostrophes for backticks. Never trust reviewnb preview. What matters is what is shown in the website: https://pymc--274.org.readthedocs.build/projects/examples/en/274/splines/spline.html?highlight=splines#prepare-the-spline

Copy link
Member

it does and it's standard code formatting. The strange view is all because of reviewnb. The actual change is swapping apostrophes for backticks. Never trust reviewnb preview. What matters is what is shown in the website: https://pymc--274.org.readthedocs.build/projects/examples/en/274/splines/spline.html?highlight=splines#prepare-the-spline


View entire conversation on ReviewNB

@review-notebook-app
Copy link

review-notebook-app bot commented Apr 1, 2022

View / edit / reply to this conversation on ReviewNB

aloctavodia commented on 2022-04-01T07:06:04Z
----------------------------------------------------------------

Remove line

Below is an exmaple of how to fit a spline using the Bayesian framework PyMC.

We can also remove this (I found them very distracting)

As the book uses Stan (another advanced probabilitistic programming language), the modeling code is primarily taken from the GitHub repository of the PyMC implementation of Statistical Rethinking. My contributions are primarily of explanation and additional analyses of the data and results. Note that this is not a comprehensive review of splines – I primarily focus on the implementation in PyMC.

We can also direct people to https://bayesiancomputationbook.com/markdown/chp_05.html :-)


@review-notebook-app
Copy link

review-notebook-app bot commented Apr 1, 2022

View / edit / reply to this conversation on ReviewNB

aloctavodia commented on 2022-04-01T07:06:05Z
----------------------------------------------------------------

We are going to use 'patsy' library to generate the basis for the spline (more on that below).


@review-notebook-app
Copy link

review-notebook-app bot commented Apr 1, 2022

View / edit / reply to this conversation on ReviewNB

aloctavodia commented on 2022-04-01T07:06:05Z
----------------------------------------------------------------

I think this is also distracting from the main topic of the example


@review-notebook-app
Copy link

review-notebook-app bot commented Apr 1, 2022

View / edit / reply to this conversation on ReviewNB

aloctavodia commented on 2022-04-01T07:06:06Z
----------------------------------------------------------------

Why is this another way of visualizing splines can be confusing if splines has not been explained before. Also can be confusing given the previous explanation that splines are local fit that are "connected at the knots", because in this example the lines are not connected.

We can simply remove this in this PR, alternatively we can keep this and I offer myself to improve this section in a future PR


@review-notebook-app
Copy link

review-notebook-app bot commented Apr 1, 2022

View / edit / reply to this conversation on ReviewNB

aloctavodia commented on 2022-04-01T07:06:07Z
----------------------------------------------------------------

Again, not sure we need this kind of comments in our examples


@fonnesbeck
Copy link
Member Author

Should be good to go now.

@fonnesbeck fonnesbeck merged commit 1665736 into pymc-devs:main May 7, 2022
@fonnesbeck fonnesbeck deleted the splines_v4 branch May 7, 2022 17:44
:::{post} Oct 8, 2021
:tags: patsy, pymc3.Deterministic, pymc3.Exponential, pymc3.Model, pymc3.Normal, regression, spline
:::{post} May 6, 2022
:tags: patsy, pymc.Deterministic, pymc.Exponential, pymc.Model, pymc.Normal, regression, spline
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should not be pymc.<object> tags anymore, that generates way too many tags that are not that useful so we decided to use https://docs.pymc.io/projects/examples/en/latest/object_index/index.html instead for this. The issue to track progress on this is #289

:category: beginner
:author: Joshua Cook, Tyler James Burch
:author: Joshua Cook updated by Tyler James Burch, Chris Fonnesbeck
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is metadata used to generate the citation advise at the bottom. It should only have names and exclude those who have only re-executed the notebooks without making changes. The Authors section should always be present and list everyone and the changes they did.

We discussed that both in an issue, in the docs channel in slack and in a couple doc meetings. Like I often say, I do not have strong feelings about what was chosen and would have been perfectly happy (or even more happy) with another choice or with changing this now. What I do have strong feelings about is that all notebooks do the same because otherwise some people don't dare add their name at all whereas others do which only increases already present inequalities. Therefore changing that would mean changing the style guide and updating all the notebooks that are already in the Done column.

@OriolAbril OriolAbril mentioned this pull request May 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants