Skip to content

Return posterior predictive samples from all chains in ModelBuilder #140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 20, 2023

Conversation

mbjoseph
Copy link
Contributor

@mbjoseph mbjoseph commented Apr 13, 2023

This fixes a bug where only values from one chain were returned. It also refactors the prediction logic to reduce duplication. I've used arviz.extract() to get the posterior predictive samples as hinted by @twiecki in #139, removing the _extract_samples() method from the model builder class.

Behavior change proposal

This PR also includes a proposed behavior change that makes the output of predict_posterior() consistent in type and shape with the output of pymc.sample_posterior_predictive().

Namely, predict_posterior() would here return an xarray Dataset from the posterior predictive distribution, rather than a dictionary of numpy arrays. This is an admittedly opinionated change in behavior, though it does also streamline the code. From my perspective, retaining the metadata related to chains, draws, and other attributes is useful for three key reasons:

  1. downstream processing for posterior predictive checks, enabling easier alignment of chains and draws with other pushforward quantities, and
  2. tracking the provenance of predictions (e.g., creation time and package versions attributes) in a production environment, and
  3. consistency with the output from pymc.sample_posterior_predictive(), which could make behavior easier to predict for users, and allow greater interoperability with other functions that process and plot posterior predictive samples

That said, I'm very open to other perspectives on how to bundle up posterior predictive samples.

Max Joseph added 2 commits April 13, 2023 10:06
This fixes a bug where only values from one chain were returned.
It also refactors the prediction logic to reduce duplication, and
makes the output of predict_posterior() consistent in type and shape
with the output of pymc.sample_posterior_predictive().
@twiecki twiecki requested a review from michaelraczycki April 13, 2023 19:29
Copy link
Collaborator

@michaelraczycki michaelraczycki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, for the future reference please try not to just switch places of functions if it's not needed, as it makes the code changes less readable for PR

@mbjoseph
Copy link
Contributor Author

Thanks for the review @michaelraczycki. I've added combined as a named argument, tested behavior of combined=True and combined=False, and restored the original method order in 02faac7.

twiecki
twiecki previously approved these changes Apr 15, 2023
@ricardoV94 ricardoV94 added the bug Something isn't working label Apr 19, 2023
@ricardoV94 ricardoV94 changed the title Return posterior predictive samples from all chains Return posterior predictive samples from all chains in ModelBuilder Apr 19, 2023
@twiecki twiecki dismissed stale reviews from michaelraczycki and themself via b9eb478 April 20, 2023 06:52
Copy link
Collaborator

@michaelraczycki michaelraczycki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@twiecki twiecki merged commit b3be15f into pymc-devs:main Apr 20, 2023
@twiecki
Copy link
Member

twiecki commented Apr 20, 2023

Thanks @mbjoseph and for the review @michaelraczycki!

michaelraczycki pushed a commit to michaelraczycki/pymc-experimental that referenced this pull request Apr 20, 2023
…pymc-devs#140)

* Return posterior predictive samples from all chains

This fixes a bug where only values from one chain were returned.
It also refactors the prediction logic to reduce duplication, and
makes the output of predict_posterior() consistent in type and shape
with the output of pymc.sample_posterior_predictive().

* keep attributes even when computing posterior means

* Add/test combined arg, revert method order

* Fix import order.

---------

Co-authored-by: Max Joseph <[email protected]>
Co-authored-by: Thomas Wiecki <[email protected]>
twiecki added a commit that referenced this pull request Apr 20, 2023
* docstrings update in model_builder.py

* bringing back accidentally removed line from example

* Return posterior predictive samples from all chains in `ModelBuilder` (#140)

* Return posterior predictive samples from all chains

This fixes a bug where only values from one chain were returned.
It also refactors the prediction logic to reduce duplication, and
makes the output of predict_posterior() consistent in type and shape
with the output of pymc.sample_posterior_predictive().

* keep attributes even when computing posterior means

* Add/test combined arg, revert method order

* Fix import order.

---------

Co-authored-by: Max Joseph <[email protected]>
Co-authored-by: Thomas Wiecki <[email protected]>

* fixing merge conflicts

---------

Co-authored-by: Max Joseph <[email protected]>
Co-authored-by: Max Joseph <[email protected]>
Co-authored-by: Thomas Wiecki <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants