Skip to content

Update NBs prior to 3.9.0 release #3955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 13, 2020

Conversation

AlexAndorra
Copy link
Contributor

As part of the 3.9.0 release @michaelosthege and I are updating and re-running all the NBs:

  • I added a watermark cell at the end of each NB, to improve reproducibility.
  • I updated some of them with cleaner import statements and ArviZ style but stopped at one point because there are soooooo many NBs -- it would be nice to standardize and rejuvenate our NBs in that regard though; I'll open a "beginner-friendly" issue about that.

Once Michael pushes all the re-run NBs here, this will be ready to merge.

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

Review Jupyter notebook visual diffs & provide feedback on notebooks.


Powered by ReviewNB

because we use it in notebooks
@michaelosthege
Copy link
Member

michaelosthege commented Jun 12, 2020

This is a list of all notebooks.

Literally every second one has FutureWarnings:

  • about Theano/Numpy indexing thing
  • from_pymc3 requiring to pass model object

Please do not commit changes to [x] ticked notebooks so we don't get git conflicts!

Good-Looking & committed

  • AR
  • BEST
  • Bayes_factor
  • DEMetropolisZ_tune_drop_fraction
  • Diagnosing_biased_Inference_with_Divergences
  • GLM-hierarchical-binominal-model
  • GLM-linear
  • GLM-robust-with-outlier-detection
  • GLM-robust
  • GP-MeansAndCovs
  • GP-smoothingx
  • MvGaussianRandomWalk_demo
  • ODE_API_shapes_and_benchmarking
  • SMC2_gaussians
  • api_quickstart
  • bayesian_neural_network_advi
  • empirical-approx-overview
  • factor_analysis
  • gaussian_mixture_model
  • howto_debugging
  • lasso_block_update
  • normalizing_flows_overview
  • posterior_predictive
  • sampler-stats
  • updating_priors
  • variational_api_quickstart
  • sampling_compound_step
  • bayes_param_survival_pymc3
  • getting_started
  • SMC-ABC_Lotka-Volterra_example
  • LKJ
  • gaussian_process
  • GLM-hierarchical-advi-minibatch
  • GP-Kron
  • GP-Latent
  • GP-Marginal
  • cox_model
  • dp_mix
  • data_container
  • gaussian-mixture-model-advi
  • DEMetropolisZ_EfficiencyComparison
  • ODE_API_introduction

Math Issues

  • GLM-model-selection Mass matrix contains zeros on the diagonal
  • Euler-Maruyama_and_SDEs Mass matrix contains zeros on the diagonal
  • GLM-hierarchical Mass matrix contains zeros on the diagonal
  • GLM-negative-binomial-regression Mass matrix contains zeros on the diagonal
  • GLM-poisson-regression Mass matrix contains zeros on the diagonal
  • GLM-rolling-regression Mass matrix contains zeros on the diagonal
  • GP-MaunaLoa2 Mass matrix contains zeros on the diagonal
  • GP-SparseApprox Mass matrix contains zeros on the diagonal
  • censored_data Mass matrix contains zeros on the diagonal
  • probabilistic_matrix_factorization Mass matrix contains zeros on the diagonal
  • putting_workflow Mass matrix contains zeros on the diagonal
  • stochastic_volatility Mass matrix contains zeros on the diagonal.
  • GP-TProcess runs but has way too many divergences

Exotic

  • blackbox_external_likelihood needs Cython
  • convolutional_vae_keras_advi needs Keras

Other Issues

  • GLM theano.gof.fg.MissingInputError
  • GP-MaunaLoa ValueError: Units 'M' and 'Y' are no longer supported, as they do not represent unambiguous timedelta values durations.
  • PyMC3_tips_and_heuristic KeyError: Rhat
  • dependent_density_regression AttributeError: 'DataFrame' object has no attribute 'range'
  • hierarchical_partial_pooling not enough values to unpack (expected 2, got 1)
  • lda-advi-aevb TypeError: init() got an unexpected keyword argument 'n_topics'
  • marginalized_gaussian_mixture_model AttributeError: 'Rectangle' object has no property 'normed'
  • GLM-logistic AttributeError: 'Rectangle' object has no property 'normed'
  • model_averaging FileNotFoundError: File ../data/milk.csv does not exist
  • model_comparison AttributeError: 'ELPDData' object has no attribute 'WAIC'
  • multilevel_modeling More chains (4000) than draws (2) and some plots may be wrong
  • profiling has a shape error
  • rugby_analytics ValueError: not enough values to unpack (expected 2, got 1)
  • sampling_callback has a shape error (looks like a threading problem)
  • survival_analysis cell 11 raises a NotImplementedError in numpy/pandas
  • weibull_aft AttributeError: module 'statsmodels' has no attribute 'datasets'
  • ODE_with_manual_gradients ValueError: array must not contain infs or NaNs

@Sayam753
Copy link
Member

Hi @michaelosthege , @AlexAndorra
I am willing to look upon survival_analysis, rugby_analytics and lda-advi-aevb notebooks for resolving the errors. I have one question - while running the notebooks, some warnings are also popping out. So, are we going to consider only the errors or take into consideration those warnings as well?

@michaelosthege
Copy link
Member

Hi @Sayam753 ,
Awesome! It's best if you clone/check out this branch and open a PR to merge from your branch into this one.
The hierarchical partial pooling notebook has the same problem as the rugby analytics - it's related to a set_xlim call IIRC.

Depending on the warning it may be easy, hard or impossible to fix. My priority here is to get as many notebook re-runs as possible without investing too much time. We can & should fix style and warnings, but that can happen after the 3.9.0 release.

@AlexAndorra
Copy link
Contributor Author

Thanks @Sayam753 ! Yeah, as @michaelosthege says here, the goal here is to re-run successfully as many NBs as possible, as quick and possible. So, if you manage to fix some warnings quickly, then do it.
Otherwise, fixing warnings, style and other tweaks are a tailored treatment that will be done in #3959 -- here we're doing an industrialized treatment 😉

Copy link
Member

@michaelosthege michaelosthege left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just Notebooks changed. Merging...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants