|
18 | 18 | "name": "stdout", |
19 | 19 | "output_type": "stream", |
20 | 20 | "text": [ |
21 | | - "Runing on PyMC3 v3.11.0\n" |
| 21 | + "Runing on PyMC v3.11.0\n" |
22 | 22 | ] |
23 | 23 | } |
24 | 24 | ], |
|
27 | 27 | "import matplotlib.pyplot as plt\n", |
28 | 28 | "import numpy as np\n", |
29 | 29 | "import pandas as pd\n", |
30 | | - "import pymc3 as pm\n", |
| 30 | + "import pymc as pm\n", |
31 | 31 | "\n", |
32 | | - "print(f\"Runing on PyMC3 v{pm.__version__}\")" |
| 32 | + "print(f\"Runing on PyMC v{pm.__version__}\")" |
33 | 33 | ] |
34 | 34 | }, |
35 | 35 | { |
|
71 | 71 | "\n", |
72 | 72 | "One alternative is to perform model selection but discuss all the different models together with the computed values of a given Information Criterion. It is important to put all these numbers and tests in the context of our problem so that we and our audience can have a better feeling of the possible limitations and shortcomings of our methods. If you are in the academic world you can use this approach to add elements to the discussion section of a paper, presentation, thesis, and so on.\n", |
73 | 73 | "\n", |
74 | | - "Yet another approach is to perform model averaging. The idea now is to generate a meta-model (and meta-predictions) using a weighted average of the models. There are several ways to do this and PyMC3 includes 3 of them that we are going to briefly discuss, you will find a more thorough explanation in the work by [Yuling Yao et. al.](https://arxiv.org/abs/1704.02030)\n", |
| 74 | + "Yet another approach is to perform model averaging. The idea now is to generate a meta-model (and meta-predictions) using a weighted average of the models. There are several ways to do this and PyMC includes 3 of them that we are going to briefly discuss, you will find a more thorough explanation in the work by [Yuling Yao et. al.](https://arxiv.org/abs/1704.02030)\n", |
75 | 75 | "\n", |
76 | 76 | "## Pseudo Bayesian model averaging\n", |
77 | 77 | "\n", |
|
89 | 89 | "\n", |
90 | 90 | "## Stacking\n", |
91 | 91 | "\n", |
92 | | - "The third approach implemented in PyMC3 is know as _stacking of predictive distributions_ and it has been recently [proposed](https://arxiv.org/abs/1704.02030). We want to combine several models in a metamodel in order to minimize the diverge between the meta-model and the _true_ generating model, when using a logarithmic scoring rule this is equivalently to:\n", |
| 92 | + "The third approach implemented in PyMC is know as _stacking of predictive distributions_ and it has been recently [proposed](https://arxiv.org/abs/1704.02030). We want to combine several models in a metamodel in order to minimize the diverge between the meta-model and the _true_ generating model, when using a logarithmic scoring rule this is equivalently to:\n", |
93 | 93 | "\n", |
94 | 94 | "$$\\max_{w} \\frac{1}{n} \\sum_{i=1}^{n}log\\sum_{k=1}^{K} w_k p(y_i|y_{-i}, M_k)$$\n", |
95 | 95 | "\n", |
|
99 | 99 | "\n", |
100 | 100 | "## Weighted posterior predictive samples\n", |
101 | 101 | "\n", |
102 | | - "Once we have computed the weights, using any of the above 3 methods, we can use them to get a weighted posterior predictive samples. PyMC3 offers functions to perform these steps in a simple way, so let see them in action using an example.\n", |
| 102 | + "Once we have computed the weights, using any of the above 3 methods, we can use them to get a weighted posterior predictive samples. PyMC offers functions to perform these steps in a simple way, so let see them in action using an example.\n", |
103 | 103 | "\n", |
104 | | - "The following example is taken from the superb book [Statistical Rethinking](http://xcelab.net/rm/statistical-rethinking/) by Richard McElreath. You will find more PyMC3 examples from this book in this [repository](https://github.com/aloctavodia/Statistical-Rethinking-with-Python-and-PyMC3). We are going to explore a simplified version of it. Check the book for the whole example and a more thorough discussion of both, the biological motivation for this problem and a theoretical/practical discussion of using Information Criteria to compare, select and average models.\n", |
| 104 | + "The following example is taken from the superb book [Statistical Rethinking](http://xcelab.net/rm/statistical-rethinking/) by Richard McElreath. You will find more PyMC examples from this book in this [repository](https://github.com/aloctavodia/Statistical-Rethinking-with-Python-and-PyMC). We are going to explore a simplified version of it. Check the book for the whole example and a more thorough discussion of both, the biological motivation for this problem and a theoretical/practical discussion of using Information Criteria to compare, select and average models.\n", |
105 | 105 | "\n", |
106 | 106 | "Briefly, our problem is as follows: We want to explore the composition of milk across several primate species, it is hypothesized that females from species of primates with larger brains produce more _nutritious_ milk (loosely speaking this is done _in order to_ support the development of such big brains). This is an important question for evolutionary biologists and try to give and answer we will use 3 variables, two predictor variables: the proportion of neocortex compare to the total mass of the brain and the logarithm of the body mass of the mothers. And for predicted variable, the kilocalories per gram of milk. With these variables we are going to build 3 different linear models:\n", |
107 | 107 | " \n", |
|
299 | 299 | " mu = alpha + beta * d[\"neocortex\"]\n", |
300 | 300 | "\n", |
301 | 301 | " kcal = pm.Normal(\"kcal\", mu=mu, sigma=sigma, observed=d[\"kcal.per.g\"])\n", |
302 | | - " trace_0 = pm.sample(2000, return_inferencedata=True)" |
| 302 | + " trace_0 = pm.sample(2000)" |
303 | 303 | ] |
304 | 304 | }, |
305 | 305 | { |
|
389 | 389 | "\n", |
390 | 390 | " kcal = pm.Normal(\"kcal\", mu=mu, sigma=sigma, observed=d[\"kcal.per.g\"])\n", |
391 | 391 | "\n", |
392 | | - " trace_1 = pm.sample(2000, return_inferencedata=True)" |
| 392 | + " trace_1 = pm.sample(2000)" |
393 | 393 | ] |
394 | 394 | }, |
395 | 395 | { |
|
479 | 479 | "\n", |
480 | 480 | " kcal = pm.Normal(\"kcal\", mu=mu, sigma=sigma, observed=d[\"kcal.per.g\"])\n", |
481 | 481 | "\n", |
482 | | - " trace_2 = pm.sample(2000, return_inferencedata=True)" |
| 482 | + " trace_2 = pm.sample(2000)" |
483 | 483 | ] |
484 | 484 | }, |
485 | 485 | { |
|
586 | 586 | "tags": [] |
587 | 587 | }, |
588 | 588 | "source": [ |
589 | | - "Now that we have sampled the posterior for the 3 models, we are going to use WAIC (Widely applicable information criterion) to compare the 3 models. We can do this using the `compare` function included with PyMC3." |
| 589 | + "Now that we have sampled the posterior for the 3 models, we are going to use WAIC (Widely applicable information criterion) to compare the 3 models. We can do this using the `compare` function included with PyMC." |
590 | 590 | ] |
591 | 591 | }, |
592 | 592 | { |
|
607 | 607 | "name": "stderr", |
608 | 608 | "output_type": "stream", |
609 | 609 | "text": [ |
610 | | - "/Users/CloudChaoszero/opt/anaconda3/envs/pymc3-dev-py38/lib/python3.8/site-packages/arviz/stats/stats.py:146: UserWarning: The default method used to estimate the weights for each model,has changed from BB-pseudo-BMA to stacking\n", |
| 610 | + "/Users/CloudChaoszero/opt/anaconda3/envs/pymc-dev-py38/lib/python3.8/site-packages/arviz/stats/stats.py:146: UserWarning: The default method used to estimate the weights for each model,has changed from BB-pseudo-BMA to stacking\n", |
611 | 611 | " warnings.warn(\n" |
612 | 612 | ] |
613 | 613 | }, |
|
724 | 724 | "\n", |
725 | 725 | "We can also see that we get a column with the relative `weight` for each model (according to the first equation at the beginning of this notebook). This weights can be _vaguely_ interpreted as the probability that each model will make the correct predictions on future data. Of course this interpretation is conditional on the models used to compute the weights, if we add or remove models the weights will change. And also is dependent on the assumptions behind WAIC (or any other Information Criterion used). So try to do not overinterpret these `weights`. \n", |
726 | 726 | "\n", |
727 | | - "Now we are going to use copmuted `weights` to generate predictions based not on a single model but on the weighted set of models. This is one way to perform model averaging. Using PyMC3 we can call the `sample_posterior_predictive_w` function as follows:" |
| 727 | + "Now we are going to use copmuted `weights` to generate predictions based not on a single model but on the weighted set of models. This is one way to perform model averaging. Using PyMC we can call the `sample_posterior_predictive_w` function as follows:" |
728 | 728 | ] |
729 | 729 | }, |
730 | 730 | { |
|
921 | 921 | "Python version : 3.8.6\n", |
922 | 922 | "IPython version : 7.20.0\n", |
923 | 923 | "\n", |
924 | | - "pymc3 : 3.11.0\n", |
| 924 | + "pymc : 3.11.0\n", |
925 | 925 | "numpy : 1.20.0\n", |
926 | 926 | "matplotlib: None\n", |
927 | 927 | "pandas : 1.2.1\n", |
|
940 | 940 | ], |
941 | 941 | "metadata": { |
942 | 942 | "kernelspec": { |
943 | | - "display_name": "Python PyMC3 (Dev)", |
| 943 | + "display_name": "Python PyMC (Dev)", |
944 | 944 | "language": "python", |
945 | | - "name": "pymc3-dev-py38" |
| 945 | + "name": "pymc-dev-py38" |
946 | 946 | }, |
947 | 947 | "language_info": { |
948 | 948 | "codemirror_mode": { |
|
0 commit comments