add myst notebook representation of examples (#283)

* add myst notebook representation of examples

* fix pre commit hook to work on notebooks onlyd

* fix incorrect mention to wiki

Author: OriolAbril

.jupytext.toml

@@ -0,0 +1,3 @@
+[formats]
+"examples/" = "ipynb"
+"myst_nbs/" = ".myst.md:myst"

.pre-commit-config.yaml

@@ -1,6 +1,12 @@
 repos:
+- repo: https://github.com/mwouts/jupytext
+  rev: v1.13.7
+  hooks:
+  - id: jupytext
+    files: ^examples/.+\.ipynb$
+    args: ["--sync"]
 - repo: https://github.com/psf/black
-  rev: 21.8b0
+  rev: 22.1.0
   hooks:
     - id: black-jupyter
 - repo: https://github.com/nbQA-dev/nbQA
@@ -48,7 +54,7 @@ repos:
       entry: '%load_ext watermark.*%watermark -n -u -v -iv -w'
       language: pygrep
       minimum_pre_commit_version: 2.8.0
-      name: Check notebooks have watermark (see Jupyter style guide from PyMC3 Wiki)
+      name: Check notebooks have watermark (see Jupyter style guide from PyMC docs)
       types: [jupyter]
     - id: add-tags
       entry: python scripts/add_tags.py
@@ -59,3 +65,9 @@ repos:
          - nbqa==1.1.1
          - beautifulsoup4==4.9.3
          - myst_parser==0.13.7
+- repo: https://github.com/mwouts/jupytext
+  rev: v1.13.7
+  hooks:
+  - id: jupytext
+    files: ^examples/.+\.ipynb$
+    args: ["--sync"]

examples/case_studies/BEST.ipynb

@@ -232,8 +232,8 @@
    "outputs": [],
    "source": [
     "with model:\n",
-    "    lambda_1 = group1_std ** -2\n",
-    "    lambda_2 = group2_std ** -2\n",
+    "    lambda_1 = group1_std**-2\n",
+    "    lambda_2 = group2_std**-2\n",
     "    group1 = pm.StudentT(\"drug\", nu=nu, mu=group1_mean, lam=lambda_1, observed=iq_drug)\n",
     "    group2 = pm.StudentT(\"placebo\", nu=nu, mu=group2_mean, lam=lambda_2, observed=iq_placebo)"
    ]
@@ -255,7 +255,7 @@
     "    diff_of_means = pm.Deterministic(\"difference of means\", group1_mean - group2_mean)\n",
     "    diff_of_stds = pm.Deterministic(\"difference of stds\", group1_std - group2_std)\n",
     "    effect_size = pm.Deterministic(\n",
-    "        \"effect size\", diff_of_means / np.sqrt((group1_std ** 2 + group2_std ** 2) / 2)\n",
+    "        \"effect size\", diff_of_means / np.sqrt((group1_std**2 + group2_std**2) / 2)\n",
     "    )"
    ]
   },

examples/case_studies/bayesian_ab_testing.ipynb

@@ -813,7 +813,7 @@
    "id": "c871fb6e",
    "metadata": {},
    "source": [
-    "### Generalising to multi-variant tests "
+    "### Generalising to multi-variant tests"
    ]
   },
   {

examples/case_studies/binning.ipynb

@@ -3267,7 +3267,7 @@
    "metadata": {},
    "source": [
     "## Authors\n",
-    "* Authored by [Eric Ma](https://github.com/ericmjl) and [Benjamin T. Vincent](https://github.com/drbenvincent) in September, 2021 ([pymc-examples#229](https://github.com/pymc-devs/pymc-examples/pull/229))\n"
+    "* Authored by [Eric Ma](https://github.com/ericmjl) and [Benjamin T. Vincent](https://github.com/drbenvincent) in September, 2021 ([pymc-examples#229](https://github.com/pymc-devs/pymc-examples/pull/229))"
    ]
   },
   {

examples/case_studies/blackbox_external_likelihood_numpy.ipynb

@@ -118,7 +118,7 @@
     "\n",
     "def my_loglike(theta, x, data, sigma):\n",
     "    model = my_model(theta, x)\n",
-    "    return -(0.5 / sigma ** 2) * np.sum((data - model) ** 2)"
+    "    return -(0.5 / sigma**2) * np.sum((data - model) ** 2)"
    ]
   },
   {

examples/case_studies/conditional-autoregressive-model.ipynb

@@ -1503,7 +1503,7 @@
    "source": [
     "`theano.scan` is much faster than using a python for loop, but it is still quite slow. One approach for improving it is to use linear algebra. That is, we should try to find a way to use matrix multiplication instead of looping (if you have experience in using MATLAB, it is the same philosophy). In our case, we can totally do that.  \n",
     "\n",
-    "For a similar problem, you can also have a look of [my port of Lee and Wagenmakers' book](https://github.com/junpenglao/Bayesian-Cognitive-Modeling-in-Pymc3). For example, in Chapter 19, the Stan code use [a for loop to generate the likelihood function](https://github.com/stan-dev/example-models/blob/master/Bayesian_Cognitive_Modeling/CaseStudies/NumberConcepts/NumberConcept_1_Stan.R#L28-L59), and I [generate the matrix outside and use matrix multiplication etc](http://nbviewer.jupyter.org/github/junpenglao/Bayesian-Cognitive-Modeling-in-Pymc3/blob/master/CaseStudies/NumberConceptDevelopment.ipynb#19.1-Knower-level-model-for-Give-N) to archive the same purpose.  "
+    "For a similar problem, you can also have a look of [my port of Lee and Wagenmakers' book](https://github.com/junpenglao/Bayesian-Cognitive-Modeling-in-Pymc3). For example, in Chapter 19, the Stan code use [a for loop to generate the likelihood function](https://github.com/stan-dev/example-models/blob/master/Bayesian_Cognitive_Modeling/CaseStudies/NumberConcepts/NumberConcept_1_Stan.R#L28-L59), and I [generate the matrix outside and use matrix multiplication etc](http://nbviewer.jupyter.org/github/junpenglao/Bayesian-Cognitive-Modeling-in-Pymc3/blob/master/CaseStudies/NumberConceptDevelopment.ipynb#19.1-Knower-level-model-for-Give-N) to archive the same purpose."
    ]
   },
   {
@@ -3286,7 +3286,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "As you can see above, the sparse representation returns the same estimates, while being much faster than any other implementation. "
+    "As you can see above, the sparse representation returns the same estimates, while being much faster than any other implementation."
    ]
   },
   {

examples/case_studies/factor_analysis.ipynb

@@ -608,7 +608,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "* This notebook was written by [chartl](https://github.com/chartl) on May 6, 2019 and updated by [Christopher Krapu](https://github.com/ckrapu) on April 4, 2021. "
+    "* This notebook was written by [chartl](https://github.com/chartl) on May 6, 2019 and updated by [Christopher Krapu](https://github.com/ckrapu) on April 4, 2021."
    ]
   },
   {

examples/case_studies/hierarchical_partial_pooling.ipynb

@@ -43,7 +43,7 @@
     "\n",
     "The idea of hierarchical partial pooling is to model the global performance, and use that estimate to parameterize a population of players that accounts for differences among the players' performances. This tradeoff between global and individual performance will be automatically tuned by the model. Also, uncertainty due to different number of at bats for each player (*i.e.* informatino) will be automatically accounted for, by shrinking those estimates closer to the global mean.\n",
     "\n",
-    "For far more in-depth discussion please refer to Stan [tutorial](http://mc-stan.org/documentation/case-studies/pool-binary-trials.html) {cite:p}`carpenter2016hierarchical` on the subject. The model and parameter values were taken from that example.\n"
+    "For far more in-depth discussion please refer to Stan [tutorial](http://mc-stan.org/documentation/case-studies/pool-binary-trials.html) {cite:p}`carpenter2016hierarchical` on the subject. The model and parameter values were taken from that example."
    ]
   },
   {
@@ -588,7 +588,7 @@
     "\n",
     ":::{bibliography}\n",
     ":filter: docname in docnames\n",
-    ":::\n"
+    ":::"
    ]
   }
  ],

examples/case_studies/item_response_nba.ipynb

@@ -812,7 +812,7 @@
     "- y: the difference between the raw mean probability (from the data) and the posterior mean probability for each disadvantaged and committing player\n",
     "- x: as a function of the number of observations per disadvantaged and committing player.\n",
     "\n",
-    "These plots show, as expected, that the hierarchical structure of our model tends to estimate posteriors towards the global mean for players with a low amount of observations. "
+    "These plots show, as expected, that the hierarchical structure of our model tends to estimate posteriors towards the global mean for players with a low amount of observations."
    ]
   },
   {
@@ -1015,7 +1015,7 @@
    "source": [
     "### Discovering extra hierarchical structure\n",
     "\n",
-    "A natural question to ask is whether players skilled as disadvantaged players (i.e. players with high `θ`) are also likely to be skilled as committing players  (i.e. with high `b`), and the other way around. So, the next two plots show the `θ` (resp. `b`) score for the top players with respect to `b` ( resp.`θ`). "
+    "A natural question to ask is whether players skilled as disadvantaged players (i.e. players with high `θ`) are also likely to be skilled as committing players  (i.e. with high `b`), and the other way around. So, the next two plots show the `θ` (resp. `b`) score for the top players with respect to `b` ( resp.`θ`)."
    ]
   },
   {
@@ -1098,7 +1098,7 @@
    "metadata": {},
    "source": [
     "These plots suggest that scoring high in `θ` does not correlate with high or low scores in `b`. Moreover, with a little knowledge of NBA basketball, one can visually note that a higher score in `b` is expected from players playing center or forward rather than guards or point guards. \n",
-    "Given the last observation, we decide to plot a histogram for the occurence of different positions for top disadvantaged (`θ`) and committing (`b`) players. Interestingly, we see below that the largest share of best disadvantaged players are guards, meanwhile, the largest share of best committing players are centers (and at the same time a very small share of guards). "
+    "Given the last observation, we decide to plot a histogram for the occurence of different positions for top disadvantaged (`θ`) and committing (`b`) players. Interestingly, we see below that the largest share of best disadvantaged players are guards, meanwhile, the largest share of best committing players are centers (and at the same time a very small share of guards)."
    ]
   },
   {

examples/case_studies/log-gaussian-cox-process.ipynb

@@ -29,7 +29,7 @@
     "* What would randomly sampled patterns with the same statistical properties look like?\n",
     "* Is there a statistical correlation between the *frequency* and *magnitude* of point events?\n",
     "\n",
-    "In this notebook, we'll use a grid-based approximation to the full LGCP with PyMC3 to fit a model and analyze its posterior summaries. We will also explore the usage of a marked Poisson process, an extension of this model to account for the distribution of *marks* associated with each data point.\n"
+    "In this notebook, we'll use a grid-based approximation to the full LGCP with PyMC3 to fit a model and analyze its posterior summaries. We will also explore the usage of a marked Poisson process, an extension of this model to account for the distribution of *marks* associated with each data point."
    ]
   },
   {
@@ -43,7 +43,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Our observational data concerns 231 sea anemones whose sizes and locations on the French coast were recorded. This data was taken from the [`spatstat` spatial modeling package in R](https://github.com/spatstat/spatstat) which is designed to address models like the LGCP and its subsequent refinements. The original source of this data is the textbook *Spatial data analysis by example* by Upton and Fingleton (1985) and a longer description of the data can be found there. \n"
+    "Our observational data concerns 231 sea anemones whose sizes and locations on the French coast were recorded. This data was taken from the [`spatstat` spatial modeling package in R](https://github.com/spatstat/spatstat) which is designed to address models like the LGCP and its subsequent refinements. The original source of this data is the textbook *Spatial data analysis by example* by Upton and Fingleton (1985) and a longer description of the data can be found there."
    ]
   },
   {
@@ -234,7 +234,7 @@
     "\n",
     "# Rescaling the unit of area so that our parameter estimates\n",
     "# are easier to read\n",
-    "area_per_cell = resolution ** 2 / 100\n",
+    "area_per_cell = resolution**2 / 100\n",
     "\n",
     "cells_x = int(280 / resolution)\n",
     "cells_y = int(180 / resolution)\n",
@@ -311,7 +311,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Our first step is to place prior distributions over the high-level parameters for the Gaussian process. This includes the length scale $\\rho$ for the covariance function and a constant mean $\\mu$ for the GP. "
+    "Our first step is to place prior distributions over the high-level parameters for the Gaussian process. This includes the length scale $\\rho$ for the covariance function and a constant mean $\\mu$ for the GP."
    ]
   },
   {
@@ -638,7 +638,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "While there is some heterogeneity in the patterns these surfaces show, we obtain a posterior mean surface with a very clearly defined spatial surface with higher intensity in the upper right and lower intensity in the lower left.\n"
+    "While there is some heterogeneity in the patterns these surfaces show, we obtain a posterior mean surface with a very clearly defined spatial surface with higher intensity in the upper right and lower intensity in the lower left."
    ]
   },
   {
@@ -787,7 +787,7 @@
     "Equivalently, $$z_i \\sim N(\\alpha +  \\beta \\lambda_i, \\sigma_\\epsilon^2)$$\n",
     "where $\\sigma_\\epsilon^2 = Var(\\epsilon_i)$.\n",
     "\n",
-    "This equation states that the distribution of the marks is a linear function of the intensity field since we assume a normal likelihood for $\\epsilon$. It's essentially a simple linear regression of the marks on the intensity field; $\\alpha$ is the intercept and $\\beta$ is the slope. Then, standard priors are used for $\\epsilon, \\alpha, \\beta$. The point of this model is to figure out whether or not the growth of the anemones is correlated with their occurrence. If we find that $\\beta$ is negative, then that might hint that locations with more numerous anemones happen to also have smaller anemones and that competition for food may keep close neighbors small. "
+    "This equation states that the distribution of the marks is a linear function of the intensity field since we assume a normal likelihood for $\\epsilon$. It's essentially a simple linear regression of the marks on the intensity field; $\\alpha$ is the intercept and $\\beta$ is the slope. Then, standard priors are used for $\\epsilon, \\alpha, \\beta$. The point of this model is to figure out whether or not the growth of the anemones is correlated with their occurrence. If we find that $\\beta$ is negative, then that might hint that locations with more numerous anemones happen to also have smaller anemones and that competition for food may keep close neighbors small."
    ]
   },
   {
@@ -980,7 +980,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "* This notebook was written by [Christopher Krapu](https://github.com/ckrapu) on September 6, 2020 and updated on April 1, 2021. "
+    "* This notebook was written by [Christopher Krapu](https://github.com/ckrapu) on September 6, 2020 and updated on April 1, 2021."
    ]
   },
   {

examples/case_studies/mediation_analysis.ipynb

@@ -67,7 +67,7 @@
     "Using definitions from Hayes (2018), we can define a few effects of interest:\n",
     "- **Direct effect:** is given by $c'$. Two cases that differ by one unit on $x$ but are equal on $m$ are estimated to differ by $c'$ units on $y$.\n",
     "- **Indirect effect:** is given by $a \\cdot b$. Two cases which differ by one unit of $x$ are estimated to differ by $a \\cdot b$ units on $y$ as a result of the effect of $x \\rightarrow m$ and $m \\rightarrow y$.\n",
-    "- **Total effect:** is $c = c' + a \\cdot b$ which is simply the sum of the direct and indirect effects. This could be understood as: two cases that differ by one unit on $x$ are estimated to differ by $a \\cdot b$ units on $y$ due to both the direct pathway $x \\rightarrow y$ and the indirect pathway $c \\rightarrow m \\rightarrow m$. The total effect could also be estimated by evaluating the alternative model $y_i \\sim \\mathrm{Normal}(i_{Y*} + c \\cdot x_i, \\sigma_{Y*})$. "
+    "- **Total effect:** is $c = c' + a \\cdot b$ which is simply the sum of the direct and indirect effects. This could be understood as: two cases that differ by one unit on $x$ are estimated to differ by $a \\cdot b$ units on $y$ due to both the direct pathway $x \\rightarrow y$ and the indirect pathway $c \\rightarrow m \\rightarrow m$. The total effect could also be estimated by evaluating the alternative model $y_i \\sim \\mathrm{Normal}(i_{Y*} + c \\cdot x_i, \\sigma_{Y*})$."
    ]
   },
   {