Merge branch 'BART' of https://github.com/aloctavodia/pymc3 into BART

Author: aloctavodia

.pre-commit-config.yaml

@@ -1,28 +1,35 @@
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v3.2.0
+  rev: v3.3.0
   hooks:
     -   id: end-of-file-fixer
     -   id: check-toml
 - repo: https://github.com/nbQA-dev/nbQA
-  rev: 0.2.3
+  rev: 0.3.5
   hooks:
-    - id: nbqa
-      args: ['isort']
-      name: nbqa-isort
-      alias: nbqa-isort
-      additional_dependencies: ['isort']
-    - id: nbqa
-      args: ['pyupgrade']
-      name: nbqa-pyupgrade
-      alias: nbqa-pyupgrade
-      additional_dependencies: ['pyupgrade']
+    - id: nbqa-black
+    - id: nbqa-isort
+    - id: nbqa-pyupgrade
 - repo: https://github.com/asottile/pyupgrade
-  rev: v2.7.2
+  rev: v2.7.3
   hooks:
     - id: pyupgrade
-      args: ['--py36-plus']
+      args: [--py36-plus]
 - repo: https://github.com/psf/black
   rev: 20.8b1
   hooks:
     - id: black
+- repo: local
+  hooks:
+    - id: watermark
+      name: Check notebooks have watermark (see Jupyter style guide from PyMC3 Wiki)
+      types: [jupyter]
+      entry: '%load_ext watermark.*%watermark -n -u -v -iv -w'
+      language: pygrep
+      args: [--negate, --multiline]
+      minimum_pre_commit_version: 2.8.0
+    - id: check-toc
+      name: Check all notebooks appear in table of contents
+      types: [jupyter]
+      entry: python scripts/check_toc_is_complete.py
+      language: python

README.rst

@@ -13,6 +13,19 @@ large suite of problems.
 Check out the `getting started guide <http://docs.pymc.io/notebooks/getting_started>`__,  or
 `interact with live examples <https://mybinder.org/v2/gh/pymc-devs/pymc3/master?filepath=%2Fdocs%2Fsource%2Fnotebooks>`__
 using Binder!
+For questions on PyMC3, head on over to our `PyMC Discourse <https://discourse.pymc.io/>`__ forum.
+
+The future of PyMC3 & Theano
+============================
+
+There have been many questions and uncertainty around the future of PyMC3 since Theano
+stopped getting developed by the original authors, and we started experiments with PyMC4.
+
+We are happy to announce that PyMC3 on Theano (which we are `developing further <https://github.com/pymc-devs/Theano-PyMC>`__) 
+with a new JAX backend is the future. PyMC4 will not be developed further. 
+
+See the `full announcement <https://pymc-devs.medium.com/the-future-of-pymc3-or-theano-is-dead-long-live-theano-d8005f8a0e9b>`__
+for more details. 
 
 Features
 ========
@@ -167,7 +180,7 @@ Contributors
 ============
 
 See the `GitHub contributor
-page <https://github.com/pymc-devs/pymc3/graphs/contributors>`__
+page <https://github.com/pymc-devs/pymc3/graphs/contributors>`__. Also read our `Code of Conduct <https://github.com/pymc-devs/pymc3/blob/master/CODE_OF_CONDUCT.md>`__ guidelines for a better contributing experience.
 
 Support
 =======

RELEASE-NOTES.md

@@ -8,15 +8,19 @@
 - Fixed numerical instability in ExGaussian's logp by preventing `logpow` from returning `-inf` (see [#4050](https://github.com/pymc-devs/pymc3/pull/4050)).
 - Use dill to serialize user defined logp functions in `DensityDist`. The previous serialization code fails if it is used in notebooks on Windows and Mac. `dill` is now a required dependency. (see [#3844](https://github.com/pymc-devs/pymc3/issues/3844)).
 - Numerically improved stickbreaking transformation - e.g. for the `Dirichlet` distribution. [#4129](https://github.com/pymc-devs/pymc3/pull/4129)
+- Enabled the `Multinomial` distribution to handle batch sizes that have more than 2 dimensions. [#4169](https://github.com/pymc-devs/pymc3/pull/4169)
 
 ### Documentation
 
 ### New features
 - `sample_posterior_predictive_w` can now feed on `xarray.Dataset` - e.g. from `InferenceData.posterior`. (see [#4042](https://github.com/pymc-devs/pymc3/pull/4042))
+- Added `pymc3.gp.cov.Circular` kernel for Gaussian Processes on circular domains, e.g. the unit circle (see [#4082](https://github.com/pymc-devs/pymc3/pull/4082)).
 - Add MLDA, a new stepper for multilevel sampling. MLDA can be used when a hierarchy of approximate posteriors of varying accuracy is available, offering improved sampling efficiency especially in high-dimensional problems and/or where gradients are not available (see [#3926](https://github.com/pymc-devs/pymc3/pull/3926))
 - Change SMC metropolis kernel to independent metropolis kernel [#4115](https://github.com/pymc-devs/pymc3/pull/4115))
 - Add alternative parametrization to NegativeBinomial distribution in terms of n and p (see [#4126](https://github.com/pymc-devs/pymc3/issues/4126))
 - Add Bayesian Additive Regression Trees (BARTs) [#4183](https://github.com/pymc-devs/pymc3/pull/4183))
+- Added a new `MixtureSameFamily` distribution to handle mixtures of arbitrary dimensions in vectorized form (see [#4185](https://github.com/pymc-devs/pymc3/issues/4185)).
+
 
 
 ## PyMC3 3.9.3 (11 August 2020)

docs/source/api/distributions/mixture.rst

@@ -6,6 +6,7 @@ Mixture
 .. autosummary::
    Mixture
    NormalMixture
+   MixtureSameFamily
 
 .. automodule:: pymc3.distributions.mixture
    :members:

docs/source/learn.rst

@@ -99,7 +99,7 @@
 
         <div class="card">
             <div class="image">
-                <img src="https://camo.githubusercontent.com/dbb5f926b3bfe7e2196044462ce2411ee3a001ee/68747470733a2f2f39623865303033322d612d36326362336131612d732d73697465732e676f6f676c6567726f7570732e636f6d2f736974652f646f696e67626179657369616e64617461616e616c797369732f776861742d732d6e65772d696e2d326e642d65642f436f7665724442444132452d46726f6e744f6e6c792d363030776964652e706e67">
+                <img src="https://lh5.googleusercontent.com/R1T8ZXbMi4vSO0JlnLQMkEQNvd2ncBb23OmHOsmw-t_oEOF6jJlfWuJoOK0MMmECSDymhUdfTS2yoMgkR2TY-xIiBTHCpeuYjzXqD3xhZ-MuIhs2ARcJ=w1280">
             </div>
             <div class="content">
                 <div class="header">Doing Bayesian Data Analysis</div>
@@ -112,7 +112,7 @@
                 <tbody>
                     <tr>
                         <td>
-                            <a href="http://doingbayesiandataanalysis.blogspot.com/">
+                            <a href="https://sites.google.com/site/doingbayesiandataanalysis/home">
                                 <i class="linkify icon"></i> Book website
                             </a>
                         </td>

docs/source/notebooks/GLM-hierarchical.ipynb

@@ -36,7 +36,7 @@
     "Gelman et al.'s (2007) radon dataset is a classic for hierarchical modeling. In this dataset the amount of the radioactive gas radon has been measured among different households in all counties of several states. Radon gas is known to be the highest cause of lung cancer in non-smokers. It is believed to be more strongly present in households containing a basement and to differ in amount present among types of soil.\n",
     "Here we'll investigate this differences and try to make predictions of radonlevels in different counties based on the county itself and the presence of a basement. In this example we'll look at Minnesota, a state that contains 85 counties in which different measurements are taken, ranging from 2 to 116 measurements per county. \n",
     "\n",
-    "![radon](http://www.fix-your-radon.com/images/how_radon_enters.jpg)"
+    "![radon](https://upload.wikimedia.org/wikipedia/commons/b/b9/CNX_Chem_21_06_RadonExpos.png)"
    ]
   },
   {

docs/source/notebooks/GLM-negative-binomial-regression.ipynb

@@ -383,9 +383,7 @@
     }
    ],
    "source": [
-    "g = sns.catplot(\n",
-    "    x=\"nsneeze\", row=\"nomeds\", col=\"alcohol\", data=df, kind=\"count\", aspect=1.5\n",
-    ")\n",
+    "g = sns.catplot(x=\"nsneeze\", row=\"nomeds\", col=\"alcohol\", data=df, kind=\"count\", aspect=1.5)\n",
     "\n",
     "# Make x-axis ticklabels less crowded\n",
     "ax = g.axes[1, 0]\n",
@@ -464,9 +462,7 @@
     "fml = \"nsneeze ~ alcohol + nomeds + alcohol:nomeds\"\n",
     "\n",
     "with pm.Model() as model:\n",
-    "    pm.glm.GLM.from_formula(\n",
-    "        formula=fml, data=df, family=pm.glm.families.NegativeBinomial()\n",
-    "    )\n",
+    "    pm.glm.GLM.from_formula(formula=fml, data=df, family=pm.glm.families.NegativeBinomial())\n",
     "    trace = pm.sample(1000, tune=1000, cores=2, return_inferencedata=True)"
    ]
   },

docs/source/notebooks/GP-Circular.ipynb

@@ -68,7 +68,7 @@
     "%config InlineBackend.figure_format = 'retina'\n",
     "RANDOM_SEED = 8927\n",
     "np.random.seed(RANDOM_SEED)\n",
-    "az.style.use('arviz-darkgrid')"
+    "az.style.use(\"arviz-darkgrid\")"
    ]
   },
   {
@@ -1310,9 +1310,7 @@
     "    ηq = pm.HalfNormal(\"ηq\", sigma=5)\n",
     "\n",
     "    cov1 = η ** 2 * pm.gp.cov.ExpQuad(1, ℓ)\n",
-    "    cov2 = η ** 2 * pm.gp.cov.ExpQuad(1, ℓ) + ηq ** 2 * pm.gp.cov.Polynomial(\n",
-    "        1, x0, 2, c\n",
-    "    )\n",
+    "    cov2 = η ** 2 * pm.gp.cov.ExpQuad(1, ℓ) + ηq ** 2 * pm.gp.cov.Polynomial(1, x0, 2, c)\n",
     "\n",
     "    # construct changepoint cov\n",
     "    sc_cov1 = pm.gp.cov.ScaledCov(1, cov1, logistic, (-a, x0))\n",
@@ -1799,10 +1797,10 @@
    "outputs": [],
    "source": [
     "class CustomWhiteNoise(pm.gp.cov.Covariance):\n",
-    "    \"\"\" Custom White Noise covariance\n",
+    "    \"\"\"Custom White Noise covariance\n",
     "    - sigma1 is applied to the first n1 points in the data\n",
     "    - sigma2 is applied to the next n2 points in the data\n",
-    "    \n",
+    "\n",
     "    The total number of data points n = n1 + n2\n",
     "    \"\"\"\n",
     "\n",
@@ -1922,11 +1920,7 @@
     "    ℓp_decay = pm.Gamma(\"ℓp_decay\", alpha=40, beta=0.1)\n",
     "    ℓp_smooth = pm.Normal(\"ℓp_smooth \", mu=1.0, sigma=0.05)\n",
     "    period = 1 * 0.01  # we know the period is annual\n",
-    "    cov_p = (\n",
-    "        ηp ** 2\n",
-    "        * pm.gp.cov.Periodic(1, period, ℓp_smooth)\n",
-    "        * pm.gp.cov.ExpQuad(1, ℓp_decay)\n",
-    "    )\n",
+    "    cov_p = ηp ** 2 * pm.gp.cov.Periodic(1, period, ℓp_smooth) * pm.gp.cov.ExpQuad(1, ℓp_decay)\n",
     "    gp_p = pm.gp.Marginal(cov_func=cov_p)\n",
     "\n",
     "    gp = gp_c + gp_m + gp_s + gp_p\n",
@@ -1943,9 +1937,7 @@
     "    σ2 = pm.Gamma(\"σ2\", alpha=3, beta=50)\n",
     "    η_noise = pm.HalfNormal(\"η_noise\", sigma=1)\n",
     "    ℓ_noise = pm.Gamma(\"ℓ_noise\", alpha=2, beta=200)\n",
-    "    cov_noise = η_noise ** 2 * pm.gp.cov.Matern32(1, ℓ_noise) + CustomWhiteNoise(\n",
-    "        σ1, σ2, 111, 545\n",
-    "    )\n",
+    "    cov_noise = η_noise ** 2 * pm.gp.cov.Matern32(1, ℓ_noise) + CustomWhiteNoise(σ1, σ2, 111, 545)\n",
     "\n",
     "    y_ = gp.marginal_likelihood(\"y\", X=t_combined[:, None], y=y_n, noise=cov_noise)"
    ]
@@ -2141,9 +2133,7 @@
    ],
    "source": [
     "plt.figure(figsize=(12, 5))\n",
-    "plt.plot(\n",
-    "    tnew * 100, y_sd * ppc[\"fnew\"][0:200:5, :].T + y_mu, color=\"lightblue\", alpha=0.8\n",
-    ")\n",
+    "plt.plot(tnew * 100, y_sd * ppc[\"fnew\"][0:200:5, :].T + y_mu, color=\"lightblue\", alpha=0.8)\n",
     "plt.plot(\n",
     "    [-1000, -1001],\n",
     "    [-1000, -1001],\n",
@@ -2198,9 +2188,7 @@
    ],
    "source": [
     "plt.figure(figsize=(12, 5))\n",
-    "plt.plot(\n",
-    "    tnew * 100, y_sd * ppc[\"fnew\"][0:200:5, :].T + y_mu, color=\"lightblue\", alpha=0.8\n",
-    ")\n",
+    "plt.plot(tnew * 100, y_sd * ppc[\"fnew\"][0:200:5, :].T + y_mu, color=\"lightblue\", alpha=0.8)\n",
     "plt.plot(\n",
     "    [-1000, -1001],\n",
     "    [-1000, -1001],\n",
@@ -2325,9 +2313,7 @@
    "source": [
     "plt.figure(figsize=(12, 5))\n",
     "\n",
-    "plt.plot(\n",
-    "    tnew * 100, y_sd * ppc[\"fnew2\"][0:200:10, :].T + y_mu, color=\"lightblue\", alpha=0.8\n",
-    ")\n",
+    "plt.plot(tnew * 100, y_sd * ppc[\"fnew2\"][0:200:10, :].T + y_mu, color=\"lightblue\", alpha=0.8)\n",
     "plt.plot(\n",
     "    [-1000, -1001],\n",
     "    [-1000, -1001],\n",
@@ -2452,9 +2438,7 @@
    "source": [
     "plt.figure(figsize=(12, 5))\n",
     "\n",
-    "plt.plot(\n",
-    "    tnew * 100, y_sd * ppc[\"fnew3\"][0:200:10, :].T + y_mu, color=\"lightblue\", alpha=0.8\n",
-    ")\n",
+    "plt.plot(tnew * 100, y_sd * ppc[\"fnew3\"][0:200:10, :].T + y_mu, color=\"lightblue\", alpha=0.8)\n",
     "plt.plot(\n",
     "    [-1000, -1001],\n",
     "    [-1000, -1001],\n",