|
10 | 10 | "name": "stdout",
|
11 | 11 | "output_type": "stream",
|
12 | 12 | "text": [
|
13 |
| - "Running on PyMC3 v3.11.2\n" |
| 13 | + "Running on PyMC v3.11.2\n" |
14 | 14 | ]
|
15 | 15 | }
|
16 | 16 | ],
|
|
22 | 22 | "import matplotlib.pyplot as plt\n",
|
23 | 23 | "import numpy as np\n",
|
24 | 24 | "import pandas as pd\n",
|
25 |
| - "import pymc3 as pm\n", |
26 |
| - "import pymc3.math as pmm\n", |
| 25 | + "import pymc as pm\n", |
| 26 | + "import pymc.math as pmm\n", |
27 | 27 | "\n",
|
28 | 28 | "from scipy.stats import bernoulli, expon\n",
|
29 | 29 | "\n",
|
30 |
| - "print(f\"Running on PyMC3 v{pm.__version__}\")" |
| 30 | + "print(f\"Running on PyMC v{pm.__version__}\")" |
31 | 31 | ]
|
32 | 32 | },
|
33 | 33 | {
|
|
98 | 98 | "\n",
|
99 | 99 | "With this, we can sample from the joint posterior of $\\theta_A, \\theta_B$. \n",
|
100 | 100 | "\n",
|
101 |
| - "You may have noticed that the Beta distribution is the conjugate prior for the Binomial, so we don't need MCMC sampling to estimate the posterior (the exact solution can be found in the VWO paper). We'll still demonstrate how sampling can be done with PyMC3 though, and doing this makes it easier to extend the model with different priors, dependency assumptions, etc.\n", |
| 101 | + "You may have noticed that the Beta distribution is the conjugate prior for the Binomial, so we don't need MCMC sampling to estimate the posterior (the exact solution can be found in the VWO paper). We'll still demonstrate how sampling can be done with PyMC though, and doing this makes it easier to extend the model with different priors, dependency assumptions, etc.\n", |
102 | 102 | "\n",
|
103 | 103 | "Finally, remember that our outcome of interest is whether B is better than A. A common measure in practice for whether B is better than is the _relative uplift in conversion rates_, i.e. the percentage difference of $\\theta_B$ over $\\theta_A$:\n",
|
104 | 104 | "\n",
|
105 | 105 | "$$\\mathrm{reluplift}_B = \\theta_B / \\theta_A - 1$$\n",
|
106 | 106 | "\n",
|
107 |
| - "We'll implement this model setup in PyMC3 below." |
| 107 | + "We'll implement this model setup in PyMC below." |
108 | 108 | ]
|
109 | 109 | },
|
110 | 110 | {
|
|
181 | 181 | "id": "8e1f6ca4",
|
182 | 182 | "metadata": {},
|
183 | 183 | "source": [
|
184 |
| - "Note that we can pass in arbitrary values for the observed data in these prior predictive checks. PyMC3 will not use that data when sampling from the prior predictive distribution." |
| 184 | + "Note that we can pass in arbitrary values for the observed data in these prior predictive checks. PyMC will not use that data when sampling from the prior predictive distribution." |
185 | 185 | ]
|
186 | 186 | },
|
187 | 187 | {
|
|
404 | 404 | " generated = generate_binomial_data(variants, true_rates, samples_per_variant)\n",
|
405 | 405 | " data = [BinomialData(**generated[v].to_dict()) for v in variants]\n",
|
406 | 406 | " with ConversionModelTwoVariant(priors=weak_prior).create_model(data):\n",
|
407 |
| - " trace_weak = pm.sample(draws=5000, return_inferencedata=True, cores=1, chains=2)\n", |
| 407 | + " trace_weak = pm.sample(draws=5000, cores=1, chains=2)\n", |
408 | 408 | " with ConversionModelTwoVariant(priors=strong_prior).create_model(data):\n",
|
409 |
| - " trace_strong = pm.sample(draws=5000, return_inferencedata=True, cores=1, chains=2)\n", |
| 409 | + " trace_strong = pm.sample(draws=5000, cores=1, chains=2)\n", |
410 | 410 | "\n",
|
411 | 411 | " true_rel_uplift = true_rates[1] / true_rates[0] - 1\n",
|
412 | 412 | "\n",
|
|
884 | 884 | " generated = generate_binomial_data(variants, true_rates, samples_per_variant)\n",
|
885 | 885 | " data = [BinomialData(**generated[v].to_dict()) for v in variants]\n",
|
886 | 886 | " with ConversionModel(priors).create_model(data=data, comparison_method=comparison_method):\n",
|
887 |
| - " trace = pm.sample(draws=5000, return_inferencedata=True, chains=2, cores=1)\n", |
| 887 | + " trace = pm.sample(draws=5000, chains=2, cores=1)\n", |
888 | 888 | "\n",
|
889 | 889 | " n_plots = len(variants)\n",
|
890 | 890 | " fig, axs = plt.subplots(nrows=n_plots, ncols=1, figsize=(3 * n_plots, 7), sharex=True)\n",
|
|
1439 | 1439 | " with RevenueModel(conversion_rate_prior, mean_purchase_prior).create_model(\n",
|
1440 | 1440 | " data, comparison_method\n",
|
1441 | 1441 | " ):\n",
|
1442 |
| - " trace = pm.sample(draws=5000, return_inferencedata=True, chains=2, cores=1)\n", |
| 1442 | + " trace = pm.sample(draws=5000, chains=2, cores=1)\n", |
1443 | 1443 | "\n",
|
1444 | 1444 | " n_plots = len(variants)\n",
|
1445 | 1445 | " fig, axs = plt.subplots(nrows=n_plots, ncols=1, figsize=(3 * n_plots, 7), sharex=True)\n",
|
|
1895 | 1895 | "* How do we plan the length and size of A/B tests using power analysis, if we're using Bayesian models to analyse the results?\n",
|
1896 | 1896 | "* Outside of the conversion rates (bernoulli random variables for each visitor), many value distributions in online software cannot be fit with nice densities like Normal, Gamma, etc. How do we model these?\n",
|
1897 | 1897 | "\n",
|
1898 |
| - "Various textbooks and online resources dive into these areas in more detail. [Doing Bayesian Data Analysis](http://doingbayesiandataanalysis.blogspot.com/) by John Kruschke is a great resource, and has been translated to PyMC3 here: https://github.com/JWarmenhoven/DBDA-python.\n", |
| 1898 | + "Various textbooks and online resources dive into these areas in more detail. [Doing Bayesian Data Analysis](http://doingbayesiandataanalysis.blogspot.com/) by John Kruschke is a great resource, and has been translated to PyMC here: https://github.com/JWarmenhoven/DBDA-python.\n", |
1899 | 1899 | "\n",
|
1900 |
| - "We also plan to create more PyMC3 tutorials on these topics, so stay tuned!\n", |
| 1900 | + "We also plan to create more PyMC tutorials on these topics, so stay tuned!\n", |
1901 | 1901 | "\n",
|
1902 | 1902 | "---\n",
|
1903 | 1903 | "\n",
|
|
1924 | 1924 | "Python version : 3.8.6\n",
|
1925 | 1925 | "IPython version : 7.23.1\n",
|
1926 | 1926 | "\n",
|
1927 |
| - "theano: 1.1.2\n", |
| 1927 | + "aesara: 1.1.2\n", |
1928 | 1928 | "xarray: 0.18.0\n",
|
1929 | 1929 | "\n",
|
1930 |
| - "pymc3 : 3.11.2\n", |
| 1930 | + "pymc : 3.11.2\n", |
1931 | 1931 | "arviz : 0.11.2\n",
|
1932 | 1932 | "matplotlib: 3.4.2\n",
|
1933 | 1933 | "pandas : 1.2.4\n",
|
|
1940 | 1940 | ],
|
1941 | 1941 | "source": [
|
1942 | 1942 | "%load_ext watermark\n",
|
1943 |
| - "%watermark -n -u -v -iv -w -p theano,xarray" |
| 1943 | + "%watermark -n -u -v -iv -w -p aesara,xarray" |
1944 | 1944 | ]
|
1945 | 1945 | }
|
1946 | 1946 | ],
|
|
0 commit comments