|
15 | 15 | "\n",
|
16 | 16 | "Here $P_0$ is the base probability measure on the space $\\Omega$. The precision parameter $\\alpha > 0$ controls how close samples from the Dirichlet process are to the base measure, $P_0$. As $\\alpha \\to \\infty$, samples from the Dirichlet process approach the base measure $P_0$.\n",
|
17 | 17 | "\n",
|
18 |
| - "Dirichlet processes have several properties that make then quite suitable to [MCMC](https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo) simulation.\n", |
| 18 | + "Dirichlet processes have several properties that make them quite suitable to [MCMC](https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo) simulation.\n", |
19 | 19 | "\n",
|
20 | 20 | "1. The posterior given [i.i.d.](https://en.wikipedia.org/wiki/Independent_and_identically_distributed_random_variables) observations $\\omega_1, \\ldots, \\omega_n$ from a Dirichlet process $P \\sim \\textrm{DP}(\\alpha, P_0)$ is also a Dirichlet process with\n",
|
21 | 21 | "\n",
|
|
41 | 41 | "\n",
|
42 | 42 | " $$P = \\sum_{i = 1}^\\infty w_i \\delta_{\\omega_i}.$$\n",
|
43 | 43 | "\n",
|
44 |
| - "3. The [stick-breaking process](https://en.wikipedia.org/wiki/Dirichlet_process#The_stick-breaking_process) gives an explicit construction of the weights $w_i$ and samples $\\omega_i$ above that is straightforward to sample from. If $\\beta_1, \\beta_2, \\ldots \\sim \\textrm{Beta}(1, \\alpha)$, then $w_i = \\beta_i \\prod_{j = 1}^{j - 1} (1 - \\beta_j)$. The relationship between this representation and stick breaking may be illustrated as follows:\n", |
| 44 | + "3. The [stick-breaking process](https://en.wikipedia.org/wiki/Dirichlet_process#The_stick-breaking_process) gives an explicit construction of the weights $w_i$ and samples $\\omega_i$ above that is straightforward to sample from. If $\\beta_1, \\beta_2, \\ldots \\sim \\textrm{Beta}(1, \\alpha)$, then $w_i = \\beta_i \\prod_{j = 1}^{n - 1} (1 - \\beta_j)$. The relationship between this representation and stick breaking may be illustrated as follows:\n", |
45 | 45 | " 1. Start with a stick of length one.\n",
|
46 | 46 | " 2. Break the stick into two portions, the first of proportion $w_1 = \\beta_1$ and the second of proportion $1 - w_1$.\n",
|
47 | 47 | " 3. Further break the second portion into two portions, the first of proportion $\\beta_2$ and the second of proportion $1 - \\beta_2$. The length of the first portion of this stick is $\\beta_2 (1 - \\beta_1)$; the length of the second portion is $(1 - \\beta_1) (1 - \\beta_2)$.\n",
|
48 | 48 | " 4. Continue breaking the second portion from the previous break in this manner forever. If $\\omega_1, \\omega_2, \\ldots \\sim P_0$, then\n",
|
49 | 49 | "\n",
|
50 | 50 | " $$P = \\sum_{i = 1}^\\infty w_i \\delta_{\\omega_i} \\sim \\textrm{DP}(\\alpha, P_0).$$\n",
|
| 51 | + " \n", |
| 52 | + "[Suggested Further Reading]: (http://mlg.eng.cam.ac.uk/tutorials/07/ywt.pdf) and (https://www.stats.ox.ac.uk/~teh/research/npbayes/Teh2010a.pdf) for a brief introduction to other flavours of Dirichlet Processes, and their applications.\n", |
51 | 53 | "\n",
|
52 | 54 | "We can use the stick-breaking process above to easily sample from a Dirichlet process in Python. For this example, $\\alpha = 2$ and the base distribution is $N(0, 1)$."
|
53 | 55 | ]
|
|
1158 | 1160 | "name": "python",
|
1159 | 1161 | "nbconvert_exporter": "python",
|
1160 | 1162 | "pygments_lexer": "ipython3",
|
1161 |
| - "version": "3.5.4" |
| 1163 | + "version": "3.6.7" |
1162 | 1164 | },
|
1163 | 1165 | "widgets": {
|
1164 | 1166 | "state": {},
|
|
0 commit comments