notebooks/pymc3_howto/profiling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Profiling\n",
    "Sometimes computing the likelihood is not as fast as we would like. Theano provides handy profiling tools which are wrapped in PyMC3 by `model.profile`. This function returns a `ProfileStats` object conveying information about the underlying Theano operations. Here we'll profile the likelihood and gradient for the stochastic volatility example.\n",
    "\n",
    "First we build the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running on PyMC3 v3.9.3\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import pymc3 as pm\n",
    "\n",
    "print(f\"Running on PyMC3 v{pm.__version__}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "RANDOM_SEED = 8927\n",
    "np.random.seed(RANDOM_SEED)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Load the data\n",
    "returns = pd.read_csv(pm.get_data(\"SP500.csv\"), index_col=0, parse_dates=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Stochastic volatility example\n",
    "with pm.Model() as model:\n",
    "    sigma = pm.Exponential(\"sigma\", 1.0 / 0.02, testval=0.1)\n",
    "    nu = pm.Exponential(\"nu\", 1.0 / 10)\n",
    "    s = pm.GaussianRandomWalk(\"s\", sigma ** -2, shape=returns.shape[0])\n",
    "    r = pm.StudentT(\"r\", nu, lam=np.exp(-2 * s), observed=returns[\"change\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we call the `profile` function and summarize its return values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Function profiling\n",
      "==================\n",
      "  Message: /home/amit/Work/OpenSourceDev/pymc3/pymc3/model.py:1192\n",
      "  Time in 1000 calls to Function.__call__: 2.042763e-01s\n",
      "  Time in Function.fn.__call__: 1.832466e-01s (89.705%)\n",
      "  Time in thunks: 1.743519e-01s (85.351%)\n",
      "  Total compile time: 7.987223e-01s\n",
      "    Number of Apply nodes: 22\n",
      "    Theano Optimizer time: 7.478609e-01s\n",
      "       Theano validate time: 2.320766e-03s\n",
      "    Theano Linker time (includes C, CUDA code generation/compiling): 3.071356e-02s\n",
      "       Import time 1.030779e-02s\n",
      "       Node make_thunk time 2.952695e-02s\n",
      "           Node Elemwise{Composite{Switch(Cast{int8}((GT(Composite{exp((i0 * i1))}(i0, i1), i2) * i3 * GT(inv(sqrt(Composite{exp((i0 * i1))}(i0, i1))), i2))), (((i4 + (i5 * log(((i6 * Composite{exp((i0 * i1))}(i0, i1)) / i7)))) - i8) - (i5 * i9 * log1p(((Composite{exp((i0 * i1))}(i0, i1) * i10) / i7)))), i11)}}(TensorConstant{(1,) of -2.0}, s, TensorConstant{(1,) of 0}, Elemwise{gt,no_inplace}.0, Elemwise{Composite{scalar_gammaln((i0 * i1))}}.0, TensorConstant{(1,) of 0.5}, TensorConstant{(1,) of 0...8861837907}, InplaceDimShuffle{x}.0, Elemwise{Composite{scalar_gammaln((i0 * i1))}}.0, Elemwise{add,no_inplace}.0, TensorConstant{[1.0435322..54666e-07]}, TensorConstant{(1,) of -inf}) time 2.954960e-03s\n",
      "           Node Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 - (i3 * i0)), i4) + i5)}}[(0, 0)](nu, TensorConstant{0}, TensorConstant{-2.3025850929940455}, TensorConstant{0.1}, TensorConstant{-inf}, nu_log__) time 2.277613e-03s\n",
      "           Node Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * sqr((i3 - i4)))) + i5)), i6)}}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{Composite{inv(sqr(i0))}}[(0, 0)].0, Subtensor{int64::}.0, Subtensor{:int64:}.0, Elemwise{Composite{log((i0 * i1))}}.0, TensorConstant{(1,) of -inf}) time 2.170801e-03s\n",
      "           Node Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 - (i3 * i0)), i4) + i5)}}(sigma, TensorConstant{0}, TensorConstant{3.912023005428146}, TensorConstant{50.0}, TensorConstant{-inf}, sigma_log__) time 2.163649e-03s\n",
      "           Node Elemwise{Composite{scalar_gammaln((i0 * i1))}}(TensorConstant{(1,) of 0.5}, InplaceDimShuffle{x}.0) time 1.938581e-03s\n",
      "\n",
      "Time in all call to theano.grad() 0.000000e+00s\n",
      "Time since theano import 5.477s\n",
      "Class\n",
      "---\n",
      "<% time> <sum %> <apply time> <time per call> <type> <#call> <#apply> <Class name>\n",
      "  93.1%    93.1%       0.162s       1.16e-05s     C    14000      14   theano.tensor.elemwise.Elemwise\n",
      "   5.8%    99.0%       0.010s       3.39e-06s     C     3000       3   theano.tensor.elemwise.Sum\n",
      "   0.5%    99.5%       0.001s       4.64e-07s     C     2000       2   theano.tensor.elemwise.DimShuffle\n",
      "   0.3%    99.8%       0.001s       2.97e-07s     C     2000       2   theano.tensor.subtensor.Subtensor\n",
      "   0.2%   100.0%       0.000s       3.08e-07s     C     1000       1   theano.tensor.opt.MakeVector\n",
      "   ... (remaining 0 Classes account for   0.00%(0.00s) of the runtime)\n",
      "\n",
      "Ops\n",
      "---\n",
      "<% time> <sum %> <apply time> <time per call> <type> <#call> <#apply> <Op name>\n",
      "  87.9%    87.9%       0.153s       1.53e-04s     C     1000        1   Elemwise{Composite{Switch(Cast{int8}((GT(Composite{exp((i0 * i1))}(i0, i1), i2) * i3 * GT(inv(sqrt(Composite{exp((i0 * i1))}(i0, i1))), i2))), (((i4 + (i5 * log(((i6 * Composite{exp((i0 * i1))}(i0, i1)) / i7)))) - i8) - (i5 * i9 * log1p(((Composite{exp((i0 * i1))}(i0, i1) * i10) / i7)))), i11)}}\n",
      "   5.8%    93.7%       0.010s       3.39e-06s     C     3000        3   Sum{acc_dtype=float64}\n",
      "   3.4%    97.1%       0.006s       5.88e-06s     C     1000        1   Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * sqr((i3 - i4)))) + i5)), i6)}}\n",
      "   0.5%    97.6%       0.001s       4.64e-07s     C     2000        2   InplaceDimShuffle{x}\n",
      "   0.4%    98.0%       0.001s       3.19e-07s     C     2000        2   Elemwise{Composite{scalar_gammaln((i0 * i1))}}\n",
      "   0.3%    98.3%       0.000s       2.49e-07s     C     2000        2   Elemwise{exp,no_inplace}\n",
      "   0.2%    98.5%       0.000s       3.47e-07s     C     1000        1   Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 - (i3 * i0)), i4) + i5)}}[(0, 0)]\n",
      "   0.2%    98.7%       0.000s       3.37e-07s     C     1000        1   Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 - (i3 * i0)), i4) + i5)}}\n",
      "   0.2%    98.9%       0.000s       3.35e-07s     C     1000        1   Subtensor{int64::}\n",
      "   0.2%    99.1%       0.000s       3.30e-07s     C     1000        1   Elemwise{Composite{log((i0 * i1))}}\n",
      "   0.2%    99.2%       0.000s       3.08e-07s     C     1000        1   MakeVector{dtype='float64'}\n",
      "   0.2%    99.4%       0.000s       2.86e-07s     C     1000        1   Elemwise{add,no_inplace}\n",
      "   0.2%    99.6%       0.000s       2.83e-07s     C     1000        1   Elemwise{Composite{Cast{int8}(GT(i0, i1))}}\n",
      "   0.1%    99.7%       0.000s       2.58e-07s     C     1000        1   Subtensor{:int64:}\n",
      "   0.1%    99.8%       0.000s       2.28e-07s     C     1000        1   Elemwise{gt,no_inplace}\n",
      "   0.1%    99.9%       0.000s       1.50e-07s     C     1000        1   Elemwise{Composite{inv(sqrt(inv(sqr(i0))))}}[(0, 0)]\n",
      "   0.1%   100.0%       0.000s       1.30e-07s     C     1000        1   Elemwise{Composite{inv(sqr(i0))}}[(0, 0)]\n",
      "   ... (remaining 0 Ops account for   0.00%(0.00s) of the runtime)\n",
      "\n",
      "Apply\n",
      "------\n",
      "<% time> <sum %> <apply time> <time per call> <#call> <id> <Apply name>\n",
      "  87.9%    87.9%       0.153s       1.53e-04s   1000    14   Elemwise{Composite{Switch(Cast{int8}((GT(Composite{exp((i0 * i1))}(i0, i1), i2) * i3 * GT(inv(sqrt(Composite{exp((i0 * i1))}(i0, i1))), i2))), (((i4 + (i5 * log(((i6 * Composite{exp((i0 * i1))}(i0, i1)) / i7)))) - i8) - (i5 * i9 * log1p(((Composite{exp((i0 * i1))}(i0, i1) * i10) / i7)))), i11)}}(TensorConstant{(1,) of -2.0}, s, TensorConstant{(1,) of 0}, Elemwise{gt,no_inplace}.0, Elemwise{Composite{scalar_gammaln((i0 * i1))}}.0, TensorConstant{(1,)\n",
      "   3.4%    91.3%       0.006s       5.88e-06s   1000    18   Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * sqr((i3 - i4)))) + i5)), i6)}}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{Composite{inv(sqr(i0))}}[(0, 0)].0, Subtensor{int64::}.0, Subtensor{:int64:}.0, Elemwise{Composite{log((i0 * i1))}}.0, TensorConstant{(1,) of -inf})\n",
      "   2.9%    94.2%       0.005s       5.02e-06s   1000    19   Sum{acc_dtype=float64}(Elemwise{Composite{Switch(i0, (i1 * ((-(i2 * sqr((i3 - i4)))) + i5)), i6)}}.0)\n",
      "   2.9%    97.0%       0.005s       4.99e-06s   1000    17   Sum{acc_dtype=float64}(Elemwise{Composite{Switch(Cast{int8}((GT(Composite{exp((i0 * i1))}(i0, i1), i2) * i3 * GT(inv(sqrt(Composite{exp((i0 * i1))}(i0, i1))), i2))), (((i4 + (i5 * log(((i6 * Composite{exp((i0 * i1))}(i0, i1)) / i7)))) - i8) - (i5 * i9 * log1p(((Composite{exp((i0 * i1))}(i0, i1) * i10) / i7)))), i11)}}.0)\n",
      "   0.3%    97.3%       0.001s       5.51e-07s   1000     4   InplaceDimShuffle{x}(nu)\n",
      "   0.2%    97.6%       0.000s       4.05e-07s   1000     8   Elemwise{Composite{scalar_gammaln((i0 * i1))}}(TensorConstant{(1,) of 0.5}, InplaceDimShuffle{x}.0)\n",
      "   0.2%    97.8%       0.000s       3.76e-07s   1000     6   InplaceDimShuffle{x}(sigma)\n",
      "   0.2%    98.0%       0.000s       3.47e-07s   1000    15   Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 - (i3 * i0)), i4) + i5)}}[(0, 0)](nu, TensorConstant{0}, TensorConstant{-2.3025850929940455}, TensorConstant{0.1}, TensorConstant{-inf}, nu_log__)\n",
      "   0.2%    98.2%       0.000s       3.39e-07s   1000     1   Elemwise{exp,no_inplace}(sigma_log__)\n",
      "   0.2%    98.4%       0.000s       3.37e-07s   1000     5   Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 - (i3 * i0)), i4) + i5)}}(sigma, TensorConstant{0}, TensorConstant{3.912023005428146}, TensorConstant{50.0}, TensorConstant{-inf}, sigma_log__)\n",
      "   0.2%    98.6%       0.000s       3.35e-07s   1000     3   Subtensor{int64::}(s, Constant{1})\n",
      "   0.2%    98.7%       0.000s       3.30e-07s   1000    16   Elemwise{Composite{log((i0 * i1))}}(TensorConstant{(1,) of 0...4309189535}, Elemwise{Composite{inv(sqr(i0))}}[(0, 0)].0)\n",
      "   0.2%    98.9%       0.000s       3.08e-07s   1000    20   MakeVector{dtype='float64'}(__logp_sigma_log__, __logp_nu_log__, __logp_s, __logp_r)\n",
      "   0.2%    99.1%       0.000s       2.86e-07s   1000     7   Elemwise{add,no_inplace}(TensorConstant{(1,) of 1.0}, InplaceDimShuffle{x}.0)\n",
      "   0.2%    99.2%       0.000s       2.83e-07s   1000    12   Elemwise{Composite{Cast{int8}(GT(i0, i1))}}(Elemwise{Composite{inv(sqrt(inv(sqr(i0))))}}[(0, 0)].0, TensorConstant{(1,) of 0})\n",
      "   0.1%    99.4%       0.000s       2.58e-07s   1000     2   Subtensor{:int64:}(s, Constant{-1})\n",
      "   0.1%    99.5%       0.000s       2.33e-07s   1000    11   Elemwise{Composite{scalar_gammaln((i0 * i1))}}(TensorConstant{(1,) of 0.5}, Elemwise{add,no_inplace}.0)\n",
      "   0.1%    99.7%       0.000s       2.28e-07s   1000     9   Elemwise{gt,no_inplace}(InplaceDimShuffle{x}.0, TensorConstant{(1,) of 0})\n",
      "   0.1%    99.8%       0.000s       1.58e-07s   1000     0   Elemwise{exp,no_inplace}(nu_log__)\n",
      "   0.1%    99.8%       0.000s       1.50e-07s   1000    10   Elemwise{Composite{inv(sqrt(inv(sqr(i0))))}}[(0, 0)](InplaceDimShuffle{x}.0)\n",
      "   ... (remaining 2 Apply instances account for 0.16%(0.00s) of the runtime)\n",
      "\n",
      "Here are tips to potentially make your code run faster\n",
      "                 (if you think of new ones, suggest them on the mailing list).\n",
      "                 Test them first, as they are not guaranteed to always provide a speedup.\n",
      "  - Try the Theano flag floatX=float32\n",
      "We don't know if amdlibm will accelerate this scalar op. scalar_gammaln\n",
      "We don't know if amdlibm will accelerate this scalar op. scalar_gammaln\n",
      "  - Try installing amdlibm and set the Theano flag lib.amdlibm=True. This speeds up only some Elemwise operation.\n"
     ]
    }
   ],
   "source": [
    "# Profiling of the logp call\n",
    "model.profile(model.logpt).summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Function profiling\n",
      "==================\n",
      "  Message: /home/amit/Work/OpenSourceDev/pymc3/pymc3/model.py:1192\n",
      "  Time in 1000 calls to Function.__call__: 2.495768e-01s\n",
      "  Time in Function.fn.__call__: 2.238679e-01s (89.699%)\n",
      "  Time in thunks: 2.018890e-01s (80.893%)\n",
      "  Total compile time: 1.827419e+00s\n",
      "    Number of Apply nodes: 47\n",
      "    Theano Optimizer time: 1.549069e+00s\n",
      "       Theano validate time: 8.029699e-03s\n",
      "    Theano Linker time (includes C, CUDA code generation/compiling): 2.464304e-01s\n",
      "       Import time 1.863251e-01s\n",
      "       Node make_thunk time 2.439311e-01s\n",
      "           Node Elemwise{Composite{Switch(i0, ((i1 * i2 * i3 * i4) / i5), i6)}}(Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{add,no_inplace}.0, Elemwise{Composite{exp((i0 * i1))}}.0, TensorConstant{[1.0435322..54666e-07]}, Elemwise{Add}[(0, 1)].0, TensorConstant{(1,) of 0}) time 3.346968e-02s\n",
      "           Node Elemwise{switch,no_inplace}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 1.0}, TensorConstant{(1,) of 0.0}) time 3.334451e-02s\n",
      "           Node Shape_i{0}(s) time 2.912211e-02s\n",
      "           Node Elemwise{Composite{inv(sqr(i0))}}(sigma) time 2.812171e-02s\n",
      "           Node Elemwise{Composite{Switch(i0, (i1 * sqr((i2 - i3))), i4)}}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Subtensor{int64::}.0, Subtensor{:int64:}.0, TensorConstant{(1,) of 0}) time 1.561642e-02s\n",
      "\n",
      "Time in all call to theano.grad() 2.058767e+00s\n",
      "Time since theano import 9.911s\n",
      "Class\n",
      "---\n",
      "<% time> <sum %> <apply time> <time per call> <type> <#call> <#apply> <Class name>\n",
      "  72.2%    72.2%       0.146s       5.83e-06s     C    25000      25   theano.tensor.elemwise.Elemwise\n",
      "  17.6%    89.8%       0.036s       5.09e-06s     C     7000       7   theano.tensor.elemwise.Sum\n",
      "   5.1%    94.9%       0.010s       5.13e-06s     C     2000       2   theano.tensor.subtensor.IncSubtensor\n",
      "   1.7%    96.7%       0.003s       3.50e-06s     C     1000       1   theano.tensor.basic.Join\n",
      "   1.4%    98.1%       0.003s       2.85e-06s     C     1000       1   theano.tensor.basic.Alloc\n",
      "   0.9%    99.0%       0.002s       4.49e-07s     C     4000       4   theano.tensor.elemwise.DimShuffle\n",
      "   0.4%    99.4%       0.001s       4.38e-07s     C     2000       2   theano.tensor.subtensor.Subtensor\n",
      "   0.3%    99.7%       0.001s       3.29e-07s     C     2000       2   theano.tensor.basic.Reshape\n",
      "   0.2%    99.9%       0.000s       3.31e-07s     C     1000       1   theano.compile.ops.Shape_i\n",
      "   0.1%   100.0%       0.000s       1.12e-07s     C     2000       2   theano.compile.ops.Rebroadcast\n",
      "   ... (remaining 0 Classes account for   0.00%(0.00s) of the runtime)\n",
      "\n",
      "Ops\n",
      "---\n",
      "<% time> <sum %> <apply time> <time per call> <type> <#call> <#apply> <Op name>\n",
      "  20.5%    20.5%       0.041s       4.14e-05s     C     1000        1   Elemwise{Composite{Switch(i0, (-log1p((i1 / i2))), i3)}}\n",
      "  17.6%    38.2%       0.036s       5.09e-06s     C     7000        7   Sum{acc_dtype=float64}\n",
      "  16.6%    54.8%       0.034s       3.36e-05s     C     1000        1   Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}\n",
      "  12.9%    67.7%       0.026s       2.59e-05s     C     1000        1   Elemwise{Composite{exp((i0 * i1))}}\n",
      "   4.6%    72.2%       0.009s       2.30e-06s     C     4000        4   Elemwise{switch,no_inplace}\n",
      "   3.8%    76.1%       0.008s       7.75e-06s     C     1000        1   Elemwise{Composite{Switch(i0, (i1 * (i2 + ((i3 * i4 * i5 * i6) / i7))), i8)}}[(0, 6)]\n",
      "   3.3%    79.4%       0.007s       6.68e-06s     C     1000        1   Elemwise{Composite{Switch(i0, ((i1 * i2 * i3 * i4) / i5), i6)}}\n",
      "   3.2%    82.5%       0.006s       6.44e-06s     C     1000        1   IncSubtensor{InplaceInc;int64::}\n",
      "   2.4%    85.0%       0.005s       4.94e-06s     C     1000        1   Elemwise{Composite{Switch(i0, (i1 * i2 * (i3 - i4)), i5)}}\n",
      "   2.3%    87.2%       0.005s       4.55e-06s     C     1000        1   Elemwise{Composite{Switch(i0, (i1 * (i2 - i3)), i4)}}\n",
      "   2.2%    89.4%       0.004s       4.42e-06s     C     1000        1   Elemwise{Composite{Switch(i0, (i1 * sqr((i2 - i3))), i4)}}\n",
      "   1.9%    91.3%       0.004s       3.83e-06s     C     1000        1   IncSubtensor{InplaceInc;:int64:}\n",
      "   1.7%    93.1%       0.003s       3.50e-06s     C     1000        1   Join\n",
      "   1.4%    94.5%       0.003s       2.85e-06s     C     1000        1   Alloc\n",
      "   1.0%    95.5%       0.002s       2.09e-06s     C     1000        1   Elemwise{mul,no_inplace}\n",
      "   0.9%    96.4%       0.002s       4.49e-07s     C     4000        4   InplaceDimShuffle{x}\n",
      "   0.4%    96.8%       0.001s       8.92e-07s     C     1000        1   Elemwise{Add}[(0, 1)]\n",
      "   0.4%    97.2%       0.001s       7.88e-07s     C     1000        1   Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 * i0), i1) + i3 + (i4 * (((i5 * i6 * Composite{inv(Composite{(sqr(i0) * i0)}(i0))}(i7)) / i8) - (i9 * Composite{inv(Composite{(sqr(i0) * i0)}(i0))}(i7))) * (i10 ** i11) * inv(Composite{(sqr(i0) * i0)}(i0)) * i0))}}[(0, 0)]\n",
      "   0.4%    97.6%       0.001s       3.89e-07s     C     2000        2   Elemwise{exp,no_inplace}\n",
      "   0.4%    98.0%       0.001s       7.70e-07s     C     1000        1   Elemwise{Composite{(Switch(Cast{int8}(GE(i0, i1)), (i2 * i0), i1) + i3 + (i4 * i5 * psi((i4 * (i6 + i0))) * i0) + (i7 * i8) + (i4 * i9 * psi((i4 * i0)) * i0) + (i4 * i10 * i0) + i11)}}[(0, 0)]\n",
      "   ... (remaining 11 Ops account for   2.00%(0.00s) of the runtime)\n",
      "\n",
      "Apply\n",
      "------\n",
      "<% time> <sum %> <apply time> <time per call> <#call> <id> <Apply name>\n",
      "  20.5%    20.5%       0.041s       4.14e-05s   1000    19   Elemwise{Composite{Switch(i0, (-log1p((i1 / i2))), i3)}}(Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}.0, Elemwise{mul,no_inplace}.0, InplaceDimShuffle{x}.0, TensorConstant{(1,) of 0})\n",
      "  16.6%    37.2%       0.034s       3.36e-05s   1000    15   Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}(Elemwise{Composite{exp((i0 * i1))}}.0, TensorConstant{(1,) of 0}, Elemwise{gt,no_inplace}.0)\n",
      "  12.9%    50.0%       0.026s       2.59e-05s   1000     5   Elemwise{Composite{exp((i0 * i1))}}(TensorConstant{(1,) of -2.0}, s)\n",
      "   3.8%    53.9%       0.008s       7.75e-06s   1000    35   Elemwise{Composite{Switch(i0, (i1 * (i2 + ((i3 * i4 * i5 * i6) / i7))), i8)}}[(0, 6)](Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}.0, TensorConstant{(1,) of -2.0}, TensorConstant{(1,) of 0.5}, TensorConstant{(1,) of -0.5}, Elemwise{add,no_inplace}.0, TensorConstant{[1.0435322..54666e-07]}, Elemwise{Composite{exp((i0 * i1))}}.0, Elemwise{Add}[(0, 1)].0, TensorConstant{(1,) of 0})\n",
      "   3.3%    57.2%       0.007s       6.68e-06s   1000    32   Elemwise{Composite{Switch(i0, ((i1 * i2 * i3 * i4) / i5), i6)}}(Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}.0, TensorConstant{(1,) of 0.5}, Elemwise{add,no_inplace}.0, Elemwise{Composite{exp((i0 * i1))}}.0, TensorConstant{[1.0435322..54666e-07]}, Elemwise{Add}[(0, 1)].0, TensorConstant{(1,) of 0})\n",
      "   3.2%    60.4%       0.006s       6.44e-06s   1000    38   IncSubtensor{InplaceInc;int64::}(Elemwise{Composite{Switch(i0, (i1 * (i2 + ((i3 * i4 * i5 * i6) / i7))), i8)}}[(0, 6)].0, Elemwise{Composite{Switch(i0, (i1 * i2 * (i3 - i4)), i5)}}.0, Constant{1})\n",
      "   2.6%    62.9%       0.005s       5.17e-06s   1000    37   Sum{acc_dtype=float64}(Alloc.0)\n",
      "   2.5%    65.5%       0.005s       5.13e-06s   1000    36   Sum{acc_dtype=float64}(Elemwise{Composite{Switch(i0, ((i1 * i2 * i3 * i4) / i5), i6)}}.0)\n",
      "   2.5%    68.0%       0.005s       5.08e-06s   1000    31   Sum{acc_dtype=float64}(Elemwise{Switch}.0)\n",
      "   2.5%    70.5%       0.005s       5.08e-06s   1000    33   Sum{acc_dtype=float64}(Elemwise{Composite{Switch(i0, (i1 * sqr((i2 - i3))), i4)}}.0)\n",
      "   2.5%    73.0%       0.005s       5.08e-06s   1000    28   Sum{acc_dtype=float64}(Elemwise{Composite{Switch(i0, (-log1p((i1 / i2))), i3)}}.0)\n",
      "   2.5%    75.5%       0.005s       5.04e-06s   1000    29   Sum{acc_dtype=float64}(Elemwise{switch,no_inplace}.0)\n",
      "   2.5%    78.0%       0.005s       5.03e-06s   1000    30   Sum{acc_dtype=float64}(Elemwise{switch,no_inplace}.0)\n",
      "   2.4%    80.4%       0.005s       4.94e-06s   1000    25   Elemwise{Composite{Switch(i0, (i1 * i2 * (i3 - i4)), i5)}}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of -1.0}, InplaceDimShuffle{x}.0, Subtensor{int64::}.0, Subtensor{:int64:}.0, TensorConstant{(1,) of 0})\n",
      "   2.3%    82.7%       0.005s       4.55e-06s   1000    24   Elemwise{Composite{Switch(i0, (i1 * (i2 - i3)), i4)}}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, InplaceDimShuffle{x}.0, Subtensor{int64::}.0, Subtensor{:int64:}.0, TensorConstant{(1,) of 0})\n",
      "   2.2%    84.9%       0.004s       4.42e-06s   1000    26   Elemwise{Composite{Switch(i0, (i1 * sqr((i2 - i3))), i4)}}(Elemwise{Composite{Cast{int8}(GT(i0, i1))}}.0, TensorConstant{(1,) of 0.5}, Subtensor{int64::}.0, Subtensor{:int64:}.0, TensorConstant{(1,) of 0})\n",
      "   1.9%    86.8%       0.004s       3.83e-06s   1000    41   IncSubtensor{InplaceInc;:int64:}(IncSubtensor{InplaceInc;int64::}.0, Elemwise{Composite{Switch(i0, (i1 * (i2 - i3)), i4)}}.0, Constant{-1})\n",
      "   1.7%    88.5%       0.003s       3.50e-06s   1000    46   Join(TensorConstant{0}, Rebroadcast{1}.0, Rebroadcast{1}.0, (d__logp/ds))\n",
      "   1.6%    90.1%       0.003s       3.17e-06s   1000    22   Elemwise{Switch}(Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}.0, TensorConstant{(1,) of 1.0}, TensorConstant{(1,) of 0.0})\n",
      "   1.4%    91.5%       0.003s       2.92e-06s   1000    21   Elemwise{switch,no_inplace}(Elemwise{Composite{Cast{int8}((GT(i0, i1) * i2 * GT(inv(sqrt(i0)), i1)))}}.0, TensorConstant{(1,) of -0..4309189535}, TensorConstant{(1,) of 0})\n",
      "   ... (remaining 27 Apply instances account for 8.47%(0.02s) of the runtime)\n",
      "\n",
      "Here are tips to potentially make your code run faster\n",
      "                 (if you think of new ones, suggest them on the mailing list).\n",
      "                 Test them first, as they are not guaranteed to always provide a speedup.\n",
      "  - Try the Theano flag floatX=float32\n",
      "We don't know if amdlibm will accelerate this scalar op. psi\n",
      "We don't know if amdlibm will accelerate this scalar op. psi\n",
      "  - Try installing amdlibm and set the Theano flag lib.amdlibm=True. This speeds up only some Elemwise operation.\n"
     ]
    }
   ],
   "source": [
    "# Profiling of the gradient call dlogp/dx\n",
    "model.profile(pm.gradient(model.logpt, model.vars)).summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pymc3  3.9.3\n",
      "numpy  1.18.5\n",
      "pandas 1.0.5\n",
      "last updated: Sun Aug 30 2020 \n",
      "\n",
      "CPython 3.8.3\n",
      "IPython 7.16.1\n",
      "watermark 2.0.2\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -n -u -v -iv -w"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}