Skip to content

Commit 8b54fb7

Browse files
mroeschkeyehoshuadimarsky
authored andcommitted
DOC: Add docs about using numba with parallel (pandas-dev#46368)
1 parent 1f277af commit 8b54fb7

File tree

1 file changed

+22
-0
lines changed

1 file changed

+22
-0
lines changed

doc/source/user_guide/enhancingperf.rst

+22
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,28 @@ a larger amount of data points (e.g. 1+ million).
350350
In [6]: %timeit roll.apply(f, engine='cython', raw=True)
351351
3.92 s ± 59 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
352352
353+
If your compute hardware contains multiple CPUs, the largest performance gain can be realized by setting ``parallel`` to ``True``
354+
to leverage more than 1 CPU. Internally, pandas leverages numba to parallelize computations over the columns of a :class:`DataFrame`;
355+
therefore, this performance benefit is only beneficial for a :class:`DataFrame` with a large number of columns.
356+
357+
.. code-block:: ipython
358+
359+
In [1]: import numba
360+
361+
In [2]: numba.set_num_threads(1)
362+
363+
In [3]: df = pd.DataFrame(np.random.randn(10_000, 100))
364+
365+
In [4]: roll = df.rolling(100)
366+
367+
In [5]: %timeit roll.mean(engine="numba", engine_kwargs={"parallel": True})
368+
347 ms ± 26 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
369+
370+
In [6]: numba.set_num_threads(2)
371+
372+
In [7]: %timeit roll.mean(engine="numba", engine_kwargs={"parallel": True})
373+
201 ms ± 2.97 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
374+
353375
Custom Function Examples
354376
~~~~~~~~~~~~~~~~~~~~~~~~
355377

0 commit comments

Comments
 (0)