Skip to content

Commit 0bc2904

Browse files
committed
PERF: releasing the GIL, #8882
1 parent b08ab8e commit 0bc2904

File tree

8 files changed

+2944
-2619
lines changed

8 files changed

+2944
-2619
lines changed

doc/source/whatsnew/v0.17.0.txt

+25
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ users upgrade to this version.
1313

1414
Highlights include:
1515

16+
- Release the Global Interpreter Lock (GIL) on some cython operations, see :ref:`here <whatsnew_0170.gil>`
1617

1718
Check the :ref:`API Changes <whatsnew_0170.api>` and :ref:`deprecations <whatsnew_0170.deprecations>` before updating.
1819

@@ -56,8 +57,32 @@ Deprecations
5657
Removal of prior version deprecations/changes
5758
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
5859

60+
.. _dask: https://dask.readthedocs.org/en/latest/
61+
62+
.. _whatsnew_0170.gil:
63+
64+
Releasing the GIL
65+
~~~~~~~~~~~~~~~~~
66+
67+
We are releasing the global-interpreter-lock (GIL) on some cython operations.
68+
This will allow other threads to run simultaneously during computation, potentially allowing performance improvements
69+
from multi-threading. Notably ``groupby`` and some indexing operations are a benefit from this. (:issue:`8882`)
70+
71+
For example the groupby expression in the following code will have the GIL released during the factorization step, e.g. ``df.groupby('key')``
72+
as well as the ``.sum()`` operation.
73+
74+
.. code-block:: python
75+
76+
N = 1e6
77+
df = DataFrame({'key' : np.random.randint(0,ngroups,size=N),
78+
'data' : np.random.randn(N) })
79+
df.groupby('key')['data'].sum()
80+
81+
Releasing of the GIL could benefit an application that uses threads for user interactions (e.g. ``QT``), or performaning multi-threaded computations. A nice example of a library that can handle these types of computation-in-parallel is the dask_ library.
82+
5983
.. _whatsnew_0170.performance:
6084

85+
6186
Performance Improvements
6287
~~~~~~~~~~~~~~~~~~~~~~~~
6388
- Added vbench benchmarks for alternative ExcelWriter engines and reading Excel files (:issue:`7171`)

pandas/core/common.py

-1
Original file line numberDiff line numberDiff line change
@@ -839,7 +839,6 @@ def take_nd(arr, indexer, axis=0, out=None, fill_value=np.nan,
839839

840840
func = _get_take_nd_function(arr.ndim, arr.dtype, out.dtype,
841841
axis=axis, mask_info=mask_info)
842-
843842
indexer = _ensure_int64(indexer)
844843
func(arr, indexer, out, fill_value)
845844

0 commit comments

Comments
 (0)