ENH: Add documentation for fast_apply on first row

Author: fjetter

doc/source/user_guide/groupby.rst

@@ -948,18 +948,38 @@ that is itself a series, and possibly upcast the result to a DataFrame:
 
 .. warning::
 
-    In the current implementation apply calls func twice on the
-    first group to decide whether it can take a fast or slow code
-    path. This can lead to unexpected behavior if func has
-    side-effects, as they will take effect twice for the first
-    group.
+    The current implementation uses a cythonized code path which requires
+    that the input data is not modified inplace. The heuristic assumes that
+    this might be happening if ``func(group) is group`` in which case we fall
+    back to a slow code path which evaluates func on the first group a second
+    time.
+    This can lead to unexpected behavior if func has side-effects,
+    as they will take effect twice for the first group.
+    This behavior is 
 
     .. ipython:: python
 
         d = pd.DataFrame({"a": ["x", "y"], "b": [1, 2]})
-        def identity(df):
-            print(df)
-            return df
+
+        def func_fast_apply(group):
+            """
+            This func doesn't modify inplace and returns
+            a scalar which is safe to fast apply
+            """
+            print(group.name)
+            return len(group)
+
+        d.groupby("a").apply(func_fast_apply)
+
+        def identity(group):
+            """
+            This triggers the slow path because ``identity(group) is group``
+            If there is no inplace modification happening
+            this may be avoided by returning a shallow copy
+            i.e. return group.copy()
+            """
+            print(group.name)
+            return group
 
         d.groupby("a").apply(identity)
 

doc/source/whatsnew/v0.25.0.rst

@@ -26,6 +26,70 @@ Other Enhancements
 Backwards incompatible API changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
+Fast GroupBy.apply on ``DataFrame`` evaluates first group only once
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+(:issue:`2936`, :issue:`2656`, :issue:`7739`, :issue:`10519`, :issue:`12155`,
+:issue:`20084`, :issue:`21417`)
+
+The implementation of ``DataFrame.groupby.apply`` previously evaluated func
+consistently twice on the first group to infer if it is safe to use a fast
+code path. Particularly for functions with side effects, this was an undesired
+behavior and may have led to surprises.
+
+The new behavior is that the first group is no longer evaluated twice if the
+fast path can be used.
+
+Previous behavior:
+
+.. code-block:: ipython
+
+    In [2]: df = pd.DataFrame({"a": ["x", "y"], "b": [1, 2]})
+
+    In [3]: side_effects = []
+
+    In [4]: def func_fast_apply(group):
+    ...:     side_effects.append(group.name)
+    ...:     return len(group)
+    ...:
+
+    In [5]: df.groupby("a").apply(func_fast_apply)
+
+    In [6]: assert side_effects == ["x", "x", "y"]
+
+New behavior:
+
+.. ipython:: python
+
+    df = pd.DataFrame({"a": ["x", "y"], "b": [1, 2]})
+
+    side_effects = []
+    def func_fast_apply(group):
+        """
+        This func doesn't modify inplace and returns
+        a scalar which is safe to fast apply
+        """
+        side_effects.append(group.name)
+        return len(group)
+
+    df.groupby("a").apply(func_fast_apply)
+    side_effects
+
+    side_effects.clear()
+    def identity(group):
+        """
+        This triggers the slow path because ``identity(group) is group``
+        If there is no inplace modification happening
+        this may be avoided by returning a shallow copy
+        i.e. return group.copy()
+        """
+        side_effects.append(group.name)
+        return group
+
+    df.groupby("a").apply(identity)
+    side_effects
+
+
 .. _whatsnew_0250.api.other:
 
 Other API Changes