TYP: Overload series/ffill and bfill #41152

pckSF · 2021-04-25T16:30:23Z

Closes part of Function overloads MarcoGorelli/pyladieslondon-sprints#29
Adds typing.overload to: ffill and ``bfill` from https://github.com/pandas-dev/pandas/blob/master/pandas/core/generic.py

Setup and Output for `ffill`

# t_ffill.py
import pandas as pd

inplace: bool

reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=False))
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=True))
reveal_type(pd.DataFrame([1, None, None]).ffill())
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=inplace))

reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=False))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=True))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=inplace))

reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=False, limit=1))
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=True, limit=1))
reveal_type(pd.DataFrame([1, None, None]).ffill(limit=1))
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=inplace, limit=1))

reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=False, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=True, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=inplace, downcast="infer"))

reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=False, limit=1))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=True, limit=1))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, limit=1))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=inplace, limit=1))

reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=False, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=True, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=inplace, downcast="infer"))

reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=False, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=True, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(inplace=inplace, limit=1, downcast="infer"))

reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=False, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=True, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).ffill(axis=0, inplace=inplace, limit=1, downcast="infer"))

which results in the following mypy output:

$ mypy t_ffill.py
  t_ffill.py:6: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:7: note: Revealed type is 'None'
  t_ffill.py:8: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:9: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:11: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:12: note: Revealed type is 'None'
  t_ffill.py:13: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:14: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:16: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:17: note: Revealed type is 'None'
  t_ffill.py:18: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:19: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:21: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:22: note: Revealed type is 'None'
  t_ffill.py:23: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:24: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:26: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:27: note: Revealed type is 'None'
  t_ffill.py:28: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:29: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:31: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:32: note: Revealed type is 'None'
  t_ffill.py:33: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:34: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:36: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:37: note: Revealed type is 'None'
  t_ffill.py:38: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:39: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:41: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:42: note: Revealed type is 'None'
  t_ffill.py:43: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:44: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'

Setup and Ouput for `bfill`

# t_bfill.py
import pandas as pd

inplace: bool

reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=False))
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=True))
reveal_type(pd.DataFrame([1, None, None]).bfill())
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=inplace))

reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=False))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=True))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=inplace))

reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=False, limit=1))
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=True, limit=1))
reveal_type(pd.DataFrame([1, None, None]).bfill(limit=1))
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=inplace, limit=1))

reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=False, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=True, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=inplace, downcast="infer"))

reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=False, limit=1))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=True, limit=1))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, limit=1))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=inplace, limit=1))

reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=False, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=True, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=inplace, downcast="infer"))

reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=False, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=True, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(inplace=inplace, limit=1, downcast="infer"))

reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=False, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=True, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, limit=1, downcast="infer"))
reveal_type(pd.DataFrame([1, None, None]).bfill(axis=0, inplace=inplace, limit=1, downcast="infer"))

which results in the following mypy output:

$ mypy t_bfill.py
  t_ffill.py:6: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:7: note: Revealed type is 'None'
  t_ffill.py:8: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:9: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:11: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:12: note: Revealed type is 'None'
  t_ffill.py:13: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:14: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:16: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:17: note: Revealed type is 'None'
  t_ffill.py:18: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:19: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:21: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:22: note: Revealed type is 'None'
  t_ffill.py:23: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:24: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:26: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:27: note: Revealed type is 'None'
  t_ffill.py:28: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:29: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:31: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:32: note: Revealed type is 'None'
  t_ffill.py:33: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:34: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:36: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:37: note: Revealed type is 'None'
  t_ffill.py:38: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:39: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'
  t_ffill.py:41: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:42: note: Revealed type is 'None'
  t_ffill.py:43: note: Revealed type is 'pandas.core.frame.DataFrame*'
  t_ffill.py:44: note: Revealed type is 'Union[pandas.core.frame.DataFrame*, None]'

MarcoGorelli

Thanks Peter, have left some comments

Not too sure about downcast but that one can always be left out for now and typed later on

MarcoGorelli · 2021-04-25T17:10:28Z

pandas/core/generic.py

    @final
    @doc(klass=_shared_doc_kwargs["klass"])
    def ffill(
        self: FrameOrSeries,
-        axis=None,
+        axis: None | str | int = None,


axis has it's own type alias, Axis (check the fillna signature)

MarcoGorelli · 2021-04-25T17:11:40Z

pandas/core/generic.py

+    def ffill(
+        self: FrameOrSeries,
+        axis: None | str | int = ...,
+        inplace: Literal[True] = ...,


this one doesn't need =... - else, if you call .ffill(), mypy won't know which overload to match

there also needs to be an extra overload for when axis isn't passed:

@overload def ffill( self: FrameOrSeries, *, inplace: Literal[True],

MarcoGorelli

Looks good, thanks! Small comment about the return type in the function itself, other than that looks good to me

MarcoGorelli · 2021-04-25T19:46:38Z

pandas/core/generic.py

        downcast=None,
-    ) -> FrameOrSeries | None:
+    ):


I think it should work to keep the return types here, then the function body's return type will be fixed (admittedly this needs fixing in my blog post, will do that now)

MarcoGorelli

Thanks @pckSF !

* TYP: Overload series/ffill and bfill * Resolved comments * Keep the return types

TYP: Overload series/ffill and bfill

f405a77

MarcoGorelli suggested changes Apr 25, 2021

View reviewed changes

Resolved comments

a72fc4c

MarcoGorelli self-requested a review April 25, 2021 19:43

MarcoGorelli suggested changes Apr 25, 2021

View reviewed changes

Keep the return types

860fb3c

pckSF marked this pull request as ready for review April 25, 2021 20:47

MarcoGorelli added the Typing type annotations, mypy/pyright type checking label Apr 26, 2021

MarcoGorelli added this to the 1.3 milestone Apr 26, 2021

MarcoGorelli approved these changes Apr 26, 2021

View reviewed changes

MarcoGorelli merged commit 44b041e into pandas-dev:master Apr 26, 2021

yeshsurya pushed a commit to yeshsurya/pandas that referenced this pull request May 6, 2021

TYP: Overload series/ffill and bfill (pandas-dev#41152)

ffd6f2b

* TYP: Overload series/ffill and bfill * Resolved comments * Keep the return types

JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021

TYP: Overload series/ffill and bfill (pandas-dev#41152)

b78e379

* TYP: Overload series/ffill and bfill * Resolved comments * Keep the return types

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TYP: Overload series/ffill and bfill #41152

TYP: Overload series/ffill and bfill #41152

pckSF commented Apr 25, 2021 •

edited

Loading

MarcoGorelli left a comment

MarcoGorelli Apr 25, 2021

pckSF Apr 25, 2021

MarcoGorelli Apr 25, 2021

MarcoGorelli Apr 25, 2021

pckSF Apr 25, 2021

MarcoGorelli left a comment

MarcoGorelli Apr 25, 2021

pckSF Apr 25, 2021

MarcoGorelli left a comment

TYP: Overload series/ffill and bfill #41152

TYP: Overload series/ffill and bfill #41152

Conversation

pckSF commented Apr 25, 2021 • edited Loading

Setup and Output for ffill

Setup and Ouput for bfill

MarcoGorelli left a comment

Choose a reason for hiding this comment

MarcoGorelli Apr 25, 2021

Choose a reason for hiding this comment

pckSF Apr 25, 2021

Choose a reason for hiding this comment

MarcoGorelli Apr 25, 2021

Choose a reason for hiding this comment

MarcoGorelli Apr 25, 2021

Choose a reason for hiding this comment

pckSF Apr 25, 2021

Choose a reason for hiding this comment

MarcoGorelli left a comment

Choose a reason for hiding this comment

MarcoGorelli Apr 25, 2021

Choose a reason for hiding this comment

pckSF Apr 25, 2021

Choose a reason for hiding this comment

MarcoGorelli left a comment

Choose a reason for hiding this comment

pckSF commented Apr 25, 2021 •

edited

Loading

Setup and Output for `ffill`

Setup and Ouput for `bfill`