DataFrame.replace incompatible with type "dict[str, dict[int, int]]"

[ClementPinard]

Describe the bug
Calling DataFrame.replace(my_dict) triggers a pyright error when my_dict  is of the type dict[str, dict[int, int]], which is odd because it does fit the accepted type Mapping[Hashable, ReplaceValue]

To Reproduce

This code triggers a pyright error:

import pandas as pd

test_df = pd.DataFrame([[0,1],[1,0]], index=[*"ab"], columns=[*"cd"])
print(test_df)

replace_dict = {"c": {0:1}}
replaced_test_df = test_df.replace(replace_dict)

print(replaced_test_df)

test.py:8:36 - error: Argument of type "dict[str, dict[int, int]]" cannot be assigned to parameter "to_replace" of type "ReplaceValue | Mapping[Hashable, ReplaceValue]" in function "replace"
    Type "dict[str, dict[int, int]]" is not assignable to type "ReplaceValue | Mapping[Hashable, ReplaceValue]"
      "dict[str, dict[int, int]]" is not assignable to "str"
      "dict[str, dict[int, int]]" is not assignable to "bytes"
      "dict[str, dict[int, int]]" is not assignable to "date"
      "dict[str, dict[int, int]]" is not assignable to "datetime"
      "dict[str, dict[int, int]]" is not assignable to "timedelta"
      "dict[str, dict[int, int]]" is not assignable to "datetime64"
      "dict[str, dict[int, int]]" is not assignable to "timedelta64"
    ... (reportArgumentType)
1 error, 0 warnings, 0 informations

This code does not. instead of defining a dict and giving it, we construct directly in the function call.

import pandas as pd

test_df = pd.DataFrame([[0,1],[1,0]], index=[*"ab"], columns=[*"cd"])
print(test_df)

replaced_test_df = test_df.replace({"c": {0:1}})

print(replaced_test_df)

I am not sure exactly if this is a pyright or a pandas stubs problem.

Please complete the following information:

• OS: MacOS
• OS Version : Sonoma 14.7.4
• python version: 3.12.9
• version of type checker:  pyright 1.1.396
• version of installed pandas-stubs : 2.2.3.250308

[ClementPinard]

I have created an issue for pyright as well, because I am not entirely sure where the problem is : microsoft/pyright#10057

[Dr-Irv]

I think this can be fixed. I tried it with a simple example.

In _typing.pyi, change the definition of ReplaceValue from where it has Mapping[Hashable, Scalar] to be Mapping[HashableT, ScalarT] and in core/frame.pyi, for the method replace(), change the types in the various overloads that say:

to_replace: ReplaceValue | Mapping[Hashable, ReplaceValue]

to

to_replace: ReplaceValue | Mapping[HashableT2, ReplaceValue]

It may be necessary to create a TypeVar for ReplaceValue, and use Mapping[HashableT2, ReplaceValueT] instead. Not sure about that.

PR with tests welcome.

Tagging @MarcoGorelli since he made the change to the stubs for DataFrame.replace() in #1129

[MarcoGorelli]

thanks for the ping

I'm a bit confused by this one:

b: Mapping[Hashable, ReplaceValue] = {'c': {0:1}}  # type-checks fine
b: Mapping[Hashable, ReplaceValue] = replace_dict  # Type "dict[str, dict[int, int]]" is not assignable to declared type "Mapping[Hashable, ReplaceValue]"

taking a look

---

even simpler:

replace_val = {0:1}
a: ReplaceValue = {0:1}
a: ReplaceValue = replace_val  # Type "dict[int, int]" is not assignable to declared type "ReplaceValue"

[Dr-Irv]

I'm a bit confused by this one:

See the discussion in this comment microsoft/pyright#10057 (comment) and below

[MarcoGorelli]

learned a couple of new things, thanks!

• bidirectional type inference
• that mapping is invariant in its first argument