Skip to content

pandas.replace with value numpy.nan causes recursion error on 1.5.1 #49514

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
timc opened this issue Nov 4, 2022 · 3 comments
Closed

pandas.replace with value numpy.nan causes recursion error on 1.5.1 #49514

timc opened this issue Nov 4, 2022 · 3 comments

Comments

@timc
Copy link

timc commented Nov 4, 2022

This appears related to #45725 which is marked closed but this is highly reproducible on 1.5.1, Python 3.8.2

import pandas
import numpy

data = list(numpy.nan for b in range(0,100))
df = pandas.DataFrame({'a': data})
df['a'].replace(numpy.nan, None)

Gives error

  File "C:\Users\carpt9\AppData\Roaming\JetBrains\PyCharm2022.2\scratches\scratch_2.py", line 9, in <module>
    df['a'].replace(numpy.nan, None)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\util\_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\series.py", line 5380, in replace
    return super().replace(
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\util\_decorators.py", line 331, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\generic.py", line 7280, in replace
    new_data = self._mgr.replace(
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\managers.py", line 467, in replace
    return self.apply(
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\managers.py", line 347, in apply
    applied = getattr(b, f)(**kwargs)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\blocks.py", line 612, in replace
    return blk.replace(
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\blocks.py", line 612, in replace
    return blk.replace(
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\blocks.py", line 612, in replace
    return blk.replace(
  [Previous line repeated 978 more times]
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\blocks.py", line 611, in replace
    blk = self.coerce_to_target_dtype(value)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\blocks.py", line 450, in coerce_to_target_dtype
    return self.astype(new_dtype, copy=False)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\internals\blocks.py", line 526, in astype
    new_values = astype_array_safe(values, dtype, copy=copy, errors=errors)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\dtypes\astype.py", line 299, in astype_array_safe
    new_values = astype_array(values, dtype, copy=copy)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\dtypes\astype.py", line 217, in astype_array
    if is_datetime64tz_dtype(dtype) and is_datetime64_dtype(values.dtype):
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\dtypes\common.py", line 390, in is_datetime64tz_dtype
    return DatetimeTZDtype.is_dtype(arr_or_dtype)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\dtypes\base.py", line 312, in is_dtype
    if isinstance(dtype, (ABCSeries, ABCIndex, ABCDataFrame, np.dtype)):
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\dtypes\generic.py", line 47, in _instancecheck
    return _check(inst) and not isinstance(inst, type)
  File "C:\Users\carpt9\AppData\Local\pypoetry\Cache\virtualenvs\ot-airflow-J7BRUYuz-py3.8\lib\site-packages\pandas\core\dtypes\generic.py", line 41, in _check
    return getattr(inst, attr, "_typ") in comp
RecursionError: maximum recursion depth exceeded while calling a Python object
@timc
Copy link
Author

timc commented Nov 4, 2022

Confirmed this works as expected on 1.4.4

@asishm
Copy link
Contributor

asishm commented Nov 4, 2022

the fix will be available in 1.5.2 when released.

@phofl
Copy link
Member

phofl commented Nov 4, 2022

Yes, this is a duplicate. Thanks for the report

@phofl phofl closed this as completed Nov 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants