Skip to content

REGR: Series.update raises ValueError with tz-aware types #34020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
simonjayhawkins opened this issue May 6, 2020 · 3 comments
Closed
3 tasks done

REGR: Series.update raises ValueError with tz-aware types #34020

simonjayhawkins opened this issue May 6, 2020 · 3 comments
Labels
Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Regression Functionality that used to work in a prior pandas version Timezones Timezone data dtype
Milestone

Comments

@simonjayhawkins
Copy link
Member

simonjayhawkins commented May 6, 2020

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

import pandas as pd

print(pd.__version__)

dti = pd.date_range("1/1/2000", periods=3, tz="Europe/London")

a = pd.Series([dti[0], None, dti[2]])
b = pd.Series([None, dti[1], None])
a.update(b)

print(a)

Problem description

on master

1.1.0.dev0+1468.g3cebe3024
Traceback (most recent call last):
  File "C:/Users/simon/test.py", line 10, in <module>
    a.update(b)
  File "c:\users\simon\pandas\pandas\core\series.py", line 2855, in update
    self._mgr = self._mgr.putmask(mask=mask, new=other)
  File "c:\users\simon\pandas\pandas\core\internals\managers.py", line 544, in putmask
    return self.apply(
  File "c:\users\simon\pandas\pandas\core\internals\managers.py", line 397, in apply
    applied = getattr(b, f)(**kwargs)
  File "c:\users\simon\pandas\pandas\core\internals\blocks.py", line 1604, in putmask
    new_values[mask] = new
  File "c:\users\simon\pandas\pandas\core\arrays\datetimelike.py", line 609, in __setitem__
    self._data[key] = value
ValueError: NumPy boolean array indexing assignment cannot assign 3 input values to the 1 output values where the mask is true

regression in #27428

d717ad8 is the first bad commit
commit d717ad8
Author: jbrockmendel [email protected]
Date: Tue Jul 23 16:55:40 2019 -0700

CLN/REF: stop allowing iNaT in DatetimeBlock (#27428)

cc @jbrockmendel

Expected Output

on 0.25.3

0.25.3
0   2000-01-01 00:00:00+00:00
1   2000-01-02 00:00:00+00:00
2   2000-01-03 00:00:00+00:00
dtype: datetime64[ns, Europe/London]

Output of pd.show_versions()

[paste the output of pd.show_versions() here leaving a blank line after the details tag]

@simonjayhawkins simonjayhawkins added Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Regression Functionality that used to work in a prior pandas version Timezones Timezone data dtype labels May 6, 2020
@simonjayhawkins
Copy link
Member Author

xref #25744 AFAICT this is a regression from 0.25.3 whereas #25744 was already failing on 0.25.3 hence separate issue.

@simonjayhawkins simonjayhawkins modified the milestones: Contributions Welcome, 1.0.4 May 6, 2020
@simonjayhawkins
Copy link
Member Author

fixed in master, xref #33984

>>> import pandas as pd
>>>
>>> print(pd.__version__)
1.1.0.dev0+1482.g18a7e973c
>>>
>>> dti = pd.date_range("1/1/2000", periods=3, tz="Europe/London")
>>>
>>> a = pd.Series([dti[0], None, dti[2]])
>>> b = pd.Series([None, dti[1], None])
>>> a.update(b)
>>>
>>> print(a)
0   2000-01-01 00:00:00+00:00
1   2000-01-02 00:00:00+00:00
2   2000-01-03 00:00:00+00:00
dtype: datetime64[ns, Europe/London]
>>>

@simonjayhawkins simonjayhawkins added the Needs Tests Unit test(s) needed to prevent regressions label May 6, 2020
@simonjayhawkins simonjayhawkins removed good first issue Needs Tests Unit test(s) needed to prevent regressions labels May 11, 2020
@simonjayhawkins simonjayhawkins modified the milestones: 1.0.4, 1.0.5 May 30, 2020
@simonjayhawkins simonjayhawkins modified the milestones: 1.0.5, 1.1 Jun 15, 2020
@simonjayhawkins
Copy link
Member Author

moving this off the 1.0.x milestone, see #33984 (comment). fixed in master so closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Regression Functionality that used to work in a prior pandas version Timezones Timezone data dtype
Projects
None yet
Development

No branches or pull requests

2 participants