Skip to content

BUG: Series construction sharing data with DTI #22113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed

BUG: Series construction sharing data with DTI #22113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
72b7ee5
Fix Bug #21907
makbigc Jul 29, 2018
260afd8
Update test_constructors.py
makbigc Aug 7, 2018
ca4ff5d
Update v0.24.0.txt
makbigc Aug 7, 2018
f0397ef
Add assertion that expected[0] is not NaT
makbigc Nov 11, 2018
388a21a
Change data to data._values in checking condition
makbigc Nov 18, 2018
aba2368
remove duplicate line in v0.24.0.rst
makbigc Nov 19, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.24.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1303,6 +1303,7 @@ Indexing
- Bug where setting a timedelta column by ``Index`` causes it to be casted to double, and therefore lose precision (:issue:`23511`)
- Bug in :func:`Index.union` and :func:`Index.intersection` where name of the ``Index`` of the result was not computed correctly for certain cases (:issue:`9943`, :issue:`9862`)
- Bug in :class:`Index` slicing with boolean :class:`Index` may raise ``TypeError`` (:issue:`22533`)
- Bug when :class:`Series` was created from :class:`DatetimeIndex`, the series shares the same data with that index (:issue:`21907`)

Missing
^^^^^^^
Expand Down
7 changes: 5 additions & 2 deletions pandas/core/series.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,8 +181,11 @@ def __init__(self, data=None, index=None, dtype=None, name=None,
# astype copies
data = data.astype(dtype)
else:
# need to copy to avoid aliasing issues
data = data._values.copy()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't passing deep=True simply solve this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the problem only occurs with tz-aware cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a good solution here, to special case things. pass deep=True here; if this doesn't work, then the copy constructor is broken.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this bug is caused by the return value of _values. In datetimes.py, _values returns the DatetimeIndex itself for tz-aware and ndarray for tz-naive.

    @property
    def _values(self):
        # tz-naive -> ndarray
        # tz-aware -> DatetimeIndex
        if self.tz is not None:
            return self
        else:
            return self.values

That 's why this problem only occurs with tz-aware cases as jbrockmendel said.


In [2]: dti1 = pd.date_range('2016-01-01', periods=10, tz='US/Pacific')

In [3]: dti2 = pd.date_range('2016-01-01', periods=10)

In [4]: ser1 = pd.Series(dti1)

In [5]: ser2 = pd.Series(dti2)

In [6]: ser1[::3]=pd.NaT

In [7]: ser2[::3]=pd.NaT

In [8]: dti1
Out[8]: 
DatetimeIndex([                      'NaT', '2016-01-02 00:00:00-08:00',
               '2016-01-03 00:00:00-08:00',                       'NaT',
               '2016-01-05 00:00:00-08:00', '2016-01-06 00:00:00-08:00',
                                     'NaT', '2016-01-08 00:00:00-08:00',
               '2016-01-09 00:00:00-08:00',                       'NaT'],
              dtype='datetime64[ns, US/Pacific]', freq='D')

In [9]: dti2
Out[9]: 
DatetimeIndex(['2016-01-01', '2016-01-02', '2016-01-03', '2016-01-04',
               '2016-01-05', '2016-01-06', '2016-01-07', '2016-01-08',
               '2016-01-09', '2016-01-10'],
              dtype='datetime64[ns]', freq='D')

# GH21907
if isinstance(data._values, DatetimeIndex):
data = data.copy(deep=True)
else:
data = data._values.copy()
copy = False

elif isinstance(data, np.ndarray):
Expand Down
18 changes: 18 additions & 0 deletions pandas/tests/series/test_constructors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -970,6 +970,24 @@ def test_constructor_set(self):
values = frozenset(values)
pytest.raises(TypeError, Series, values)

@pytest.mark.parametrize(
"src",
[pd.date_range('2016-01-01', periods=10, tz='US/Pacific'),
pd.timedelta_range('1 day', periods=10),
pd.period_range('2012Q1', periods=3, freq='Q'),
pd.Index(list('abcdefghij')),
pd.Int64Index(np.arange(10)),
pd.RangeIndex(0, 10)])
def test_fromIndex(self, src):
# GH21907
# When constructing a series from an index,
# the series does not share data with that index
expected = src.copy(deep=True)
prod = Series(src)
prod[::3] = NaT
assert expected[0] is not NaT
tm.assert_index_equal(src, expected)

# https://github.com/pandas-dev/pandas/issues/22698
@pytest.mark.filterwarnings("ignore:elementwise comparison:FutureWarning")
def test_fromDict(self):
Expand Down