Skip to content

BUG: Series creation with datetime64 with non-ns unit as object dtype #13876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 2, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v0.19.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -842,6 +842,8 @@ Bug Fixes
- Bug in ``RangeIndex`` can be created without no arguments rather than raises ``TypeError`` (:issue:`13793`)
- Bug in ``.value_counts`` raises ``OutOfBoundsDatetime`` if data exceeds ``datetime64[ns]`` bounds (:issue:`13663`)
- Bug in ``DatetimeIndex`` may raise ``OutOfBoundsDatetime`` if input ``np.datetime64`` has other unit than ``ns`` (:issue:`9114`)
- Bug in ``Series`` creation with ``np.datetime64`` which has other unit than ``ns`` as ``object`` dtype results in incorrect values (:issue:`13876`)

- Bug in ``isnull`` ``notnull`` raise ``TypeError`` if input datetime-like has other unit than ``ns`` (:issue:`13389`)
- Bug in ``.merge`` may raise ``TypeError`` if input datetime-like has other unit than ``ns`` (:issue:`13389`)

Expand Down
10 changes: 9 additions & 1 deletion pandas/tests/series/test_constructors.py
Original file line number Diff line number Diff line change
Expand Up @@ -381,13 +381,21 @@ def test_constructor_dtype_datetime64(self):
# coerce datetime64 non-ns properly
dates = date_range('01-Jan-2015', '01-Dec-2015', freq='M')
values2 = dates.view(np.ndarray).astype('datetime64[ns]')
expected = Series(values2, dates)
expected = Series(values2, index=dates)

for dtype in ['s', 'D', 'ms', 'us', 'ns']:
values1 = dates.view(np.ndarray).astype('M8[{0}]'.format(dtype))
result = Series(values1, dates)
assert_series_equal(result, expected)

# GH 13876
# coerce to non-ns to object properly
expected = Series(values2, index=dates, dtype=object)
for dtype in ['s', 'D', 'ms', 'us', 'ns']:
values1 = dates.view(np.ndarray).astype('M8[{0}]'.format(dtype))
result = Series(values1, index=dates, dtype=object)
assert_series_equal(result, expected)

# leave datetime.date alone
dates2 = np.array([d.date() for d in dates.to_pydatetime()],
dtype=object)
Expand Down
7 changes: 4 additions & 3 deletions pandas/tests/test_algos.py
Original file line number Diff line number Diff line change
Expand Up @@ -777,7 +777,6 @@ def test_datetime_likes(self):
exp_false = exp_first | exp_last

for case in cases:
print(case)
res_first = algos.duplicated(case, keep='first')
tm.assert_numpy_array_equal(res_first, exp_first)

Expand All @@ -788,7 +787,8 @@ def test_datetime_likes(self):
tm.assert_numpy_array_equal(res_false, exp_false)

# index
for idx in [pd.Index(case), pd.Index(case, dtype='category')]:
for idx in [pd.Index(case), pd.Index(case, dtype='category'),
pd.Index(case, dtype=object)]:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to the fix itself, but adding more tests to duplicated (this test revealed the bug).

res_first = idx.duplicated(keep='first')
tm.assert_numpy_array_equal(res_first, exp_first)

Expand All @@ -799,7 +799,8 @@ def test_datetime_likes(self):
tm.assert_numpy_array_equal(res_false, exp_false)

# series
for s in [pd.Series(case), pd.Series(case, dtype='category')]:
for s in [pd.Series(case), pd.Series(case, dtype='category'),
pd.Series(case, dtype=object)]:
res_first = s.duplicated(keep='first')
tm.assert_series_equal(res_first, pd.Series(exp_first))

Expand Down
2 changes: 2 additions & 0 deletions pandas/types/cast.py
Original file line number Diff line number Diff line change
Expand Up @@ -829,6 +829,8 @@ def _possibly_cast_to_datetime(value, dtype, errors='raise'):
# coerce datetimelike to object
elif is_datetime64_dtype(value) and not is_datetime64_dtype(dtype):
if is_object_dtype(dtype):
if value.dtype != _NS_DTYPE:
value = value.astype(_NS_DTYPE)
ints = np.asarray(value).view('i8')
return tslib.ints_to_pydatetime(ints)

Expand Down