Skip to content

BUG: read-only values in cython funcs #37613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 4, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -394,11 +394,13 @@ Datetimelike
- Bug in :meth:`TimedeltaIndex.sum` and :meth:`Series.sum` with ``timedelta64`` dtype on an empty index or series returning ``NaT`` instead of ``Timedelta(0)`` (:issue:`31751`)
- Bug in :meth:`DatetimeArray.shift` incorrectly allowing ``fill_value`` with a mismatched timezone (:issue:`37299`)
- Bug in adding a :class:`BusinessDay` with nonzero ``offset`` to a non-scalar other (:issue:`37457`)
- Bug in :func:`to_datetime` with a read-only array incorrectly raising (:issue:`34857`)

Timedelta
^^^^^^^^^
- Bug in :class:`TimedeltaIndex`, :class:`Series`, and :class:`DataFrame` floor-division with ``timedelta64`` dtypes and ``NaT`` in the denominator (:issue:`35529`)
- Bug in parsing of ISO 8601 durations in :class:`Timedelta`, :meth:`pd.to_datetime` (:issue:`37159`, fixes :issue:`29773` and :issue:`36204`)
- Bug in :func:`to_timedelta` with a read-only array incorrectly raising (:issue:`34857`)

Timezones
^^^^^^^^^
Expand Down
2 changes: 1 addition & 1 deletion pandas/_libs/join.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ ctypedef fused join_t:

@cython.wraparound(False)
@cython.boundscheck(False)
def left_join_indexer_unique(join_t[:] left, join_t[:] right):
def left_join_indexer_unique(ndarray[join_t] left, ndarray[join_t] right):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe these need const?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can't use const join_t until cython3

cdef:
Py_ssize_t i, j, nleft, nright
ndarray[int64_t] indexer
Expand Down
4 changes: 2 additions & 2 deletions pandas/_libs/tslibs/strptime.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ from _thread import allocate_lock as _thread_allocate_lock
import numpy as np
import pytz

from numpy cimport int64_t
from numpy cimport int64_t, ndarray

from pandas._libs.tslibs.nattype cimport (
NPY_NAT,
Expand Down Expand Up @@ -51,7 +51,7 @@ cdef dict _parse_code_table = {'y': 0,
'u': 22}


def array_strptime(object[:] values, object fmt, bint exact=True, errors='raise'):
def array_strptime(ndarray[object] values, object fmt, bint exact=True, errors='raise'):
"""
Calculates the datetime structs represented by the passed array of strings

Expand Down
2 changes: 1 addition & 1 deletion pandas/_libs/tslibs/timedeltas.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ cdef convert_to_timedelta64(object ts, str unit):

@cython.boundscheck(False)
@cython.wraparound(False)
def array_to_timedelta64(object[:] values, str unit=None, str errors="raise"):
def array_to_timedelta64(ndarray[object] values, str unit=None, str errors="raise"):
"""
Convert an ndarray to an array of timedeltas. If errors == 'coerce',
coerce non-convertible objects to NaT. Otherwise, raise.
Expand Down
7 changes: 6 additions & 1 deletion pandas/tests/libs/test_join.py
Original file line number Diff line number Diff line change
Expand Up @@ -135,9 +135,14 @@ def test_cython_inner_join(self):
tm.assert_numpy_array_equal(rs, exp_rs, check_dtype=False)


def test_left_join_indexer_unique():
@pytest.mark.parametrize("readonly", [True, False])
def test_left_join_indexer_unique(readonly):
a = np.array([1, 2, 3, 4, 5], dtype=np.int64)
b = np.array([2, 2, 3, 4, 4], dtype=np.int64)
if readonly:
# GH#37312, GH#37264
a.setflags(write=False)
b.setflags(write=False)

result = libjoin.left_join_indexer_unique(b, a)
expected = np.array([1, 1, 2, 3, 3], dtype=np.int64)
Expand Down
8 changes: 8 additions & 0 deletions pandas/tests/tools/test_to_datetime.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@


class TestTimeConversionFormats:
def test_to_datetime_readonly(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameterize over readonly

# GH#34857
arr = np.array([], dtype=object)
arr.setflags(write=False)
result = to_datetime(arr)
expected = to_datetime([])
tm.assert_index_equal(result, expected)

@pytest.mark.parametrize("cache", [True, False])
def test_to_datetime_format(self, cache):
values = ["1/1/2000", "1/2/2000", "1/3/2000"]
Expand Down
8 changes: 8 additions & 0 deletions pandas/tests/tools/test_to_timedelta.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@


class TestTimedeltas:
def test_to_timedelta_readonly(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

# GH#34857
arr = np.array([], dtype=object)
arr.setflags(write=False)
result = to_timedelta(arr)
expected = to_timedelta([])
tm.assert_index_equal(result, expected)

def test_to_timedelta(self):

result = to_timedelta(["", ""])
Expand Down