Skip to content

REF: de-duplicate precision_from_unit #51483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 23, 2023

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

@@ -571,6 +571,8 @@ cdef int64_t get_conversion_factor(
return 1000 * get_conversion_factor(NPY_DATETIMEUNIT.NPY_FR_fs, to_unit)
elif from_unit == NPY_DATETIMEUNIT.NPY_FR_fs:
return 1000 * get_conversion_factor(NPY_DATETIMEUNIT.NPY_FR_as, to_unit)
else:
raise ValueError("Converting from M or Y units is not supported.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a new exception? (or just a message change)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

message change

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not tested or not public?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not reached, but want a helpful message here regardless

Copy link
Member

@mroeschke mroeschke Feb 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this may be reached and unhanded in some tests. Could you take a look?

FAILED pandas/tests/indexes/period/test_indexing.py::TestGetIndexer::test_get_indexer2 - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/timedeltas.pyx", line 436, in pandas._libs.tslibs.timedeltas.array_to_timedelta64
    ival = _item_to_timedelta64_fastpath(item)
  File "pandas/_libs/tslibs/timedeltas.pyx", line 468, in pandas._libs.tslibs.timedeltas._item_to_timedelta64_fastpath
    return parse_timedelta_string(item)
TypeError: Expected unicode, got numpy.timedelta64

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/scalar/timestamp/test_constructors.py::TestTimestampConstructors::test_constructor_int_float_with_YM_unit[int] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/scalar/timestamp/test_constructors.py::TestTimestampConstructors::test_constructor_int_float_with_YM_unit[float] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/scalar/timestamp/test_constructors.py::TestTimestampConstructors::test_constructor_float_not_round_with_YM_unit_deprecated - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150.0-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150.0-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150.0-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150.0-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[True-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[True-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[False-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[False-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@da-woods thoughts on how to not-suppress an exception raised in a function with signature

cpdef (int64_t, int) precision_from_unit(str unit, NPY_DATETIMEUNIT out_reso=*)

Removing the return type is sufficient; im hoping that it is not necessary.

@simonjayhawkins simonjayhawkins added the Refactor Internal refactoring of code label Feb 22, 2023
@simonjayhawkins simonjayhawkins added this to the 2.1 milestone Feb 22, 2023
Copy link
Member

@simonjayhawkins simonjayhawkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jbrockmendel

@mroeschke mroeschke merged commit d08d451 into pandas-dev:main Feb 23, 2023
@mroeschke
Copy link
Member

Thanks @jbrockmendel

@jbrockmendel jbrockmendel deleted the ref-precision_from_unit branch February 23, 2023 00:27
phofl added a commit that referenced this pull request Feb 23, 2023
@phofl
Copy link
Member

phofl commented Feb 23, 2023

Opened a pr to revert if there is no quick fix.

@MarcoGorelli
Copy link
Member

Is there a way to raise exceptions from a Cython function which returns a tuple? I think the issue is that

cpdef inline (int64_t, int) precision_from_unit(
    str unit,
    NPY_DATETIMEUNIT out_reso=NPY_DATETIMEUNIT.NPY_FR_ns,
):

may raise because it calls get_conversion_factor. Problem is, how to propagate it?

MarcoGorelli pushed a commit that referenced this pull request Feb 23, 2023
Revert "REF: de-duplicate precision_from_unit (#51483)"

This reverts commit d08d451.
@jbrockmendel
Copy link
Member Author

Wrapping the get_conversion_factor call in a try/except that just passed on failure (which will then raise on the next line bc m is not assigned) fixes the failing tests, but is pretty ugly. For now could check for M/Y before calling get_conversion_factor.

Problem is, how to propagate it?

This is weird. Seems like a cython bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Refactor Internal refactoring of code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants