Skip to content

REF: de-duplicate precision_from_unit #51483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 23, 2023
Merged

REF: de-duplicate precision_from_unit #51483

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 11 additions & 38 deletions pandas/_libs/tslibs/conversion.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ from pandas._libs.tslibs.np_datetime cimport (
NPY_FR_us,
check_dts_bounds,
convert_reso,
get_conversion_factor,
get_datetime64_unit,
get_datetime64_value,
get_implementation_bounds,
Expand Down Expand Up @@ -83,9 +84,9 @@ TD64NS_DTYPE = np.dtype("m8[ns]")
# Unit Conversion Helpers

cdef int64_t cast_from_unit(
object ts,
str unit,
NPY_DATETIMEUNIT out_reso=NPY_FR_ns
object ts,
str unit,
NPY_DATETIMEUNIT out_reso=NPY_FR_ns
) except? -1:
"""
Return a casting of the unit represented to nanoseconds
Expand All @@ -106,10 +107,6 @@ cdef int64_t cast_from_unit(

m, p = precision_from_unit(unit, out_reso)

# just give me the unit back
if ts is None:
return m

if unit in ["Y", "M"]:
if is_float_object(ts) and not ts.is_integer():
# GH#47267 it is clear that 2 "M" corresponds to 1970-02-01,
Expand Down Expand Up @@ -148,8 +145,8 @@ cdef int64_t cast_from_unit(


cpdef inline (int64_t, int) precision_from_unit(
str unit,
NPY_DATETIMEUNIT out_reso=NPY_DATETIMEUNIT.NPY_FR_ns,
str unit,
NPY_DATETIMEUNIT out_reso=NPY_DATETIMEUNIT.NPY_FR_ns,
):
"""
Return a casting of the unit represented to nanoseconds + the precision
Expand All @@ -162,38 +159,14 @@ cpdef inline (int64_t, int) precision_from_unit(
"""
cdef:
int64_t m
int64_t multiplier
int p
NPY_DATETIMEUNIT reso = abbrev_to_npy_unit(unit)

multiplier = periods_per_second(out_reso)

if reso == NPY_DATETIMEUNIT.NPY_FR_Y:
# each 400 years we have 97 leap years, for an average of 97/400=.2425
# extra days each year. We get 31556952 by writing
# 3600*24*365.2425=31556952
m = multiplier * 31556952
elif reso == NPY_DATETIMEUNIT.NPY_FR_M:
# 2629746 comes from dividing the "Y" case by 12.
m = multiplier * 2629746
elif reso == NPY_DATETIMEUNIT.NPY_FR_W:
m = multiplier * 3600 * 24 * 7
elif reso == NPY_DATETIMEUNIT.NPY_FR_D:
m = multiplier * 3600 * 24
elif reso == NPY_DATETIMEUNIT.NPY_FR_h:
m = multiplier * 3600
elif reso == NPY_DATETIMEUNIT.NPY_FR_m:
m = multiplier * 60
elif reso == NPY_DATETIMEUNIT.NPY_FR_s:
m = multiplier
elif reso == NPY_DATETIMEUNIT.NPY_FR_ms:
m = multiplier // 1_000
elif reso == NPY_DATETIMEUNIT.NPY_FR_us:
m = multiplier // 1_000_000
elif reso == NPY_DATETIMEUNIT.NPY_FR_ns or reso == NPY_DATETIMEUNIT.NPY_FR_GENERIC:
m = multiplier // 1_000_000_000
else:
raise ValueError(f"cannot cast unit {unit}")
if reso == NPY_DATETIMEUNIT.NPY_FR_GENERIC:
reso = NPY_FR_ns

m = get_conversion_factor(reso, out_reso)

p = <int>log10(m) # number of digits in 'm' minus 1
return m, p

Expand Down
2 changes: 2 additions & 0 deletions pandas/_libs/tslibs/np_datetime.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -571,6 +571,8 @@ cdef int64_t get_conversion_factor(
return 1000 * get_conversion_factor(NPY_DATETIMEUNIT.NPY_FR_fs, to_unit)
elif from_unit == NPY_DATETIMEUNIT.NPY_FR_fs:
return 1000 * get_conversion_factor(NPY_DATETIMEUNIT.NPY_FR_as, to_unit)
else:
raise ValueError("Converting from M or Y units is not supported.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a new exception? (or just a message change)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

message change

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not tested or not public?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not reached, but want a helpful message here regardless

Copy link
Member

@mroeschke mroeschke Feb 23, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this may be reached and unhanded in some tests. Could you take a look?

FAILED pandas/tests/indexes/period/test_indexing.py::TestGetIndexer::test_get_indexer2 - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/timedeltas.pyx", line 436, in pandas._libs.tslibs.timedeltas.array_to_timedelta64
    ival = _item_to_timedelta64_fastpath(item)
  File "pandas/_libs/tslibs/timedeltas.pyx", line 468, in pandas._libs.tslibs.timedeltas._item_to_timedelta64_fastpath
    return parse_timedelta_string(item)
TypeError: Expected unicode, got numpy.timedelta64

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/scalar/timestamp/test_constructors.py::TestTimestampConstructors::test_constructor_int_float_with_YM_unit[int] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/scalar/timestamp/test_constructors.py::TestTimestampConstructors::test_constructor_int_float_with_YM_unit[float] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/scalar/timestamp/test_constructors.py::TestTimestampConstructors::test_constructor_float_not_round_with_YM_unit_deprecated - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150.0-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[True-150.0-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150.0-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_int[False-150.0-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[True-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[True-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[False-Y] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.
FAILED pandas/tests/tools/test_to_datetime.py::TestToDatetimeUnit::test_to_datetime_month_or_year_unit_non_round_float[False-M] - pytest.PytestUnraisableExceptionWarning: Exception ignored in: 'pandas._libs.tslibs.conversion.precision_from_unit'

Traceback (most recent call last):
  File "pandas/_libs/tslibs/np_datetime.pyx", line 575, in pandas._libs.tslibs.np_datetime.get_conversion_factor
    raise ValueError("Converting from M or Y units is not supported.")
ValueError: Converting from M or Y units is not supported.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@da-woods thoughts on how to not-suppress an exception raised in a function with signature

cpdef (int64_t, int) precision_from_unit(str unit, NPY_DATETIMEUNIT out_reso=*)

Removing the return type is sufficient; im hoping that it is not necessary.



cdef int64_t convert_reso(
Expand Down