Skip to content

Fix get_datetimestruct_days overflow #56001

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Aug 19, 2024
54 changes: 32 additions & 22 deletions pandas/_libs/src/vendored/numpy/datetime/np_datetime.c
Original file line number Diff line number Diff line change
Expand Up @@ -70,14 +70,20 @@ _Static_assert(0, "__has_builtin not detected; please try a newer compiler");
#endif
#endif

#define PD_CHECK_OVERFLOW(FUNC) \
#define XSTR(a) STR(a)
#define STR(a) #a

#define PD_RAISE_FOR_OVERFLOW \
PyGILState_STATE gstate = PyGILState_Ensure(); \
PyErr_SetString(PyExc_OverflowError, \
"Overflow occurred at " __FILE__ ":" XSTR(__LINE__)); \
PyGILState_Release(gstate); \
return -1;

#define PD_CHECK_OVERFLOW(EXPR) \
do { \
if ((FUNC) != 0) { \
PyGILState_STATE gstate = PyGILState_Ensure(); \
PyErr_SetString(PyExc_OverflowError, \
"Overflow occurred in npy_datetimestruct_to_datetime"); \
PyGILState_Release(gstate); \
return -1; \
if ((EXPR) != 0) { \
PD_RAISE_FOR_OVERFLOW \
} \
} while (0)

Expand Down Expand Up @@ -150,55 +156,59 @@ npy_int64 get_datetimestruct_days(const npy_datetimestruct *dts) {
int i, month;
npy_int64 year, days = 0;
const int *month_lengths;
int did_overflow = 0;

year = dts->year - 1970;
days = year * 365;
did_overflow |= checked_int64_sub(dts->year, 1970, &year);
did_overflow |= checked_int64_mul(year, 365, &days);

/* Adjust for leap years */
if (days >= 0) {
/*
* 1968 is the closest leap year before 1970.
* Exclude the current year, so add 1.
*/
year += 1;
did_overflow |= checked_int64_add(year, 1, &year);
/* Add one day for each 4 years */
days += year / 4;
did_overflow |= checked_int64_add(days, year / 4, &days);
/* 1900 is the closest previous year divisible by 100 */
year += 68;
did_overflow |= checked_int64_add(year, 68, &year);
/* Subtract one day for each 100 years */
days -= year / 100;
did_overflow |= checked_int64_sub(days, year / 100, &days);
/* 1600 is the closest previous year divisible by 400 */
year += 300;
did_overflow |= checked_int64_add(year, 300, &year);
/* Add one day for each 400 years */
days += year / 400;
did_overflow |= checked_int64_add(days, year / 400, &days);
} else {
/*
* 1972 is the closest later year after 1970.
* Include the current year, so subtract 2.
*/
year -= 2;
did_overflow |= checked_int64_sub(year, 2, &year);
/* Subtract one day for each 4 years */
days += year / 4;
did_overflow |= checked_int64_add(days, year / 4, &days);
/* 2000 is the closest later year divisible by 100 */
year -= 28;
did_overflow |= checked_int64_sub(year, 28, &year);
/* Add one day for each 100 years */
days -= year / 100;
did_overflow |= checked_int64_add(days, year / 100, &days);
/* 2000 is also the closest later year divisible by 400 */
/* Subtract one day for each 400 years */
days += year / 400;
did_overflow |= checked_int64_add(days, year / 400, &days);
}

month_lengths = days_per_month_table[is_leapyear(dts->year)];
month = dts->month - 1;

/* Add the months */
for (i = 0; i < month; ++i) {
days += month_lengths[i];
did_overflow |= checked_int64_add(days, month_lengths[i], &days);
}

/* Add the days */
days += dts->day - 1;
did_overflow |= checked_int64_add(days, dts->day - 1, &days);

if (did_overflow) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also just PD_CHECK_OVERFLOW every expression but that is potentially a lot of branching. This allows more calls to build up in a pipeline and just raises at the very end if needed

PD_RAISE_FOR_OVERFLOW;
}
return days;
}

Expand Down