-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
Simplify Timedelta init, standardize overflow errors #47004
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
14f5bdf
to
90a7945
Compare
Hello @patrickmckenna! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found: There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻 Comment last updated at 2022-05-24 20:22:07 UTC |
90a7945
to
25c3412
Compare
25c3412
to
e35f7be
Compare
I'm a little confused by the errors in the "Minimum Versions" check. (The others errors are also confuse me, but appear to be issues in the CI setup phases.) The last commit resulted in one less test failure: Thoughts/feedback here appreciated. |
eda556c
to
975752c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor comments. any performance impact here?
oh you put that in the top post cool. then just minor comments. |
out_value = parse_timedelta_string(value) | ||
else: | ||
out_value = convert_to_timedelta64(value, in_unit).view(np.int64) | ||
except OverflowError as err: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
any way for less of this logic to be within the try/except?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree, it's not ideal.
Initially, I tried to leave as much conditional processing as possible in convert_to_timedelta64
, i.e. no if/else
here, but that led to considerable slowdowns in many benchmarks IIRC. So that would be a simpler, albeit slower, option.
A similar option would be to let all the dispatching on arg type happen automatically, and add overflow checking as needed within each variant, e.g.
@functools.singledispatch
def value_to_int(value):
...
@value_to_int.register(str)
def _(value: str):
...
def create_timedelta(value):
...
try:
out_value = value_to_int(value)
catch OverflowError:
raise OutOfBoundsTimedelta(msg) from err
...
I'm not sure if cython plays well with singledispatch
? This approach would also expand the scope of this PR a lot.
Definitely open to other ideas though!
pandas/tests/scalar/conftest.py
Outdated
|
||
@pytest.fixture(name="td_overflow_msg") | ||
def fixture_td_overflow_msg() -> str: | ||
return R"outside allowed range \[-9223372036854775807ns, 9223372036854775807ns\]" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lower case r, also pretty sure this can just be a constant instead of a fixture
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can switch 👍 (Since this repo includes a devcontainer.json
, I thought maybe use R
for VS Code (and GitHub) compatibility.)
367372e
to
51274d2
Compare
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen. |
doc/source/whatsnew/vX.X.X.rst
file if fixing a bug or adding a new feature.Per #46936 (comment), this PR extracts the functional updates in #46936, to simplify review. The main motivations are those explained in #46936 (comment):
As a bonus, this appears to speed things up considerably, especially in the case of passing
int
s (by avoiding the unnecessaryint -> td64[ns] -> int
conversion):I've left out all the test suite updates from #46936, save for those changes associated with
s/OverflowError/OutOfBoundsTimedelta
./cc @jbrockmendel @jreback