Skip to content

CLN: _try_coerce_result, redundant dtypes.missing method #24619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 5, 2019

Conversation

jbrockmendel
Copy link
Member

Started this branch with the idea of making DatetimeBlock, TimedeltaBlock, DatetimeTZBlock define _try_coerce_result using self._holder._from_sequence (which I still think is worthwhile, cc @TomAugspurger ) and got side-tracked. This ends up just being some cleanup and removal of a redundant method.

Tiny perf bump expected from optimizing NaT checks in _libs.missing. Further improvements/cleanups may be possible pending #24607.

The object-dtype cases in try_coerce_result are no longer hit following #24606.

@@ -2250,6 +2244,8 @@ def __init__(self, values, placement, ndim=2, dtype=None):
# and just use DatetimeBlock's.
if dtype is not None:
values = self._maybe_coerce_values(values, dtype=dtype)
# TODO: this gets hit in msgpack tests, but in all cases the values
# are already DatetimeArrays with the appropriate dtype
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment should be removed before merging, but if someone well-informed says its OK we may be able to get rid of this check and get some further simplification.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where does this get hit?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the tests the only place this comes up is in test_packers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can certainly do this at a higher level. in packers you know the block type & have access to the _holder; can just construct it and pass fully formed values, then don't need to pass the dtype thru

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah thats what Im thinking now, and it may actually make some of the other 24024 follow-ups easier. Anyway, just removed the comment.

@jreback jreback added Datetime Datetime data dtype Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Clean labels Jan 4, 2019
@jreback jreback added this to the 0.24.0 milestone Jan 4, 2019
@codecov
Copy link

codecov bot commented Jan 4, 2019

Codecov Report

Merging #24619 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #24619      +/-   ##
==========================================
+ Coverage   92.37%   92.38%   +<.01%     
==========================================
  Files         166      166              
  Lines       52396    52377      -19     
==========================================
- Hits        48403    48388      -15     
+ Misses       3993     3989       -4
Flag Coverage Δ
#multiple 90.8% <100%> (ø) ⬆️
#single 43% <60%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/dtypes/missing.py 92.77% <ø> (-0.42%) ⬇️
pandas/core/internals/blocks.py 94.38% <100%> (+0.16%) ⬆️
pandas/_libs/tslibs/__init__.py 100% <100%> (ø) ⬆️
pandas/core/sparse/series.py 95.53% <100%> (ø) ⬆️
pandas/util/testing.py 88.09% <0%> (+0.09%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19f715c...2577c9a. Read the comment docs.

@codecov
Copy link

codecov bot commented Jan 4, 2019

Codecov Report

Merging #24619 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #24619      +/-   ##
==========================================
+ Coverage   92.37%   92.38%   +<.01%     
==========================================
  Files         166      166              
  Lines       52396    52376      -20     
==========================================
- Hits        48403    48386      -17     
+ Misses       3993     3990       -3
Flag Coverage Δ
#multiple 90.8% <100%> (ø) ⬆️
#single 43% <60%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/dtypes/missing.py 92.77% <ø> (-0.42%) ⬇️
pandas/core/internals/blocks.py 94.38% <100%> (+0.16%) ⬆️
pandas/_libs/tslibs/__init__.py 100% <100%> (ø) ⬆️
pandas/core/sparse/series.py 95.53% <100%> (ø) ⬆️
pandas/core/indexes/period.py 92.06% <0%> (-0.06%) ⬇️
pandas/core/resample.py 97.23% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19f715c...e1033e1. Read the comment docs.

@jreback
Copy link
Contributor

jreback commented Jan 5, 2019

this overlaps with #24607 ?

@jreback jreback merged commit 05a9279 into pandas-dev:master Jan 5, 2019
@jreback
Copy link
Contributor

jreback commented Jan 5, 2019

thanks!

@jbrockmendel jbrockmendel deleted the coerce branch January 5, 2019 15:40
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Datetime Datetime data dtype Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants