Skip to content

CLN: allow label memviews to be None #42260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Jul 1, 2021

Conversation

mzeitlin11
Copy link
Member

Saves some unnecessary memory allocations, plus moving towards making rank_sorted_1d easier to call. Also have done something similar for allowing mask to be None, but split that off this pr to keep it simpler.

@mzeitlin11 mzeitlin11 added the Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff label Jun 26, 2021
rankedy = rank_1d(np.array(maskedy)[:nobs],
labels=labels_nobs)
rankedx = rank_1d(np.array(maskedx)[:nobs])
rankedy = rank_1d(np.array(maskedy)[:nobs])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if it makes a difference, but np.array -> np.asarray?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, was an oversight no reason to copy here

const intp_t[:] sort_indexer,
# Can make const with cython3 (https://github.com/cython/cython/issues/3222)
rank_t[:] masked_vals,
const uint8_t[:] mask,
TiebreakEnumType tiebreak,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as long as you're shifting these around, can you make everything from here-ish on keyword-only

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think cdef functions support keyword-only enforced by "*" (unless you have a workaround, I just tried and saw a cython compile error :)

Changed callers to treat it as keyword-only from after mask, though, which should help

@jbrockmendel
Copy link
Member

Small comments, generally looks very nice

@jreback jreback added this to the 1.4 milestone Jul 1, 2021
@jreback jreback merged commit 1d4209d into pandas-dev:master Jul 1, 2021
@jreback
Copy link
Contributor

jreback commented Jul 1, 2021

thanks @mzeitlin11

@mzeitlin11 mzeitlin11 deleted the ref/memviews_none_labels branch July 1, 2021 23:45
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Clean
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants