Skip to content

REF: use fused types in algos_rank_helper #29044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 18, 2019

Conversation

jbrockmendel
Copy link
Member

No description provided.

order = (values, mask)
if rank_t is not uint64_t:
if ascending ^ (na_option == 'top'):
if rank_t is object:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily in scope for this but would be nice if there was a better way to do this via fused types (a dict or inline func?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah I've been giving that some thought. I think we can do that with something like:

cdef inline rank_t get_thing(rank_t val):  # <-- totally ignore `val`, just care about rank_t
    if rank_t is object: ...

def actual_func(rank_t ...):
    cdef:
          rank_t val, thing
    
    thing = get_thing(val)

{{endif}}
if tiebreak == TIEBREAK_FIRST:
# need to use a stable sort here
_as = np.lexsort(keys=order)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe reading wrong but this is duplicated in each conditional branch? Could clean in follow up if so

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed

for i in range(n):
sum_ranks += i + 1
dups += 1

{{if dtype == 'object'}}
val = util.get_value_at(sorted_data, i)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this line no longer required?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT it hasnt been needed for a while. I recently removed most usages of util.get_value_at as unnecessary when we know i is an integer.

@WillAyd WillAyd added the Clean label Oct 16, 2019
@jbrockmendel jbrockmendel changed the title REF: use fused types in groupby_rank_helper REF: use fused types in algos_rank_helper Oct 17, 2019
@jreback jreback added this to the 1.0 milestone Oct 17, 2019
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. just some small requests, i suppose you have some followups to de-dupliacate

elif rank_t is int64_t:
nan_value = np.iinfo(np.int64).max

order = (values, mask)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add some comments here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you be more specific? there is a pretty nice comment up on L69

if rank_t is object:
nan_value = Infinity()
elif rank_t is float64_t:
nan_value = np.inf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments here as well

@jreback jreback merged commit 2701f52 into pandas-dev:master Oct 18, 2019
@jreback
Copy link
Contributor

jreback commented Oct 18, 2019

thanks

@jbrockmendel jbrockmendel deleted the fusemore3 branch October 18, 2019 22:19
HawkinsBA pushed a commit to HawkinsBA/pandas that referenced this pull request Oct 29, 2019
proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
bongolegend pushed a commit to bongolegend/pandas that referenced this pull request Jan 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants