Skip to content

CLN: remove unused parts of skiplist (most of it) #27465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 11, 2019

Conversation

jbrockmendel
Copy link
Member

As a follow-up I'll take a look at either removing the now-empty .pyx file or trying to port the c file to pyx

@WillAyd WillAyd added the Clean label Jul 19, 2019
Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - that's a lot of dead code!

@jbrockmendel
Copy link
Member Author

Yep, did a cython coverage run last night, found a bunch of unused code

@WillAyd WillAyd added this to the 0.26.0 milestone Jul 19, 2019
@jreback jreback modified the milestones: 0.26.0, 1.0 Jul 20, 2019
@jreback
Copy link
Contributor

jreback commented Jul 20, 2019

I think we would actually prefer this in cython for ease of understanding, but I understand if there is a perf diff. Can you do a quick check and see how much differ the c-impl is vs the cython (don't spend too much time); but if its close then might be worth using the cython rather than the c impl.

@jbrockmendel
Copy link
Member Author

@jreback not sure we're on the same page. AFAICT the code this removes isn't a cython imlementation of skiplist.h, it's just not used. Porting the c-impl to cython will be a separate task

@jreback
Copy link
Contributor

jreback commented Jul 20, 2019

@jreback not sure we're on the same page. AFAICT the code this removes isn't a cython imlementation of skiplist.h, it's just not used. Porting the c-impl to cython will be a separate task

note sure you read my comment. this IS a cython impl that is not used of course; the point is I would rather use a cython impl than the c one if the perf impact is not bad. Sure that would be a separate task; I asked to benchmark whether a simple use of this is a performance bottleneck.

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments

@jbrockmendel
Copy link
Member Author

I asked to benchmark whether a simple use of this is a performance bottleneck.

OK, what exactly would you want to benchmark? The code being removed here isn't a drop-in replacement for any C code, so there is nothing to benchmark.

I would rather use a cython impl than the c one if the perf impact is not bad

I agree, as mentioned in the OP.

note sure you read my comment.

The obvious wrongness of this upsets me. You can disagree without being disparaging.

@jreback
Copy link
Contributor

jreback commented Jul 20, 2019

OK, what exactly would you want to benchmark? The code being removed here isn't a drop-in replacement for any C code, so there is nothing to benchmark.

Sure there is, this code looks like its working, how about say inserting and getting some items. just something basic to see if these are like 10x difference or 2x difference.

I agree, as mentioned in the OP.

great, but then why are we removing this

The obvious wrongness of this upsets me. You can disagree without being disparaging.

well your response does not indicate that. I disagree, I certainly am not being disparaging in any way.

@jbrockmendel
Copy link
Member Author

@TomAugspurger or @jorisvandenbossche can one of you take over here? jeff and I are having trouble communicating

@jreback
Copy link
Contributor

jreback commented Jul 20, 2019

@TomAugspurger or @jorisvandenbossche can one of you take over here? jeff and I are having trouble communicating

I think you should be more clear.

@jbrockmendel
Copy link
Member Author

I think you should be more clear.

And I think that backs up my "trouble communicating" claim.

It appears to Jeff that I am not reading his comments. It appears to me that he is not reading mine. So I'm asking for a fresh pair of eyeballs, and in the interim there is no harm in putting this on the backburner.

@TomAugspurger
Copy link
Contributor

I can try to summarize

Jeff is requesting that we take places currently using skiplist.h and make it use the Cython implementation.

Brock is replying that the stuff he's deleting isn't related to code that's in skiplist.h. IOW, even if we wanted to remove skiplist.h and use the Cython code, we wouldn't be using what's being deleted in this PR.

Things devolve from there. I'm not going to wade into the "did or didn't read comments" stuff, other than to observe that comments like that are rarely helpful. Phrasing like "Did you see ?" are probably better.


The "AFAICT" qualifier in #27465 (comment) seems to be important. Jeff says otherwise "note sure you read my comment. this IS a cython impl that is not used of course"?

So, what's the relationship between the deleted code here and skiplist.h?

@WillAyd
Copy link
Member

WillAyd commented Aug 26, 2019

IIUC I don't have a problem merging this as is and leaving as a separate exercise to port the header file over to Cython. Of course we would probably be reimplementing these methods in that case anyway, but I could see it being less confusing as to how this works in the interim if this code isn't around (i.e. dead code can only be confusing)

@jreback
Copy link
Contributor

jreback commented Sep 8, 2019

ok, re-reading, this is what I would do.

ok on this PR to remove this dead code, but I would add a comment to a link to the cython impl (put in a link to a commit before this), so that if/when we need to port to cython we already have a reference impl.

sound good?

@jbrockmendel
Copy link
Member Author

sure

@jreback
Copy link
Contributor

jreback commented Sep 8, 2019

lgtm. ping on green.

@jbrockmendel
Copy link
Member Author

ping

@jbrockmendel
Copy link
Member Author

@WillAyd since you're on a roll

@WillAyd WillAyd merged commit c82f57d into pandas-dev:master Sep 11, 2019
@WillAyd
Copy link
Member

WillAyd commented Sep 11, 2019

Yep thanks @jbrockmendel

@jbrockmendel jbrockmendel deleted the skiplist branch September 11, 2019 02:10
@@ -5,144 +5,3 @@
# Link: http://code.activestate.com/recipes/576930/

# Cython version: Wes McKinney
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason that we need this file (now without any code) ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we'd need to update setup.py

@WillAyd WillAyd mentioned this pull request Dec 13, 2019
proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants