Skip to content

add numba example to enhancingperf.rst #10257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 3, 2015
Merged

Conversation

jreback
Copy link
Contributor

@jreback jreback commented Jun 3, 2015

enhancing performance pandas 0 16 1-81-g676cb95 documentation 1

@jreback jreback added Docs Performance Memory or execution speed performance labels Jun 3, 2015
@jreback jreback added this to the 0.16.2 milestone Jun 3, 2015
@jorisvandenbossche
Copy link
Member

+1 !

With the latest numba, I think you can do the empty array allocation inside the jitted function, instead of passing it as an argument. I find this a bit easier to follow, but of course in this way it also works with older versions.


Read more in the `cython docs <http://docs.cython.org/>`__.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you leave this link at the end of the cython section?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

links already at the top of the section

@jreback
Copy link
Contributor Author

jreback commented Jun 3, 2015

@jorisvandenbossche for some reason, sphinx wouldn't render the decorators in the ipython blocks (e.g. the @jit. very odd. (it would work with verbatim, but then you have to repeat the entire code in a suppress block to actually have it executed).

def compute_numba(df):
result = np.empty(len(df), dtype='float64')
apply_integrate_f_numba(df['a'].values,df['b'].values,df['N'].values,result)
return Series(result,index=df.index,name='result')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small remark: PEP8, spaces after comma's

@jreback
Copy link
Contributor Author

jreback commented Jun 3, 2015

@jorisvandenbossche I put the allocation back. I think its ok. In the next version of numba, this will just be 'faster' (e.g. its all done in nopython mode). But seems to work ok.


@numba.jit
def apply_integrate_f_numba(col_a, col_b, col_N):
result = np.empty(len(df), dtype='float64')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

df is not defined here, can use n probably

@jreback jreback force-pushed the numba branch 2 times, most recently from b6f5e6a to 412e375 Compare June 3, 2015 12:32
@seibert
Copy link

seibert commented Jun 3, 2015

Note that the version of Numba that will be able to do the memory allocation in nopython mode will be coming out today.

@jreback
Copy link
Contributor Author

jreback commented Jun 3, 2015

@seibert hmm, this is seeming to execute in nopython mode already though? (I guess except for the allocation, which prob doesn't matter in perf as its a single event), right?

@jorisvandenbossche
Copy link
Member

It does not execute in nopython mode with current stable version of numba, but because of the loop lifting of the main part of the function, it does not make that much of a difference I think.

@seibert yes, I was referring to that version, but forgot that it was only a beta release announcement that I saw

@seibert
Copy link

seibert commented Jun 3, 2015

Right, loop-lifting means that allocating memory in nopython mode is not as exciting as you might initially think. :)

@shoyer
Copy link
Member

shoyer commented Jun 3, 2015

@jreback Could you replace the cython timings with a :verbatim:, too? Otherwise the numbers will get out of sync.

Generally this looks good to me

jreback added a commit that referenced this pull request Jun 3, 2015
add numba example to enhancingperf.rst
@jreback jreback merged commit 38a95d1 into pandas-dev:master Jun 3, 2015
@jreback
Copy link
Contributor Author

jreback commented Jun 3, 2015

@shoyer updated to use all verbatims..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Docs Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants