Skip to content

Add content on performance benchmarks. #458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Add content on performance benchmarks. #458

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
375be0f
Create benchmark.md
khushi-411 Jul 30, 2021
d23fd31
Add files via upload
khushi-411 Aug 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions content/en/benchmark.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# NumPy Benchmarking


![output-modify-2-new](https://user-images.githubusercontent.com/62256509/127653679-5151e22d-d141-4621-abb9-b3e6dd7988b7.jpg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to commit the figure in this PR. The place to put it is static/images/content_images/



## Implemented Algorithms

Till this point 5 algorithms are implemented as mentioned bellow:

### NumPy

NumPy is a most fundamental package for scientific computing in Python. The major advantage of using NumPy is these gives same computational speed as C and Fortran. Let's see our inplementation details:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking what should come next is indeed a comparison with C/C++ and Fortran, like this sentence says?

The accelerators can be a second part of the story.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text should explain that this is a numerical simulation of the N-body problem and giv ethe constraints of the problem

C/C++ and Fortran code simulations can be found in the original repo https://github.com/paugier/nbabel/


#### NumPy Python

Here, we used optimised code which consists of both type of implementation i.e. via Python and NumPy. This algorithm took 10.15 seconds for datasets with 16 points, 45.2 seconds for datasets with 32 points and 161.01 seconds for datasets with 64 points.

#### Pure NumPy

This algorithm is purely implemented via NumPy library functions. It takes 10.56 seconds to run a program with datasets of 16 points, 41.59 seconds for datasets with 32 points and 169.4 seconds for datasets with 64 points.

### Transonic

Transonic is a pure python package. It is one of the actively growing libraries which is used to increase the speed of our codes. Here, we are operating with two kinds of accelerators in the backend jit and boost to accelerate our code. Transonic uses four different types of accelerators in the backend Pythran, Cython, Numba, and Python. Presently, we have implemented Jit and boost accelerators, in the next iteration, we will be using Cython, with different types of backends.

#### Transonic - JIT

We used Transonic jit to accelerate our code. To implement this we used Pythran (Ahead-Of-Time) compiler in Just-In-Time mode. This algorithm took 0.01 seconds for input datasets with 16 points, 0.05 seconds for input datasets with 32 points and 0.17 seconds for input datasets with 64 points.

#### Transonic - Boost: Pythran

Pythran is an Ahead Of Time compiler. This module takes a Python file as an input then converts it into some kind of interface which makes the code faster. Transonic's boost decorator replaces the python function with the pythranized function. This algorithm took 10.37 seconds for input data of 16 points, 41.88 seconds for input data with 32 points and 160.15 seconds for input data with 64 points.

### Numba

Numba converts the Python files to optimized machine code during runtime. The major advantage is after compiling codes using Numba, the speed of code becomes approachable to C or FORTRAN. Numba caches all the compiled functions into a file system for future use of the same functions. We used the jit decorated functions to call another jit decorated function while implementing our code. It is advisable to add the jit decorated function to other functions too, so as to prevent it from generating slower outputs. This algorithm took 0.89 seconds for input data with 16 points, 0.05 seconds for input data with 32 points and 0.17 seconds for input data with 64 points.

## Results

Table values represent the time taken by each algorithm to run, in respected datasets for 5 iterations.

<html>
<table>
<tr>
<td></td>
<td><b>Input 16</b></td>
<td><b>Input 32</b></td>
<td><b>Input 64</b></td>
</tr>
<tr>
<td><b>Pythran</b></td>
<td>10.37</td>
<td>41.88</td>
<td>160.15</td>
</td>
<tr>
<td><b>Numba</b></td>
<td>0.89</td>
<td>0.05</td>
<td>o.17</td>
</tr>
<tr>
<td><b>Pythran Naive</b></td>
<td>0.01</td>
<td>0.05</td>
<td>0.17</td>
</tr>
<tr>
<td><b>NumPy</b></td>
<td>10.15</td>
<td>45.2</td>
<td>161.01</td>
</tr>
<tr>
<td><b>Pure NumPy</b></td>
<td>10.56</td>
<td>41.59</td>
<td>169.4</td>
</tr>
</table>
</html>

## Code Folders

* These codes are originally taken from [here](https://github.com/paugier/nbabel)
* Benchmark code: [here](https://github.com/khushi-411/numpy-benchmarks/blob/0.3/python/benchmark-2.py)
* Graph: [here](https://github.com/khushi-411/numpy-benchmarks/blob/0.3/python/plot-modified-1.py)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linking to your own repo is fine for now. We can make it a separate NumPy repo or put the scripts in an existing repo somewhere, when you are closer to done with this PR.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure definitely, these are just sample codes. I feel that these need to be improved, and once it's finalized, we can create a repo or update somewhere in the existing repo as you'll wish. Thanks!


<html>
<table>
<tr>
<td><b>Algorithms</b></td>
<td><b>Source Codes</b></td>
</tr>
<tr>
<td>Pythran</td>
<td> <a href = "https://github.com/khushi-411/numpy-benchmarks/blob/0.3/python/bench.py">bench.py</a></td>
</tr>
<tr>
<td>Numba</td>
<td> <a href = "https://github.com/khushi-411/numpy-benchmarks/blob/0.3/python/bench_numba.py">bench_numba.py</a></td>
</tr>
<tr>
<td>Pythran Naive</td>
<td><a href = "https://github.com/khushi-411/numpy-benchmarks/blob/0.3/python/bench_numpy_highlevel_jit.py">bench_numpy_highlevel_jit.py</a></td>
</tr>
<tr>
<td>NumPy</td>
<td><a href = "https://github.com/khushi-411/numpy-benchmarks/blob/0.3/python/bench_numpy_highlevel.py">bench_numpy_highlevel.py</a></td>
</tr>
<tr>
<td>Pure NumPy</td>
<td><a href = "https://github.com/khushi-411/numpy-benchmarks/blob/0.3/python/bench_numpy_highlevel.py">bench_pure_numpy_highlevel.py</a></td>
</tr>
</table>
</html>


## Conclusion

* These benchmarks are run with an Intel Core i7 processor.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably belongs in the caption of the graph. With a bit more details about the exact CPU. For example:

$ cat /proc/cpuinfo | grep Intel

And then slightly reformat like: Intel i9-7920X @ 2.90GHz

Copy link
Author

@khushi-411 khushi-411 Aug 1, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologizes, I will update that. Thanks!

* We found that codes that use Numba and Pythran Naive as accelerators give the best results among all.

## Acknowledgement

I would like to thank [Matti Picus](https://github.com/mattip) for helping me and guiding me throughout these days. Thank you so much!

Binary file added static/images/content_images/output-modify-2b-new.jpg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.