Using LGTM.com to find and fix issues in Python and C code

[AlexTereshenkov]

There are a few issues in the code that were flagged up by LGTM.com website: https://lgtm.com/projects/g/pandas-dev/pandas/alerts/?mode=tree. Some of the other numerical computing repositories have been analyzed there as well such as numpy and scipy.

You can see the issues in both Python and C code. For instance, here is the issue Implicit scaling of pointer arithmetic expressions can cause buffer overflow conditions that is found in SciPy source C code. Some of the issues may be of higher importance to the project (e.g. Allocating memory with a size controlled by an external user can result in integer overflow vs A pure expression whose value is ignored is likely to be the result of a typo.). It is completely up to the developers to pick what's relevant.

Due to the dynamic nature of Python and some heavy use of all kinds of magic in numpy (e.g. Comparison of identical values, the intent of which is unclear and scipy (e.g. Using a named argument whose name does not correspond to a parameter of the init method of the class being instantiated, will result in a TypeError at runtime, some of the alerts may look like false positives and I would love to find out which of them are to be able to fix the code to avoid raising them.

If you like, you can use LGTM for automatically reviewing code in pull requests. Here's an example of how Google's AMPHTML use that to flag up security vulnerabilities in their code base: ampproject/amphtml#13060. This will prevent introducing new issues as you can see if there are any issues introduced before merging.

(full disclosure: I'm a huge fan of pandas and also part of the team that runs LGTM.com)

[mroeschke]

Duplicate of #20589. We would probably be open to integrate LGTM in our CI.

[AlexTereshenkov]

@mroeschke Cool. To get help setting this up, review https://lgtm.com/help/lgtm/github-apps-integration and for step-by-step guide see https://lgtm.com/help/lgtm/github-apps-integration. Ping me any time if you need help with this!