Skip to content

TYP: numba stub #44233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 31, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion pandas/core/_numba/kernels/shared.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,13 @@
import numpy as np


@numba.jit(numba.boolean(numba.int64[:]), nopython=True, nogil=True, parallel=False)
@numba.jit(
# error: Any? not callable
numba.boolean(numba.int64[:]), # type: ignore[misc]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial stub does not include numba types so they resolve to Any (and not callable)

@mroeschke not sure of best practice, is it preferable to use the numba types instead of the string representation for production code?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either is recommended in the docs. Strings just have some parsing overhead looking at the code, but probably just fine to use the string representation if it makes typing here easier.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could evolve the stubs as needed so will leave as-is for now.

nopython=True,
nogil=True,
parallel=False,
)
def is_monotonic_increasing(bounds: np.ndarray) -> bool:
"""Check if int64 values are monotonically increasing."""
n = len(bounds)
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ markers = [

[tool.mypy]
# Import discovery
mypy_path = "stubs"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

microsoft/pyright#2514 (comment)

stubPath = "stubs" for pyright or rename the folder to typings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably also need to append "stubs" to include to make sure that pyright also checks stubs itself for any errors/suggestions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done for mypy. what do I need to change for pyright?

Copy link
Member

@twoertwein twoertwein Oct 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the following should work

include = ["pandas", "stubs"]
stubPath = "stubs"

You can probably also remove the pyright file-based ignore comments: pandas/core/window/numba_.py and pandas/core/util/numba_.py

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is setup correctly, pyright will complain about the incomplete __getattr__ (add a file-based ignore comment).

namespace_packages = false
explicit_package_bases = false
ignore_missing_imports = true
Expand Down
41 changes: 41 additions & 0 deletions stubs/numba.pyi
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
from typing import (
Any,
Callable,
Literal,
overload,
)

import numba

from pandas._typing import F

def __getattr__(name: str) -> Any: ... # incomplete
@overload
def jit(
signature_or_function: F = ...,
) -> F: ...
@overload
def jit(
signature_or_function: str
| list[str]
| numba.core.types.abstract.Type
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is added for completeness but will resolve to Any.

| list[numba.core.types.abstract.Type] = ...,
locals: dict = ..., # TODO: Mapping of local variable names to Numba types
cache: bool = ...,
pipeline_class: numba.compiler.CompilerBase = ...,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will resolve to Any

boundscheck: bool | None = ...,
*,
nopython: bool = ...,
forceobj: bool = ...,
looplift: bool = ...,
error_model: Literal["python", "numpy"] = ...,
inline: Literal["never", "always"] | Callable = ...,
# TODO: If a callable is provided it will be called with the call expression
# node that is requesting inlining, the caller's IR and callee's IR as
# arguments, it is expected to return Truthy as to whether to inline.
target: Literal["cpu", "gpu", "npyufunc", "cuda"] = ..., # deprecated
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could leave this out, we don't currently use this and since it is deprecated may want to not do so.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally we only use signature_or_function, nopython, nogil, parallel if that would simplify the typing. looplift may be utilized in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an open issue on numba tracker numba/numba#7424

The open PR to fix only deals with the decorator preserving types

I will also be using this stub in my numba branch and adding additional types as needed. Will look at the types for the numba types next if this PR is accepted.

It maybe that being more complete helps the community to share effort until official numba stubs are available.

nogil: bool = ...,
parallel: bool = ...,
) -> Callable[[F], F]: ...

njit = jit