Skip to content

Narrow is with final types correctly #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Conversation

arvi18
Copy link

@arvi18 arvi18 commented Apr 24, 2025

Now we can narrow @final types with is and is not.

Closes python#15553

Summary by CodeRabbit

  • New Features

    • Improved type narrowing for final classes and type identity checks, enabling more precise type inference in relevant scenarios.
  • Bug Fixes

    • Enhanced handling of type coverage checks involving special type cases and singleton types for more accurate runtime behavior.
  • Tests

    • Added new test cases to verify correct narrowing behavior with final and non-final classes, as well as interactions with the Any type.

@arvi18
Copy link
Author

arvi18 commented Apr 24, 2025

Diff from mypy_primer, showing the effect of this PR on open source code:

koda-validate (https://github.com/keithasaurus/koda-validate)
+ koda_validate/typehints.py:170: error: Statement is unreachable  [unreachable]
+ koda_validate/serialization/json_schema.py:85: error: Statement is unreachable  [unreachable]
+ koda_validate/serialization/json_schema.py:377: error: Statement is unreachable  [unreachable]

@arvi18
Copy link
Author

arvi18 commented Apr 24, 2025

Problems looks real, checking.

@arvi18
Copy link
Author

arvi18 commented Apr 24, 2025

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@arvi18
Copy link
Author

arvi18 commented Apr 24, 2025

Diff from mypy_primer, showing the effect of this PR on open source code:

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_local.py:54: error: Unused "type: ignore" comment  [unused-ignore]
+ src/trio/_core/_local.py:57: error: Unused "type: ignore" comment  [unused-ignore]

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/ext/commands/converters.py:532: error: Unused "type: ignore" comment  [unused-ignore]

Copy link

coderabbitai bot commented Apr 24, 2025

"""

Walkthrough

This update introduces enhancements to type narrowing logic, particularly for final classes and type objects, within the type checking system. Internal logic in the covers_at_runtime function is modified to handle early exits for certain AnyType and singleton cases. The CallableType and TypeType classes receive new is_singleton_type methods to identify singleton types based on their internal characteristics. Additionally, new test cases are added to verify correct type narrowing behavior when using identity checks with final and non-final types, as well as interactions with Any.

Changes

File(s) Change Summary
mypy/subtypes.py Updated covers_at_runtime to add early exit conditions for AnyType, TypeType wrapping AnyType, and singleton types.
mypy/types.py Added is_singleton_type methods to CallableType and TypeType classes to identify singleton types.
test-data/unit/check-narrowing.test Added three new test cases covering narrowing with final classes, Any, and non-final types using identity checks.

Sequence Diagram(s)

sequenceDiagram
    participant TypeChecker
    participant covers_at_runtime
    participant CallableType
    participant TypeType

    TypeChecker->>covers_at_runtime: covers_at_runtime(item, supertype)
    covers_at_runtime->>covers_at_runtime: Check if item is AnyType or TypeType[AnyType]
    covers_at_runtime-->>TypeChecker: Return False if so
    covers_at_runtime->>CallableType: item.is_singleton_type()
    covers_at_runtime->>TypeType: item.is_singleton_type()
    covers_at_runtime-->>TypeChecker: Return True if singleton and subtype
    covers_at_runtime->>covers_at_runtime: Continue with existing logic if not early exit
Loading

Poem

In the warren of types, where rabbits do leap,
Singleton secrets are no longer deep.
With finals and Any, we now check with care,
Identity narrows—no bugs to snare!
Test cases sprout up like carrots anew,
Hopping through logic, our code shines true.
🐇✨
"""


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 380cb8d and 2372c12.

📒 Files selected for processing (3)
  • mypy/subtypes.py (1 hunks)
  • mypy/types.py (2 hunks)
  • test-data/unit/check-narrowing.test (1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
mypy/subtypes.py (2)
mypy/types.py (9)
  • AnyType (1122-1208)
  • TypeType (3048-3128)
  • CallableType (1814-2320)
  • is_singleton_type (290-291)
  • is_singleton_type (1288-1289)
  • is_singleton_type (1557-1564)
  • is_singleton_type (2170-2171)
  • is_singleton_type (2890-2891)
  • is_singleton_type (3117-3120)
mypy/typeops.py (1)
  • is_singleton_type (916-933)
⏰ Context from checks skipped due to timeout of 90000ms (13)
  • GitHub Check: Test suite with py39-ubuntu, mypyc-compiled
  • GitHub Check: Test suite with py311-ubuntu, mypyc-compiled
  • GitHub Check: mypyc runtime tests with py39-macos
  • GitHub Check: Test suite with py312-ubuntu, mypyc-compiled
  • GitHub Check: Test suite with py313-ubuntu, mypyc-compiled
  • GitHub Check: Test suite with py310-ubuntu
  • GitHub Check: Test suite with py39-windows-64
  • GitHub Check: Test mypyc suite with 32-bit Python
  • GitHub Check: Run mypy_primer (2)
  • GitHub Check: Run mypy_primer (4)
  • GitHub Check: Run mypy_primer (1)
  • GitHub Check: Run mypy_primer (3)
  • GitHub Check: Run mypy_primer (0)
🔇 Additional comments (7)
mypy/types.py (2)

2170-2172: Implementation of singleton type detection for CallableType looks good.

This new method correctly identifies callables that represent final type objects as singletons, which will enable proper type narrowing when using is and is not operators with final classes.


3117-3121: Appropriate singleton type identification for TypeType.

This implementation correctly identifies when a TypeType should be considered a singleton, handling both final class instances and NoneType. This change is essential for enabling the type checker to narrow types marked with @final during identity comparisons.

mypy/subtypes.py (2)

1962-1966: Add early check to prevent Any from being considered covered at runtime

This change ensures that variables of type Any or Type[Any] are never narrowed by identity checks, which is the correct behavior since Any represents a type that could be any value at runtime.


1968-1971: Enable precise narrowing for singleton types

This is the core improvement that enables identity comparisons to correctly narrow final types. When an item is a singleton type (like a final class or None), we can reliably narrow it based on the identity check.

The implementation correctly leverages the new is_singleton_type() methods added to CallableType and TypeType classes, enabling more precise type inference when using identity checks with final classes.

test-data/unit/check-narrowing.test (3)

1377-1430: Test narrowing with identity checks on final types

Excellent test case that verifies the core functionality of narrowing types using identity checks (is and is not) with final classes. The test confirms both positive and negative paths work correctly.

This test validates that when comparing against final types like Mark, the type checker correctly narrows the union to just that type in the positive branch and removes it from the union in the negative branch.


1433-1451: Test narrowing with Any against final types

Good test case that verifies how Any and Type[Any] interact with identity checks against final types. This ensures the implementation correctly handles the special case of Any types.

The test correctly verifies that narrowing works differently for Any versus Type[Any] when compared with identity checks against final types.


1455-1472: Test non-narrowing for non-final types

This test case confirms the important behavior that identity checks against non-final types do not narrow the type union. This is the expected behavior since non-final types can have multiple runtime instances.

This test validates that the type system doesn't incorrectly narrow types when identity checks are performed against non-final types, which provides a good contrast to the final type case.

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@visz11
Copy link
Collaborator

visz11 commented Apr 24, 2025

@coderabbitai full review

Copy link

coderabbitai bot commented Apr 24, 2025

✅ Actions performed

Full review triggered.

Copy link

Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/raises.py:1490: error: Unused "type: ignore" comment  [unused-ignore]

steam.py (https://github.com/Gobot1234/steam.py)
+ steam/ext/commands/converters.py:532: error: Unused "type: ignore" comment  [unused-ignore]

trio (https://github.com/python-trio/trio)
+ src/trio/_core/_local.py:54: error: Unused "type: ignore" comment  [unused-ignore]
+ src/trio/_core/_local.py:57: error: Unused "type: ignore" comment  [unused-ignore]

@visz11
Copy link
Collaborator

visz11 commented Apr 24, 2025

/gemini review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces significant improvements to type narrowing for final types, which enhances the precision of type inference. The addition of new test cases is commendable, as it demonstrates a commitment to verifying the correctness of the changes. However, a thorough review of the performance implications and comprehensive coverage of edge cases is essential to ensure the robustness and efficiency of the new features.

Summary of Findings

  • Type coverage checks: The changes introduce improved type narrowing for final classes and type identity checks, but it's important to ensure that all possible type combinations are thoroughly covered in the test cases. Consider adding more complex scenarios to verify the robustness of the new features.
  • Runtime type checks: The code includes logic for runtime type checks, but it's crucial to ensure that these checks are efficient and don't introduce performance bottlenecks. Review the performance implications of the added checks, especially in frequently executed code paths.

Merge Readiness

The pull request is approaching readiness for merging, but there are a few issues that need to be addressed before it can be considered fully ready. Specifically, the high severity issue regarding the potential for runtime errors due to incomplete type coverage must be resolved. Additionally, the medium severity issue concerning the efficiency of runtime type checks should be investigated and optimized if necessary. Once these issues have been addressed and verified, the pull request can be confidently merged. I am unable to approve this pull request, and users should have others review and approve this code before merging.

Comment on lines +1962 to +1966
# Left `Any` or `Type[Any]` type will never be covered at runtime:
if isinstance(item, AnyType) or (
isinstance(item, TypeType) and isinstance(item.item, AnyType)
):
return False

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This block prevents Any or Type[Any] from being covered at runtime. Is this the desired behavior? Should there be a comment explaining why this is necessary?

Suggested change
# Left `Any` or `Type[Any]` type will never be covered at runtime:
if isinstance(item, AnyType) or (
isinstance(item, TypeType) and isinstance(item.item, AnyType)
):
return False
# Left `Any` or `Type[Any]` type will never be covered at runtime:
# This is because runtime type checks will not be able to infer
# the underlying type of `Any` or `Type[Any]`.
if isinstance(item, AnyType) or (
isinstance(item, TypeType) and isinstance(item.item, AnyType)
):

Comment on lines +1968 to +1970
if isinstance(item, (CallableType, TypeType)) and item.is_singleton_type():
if is_proper_subtype(item, supertype):
return True

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This condition checks if the item is a singleton type and a proper subtype of the supertype. Can you add a comment explaining what a singleton type is in this context, and why this check is needed?

Suggested change
if isinstance(item, (CallableType, TypeType)) and item.is_singleton_type():
if is_proper_subtype(item, supertype):
return True
# Check if the item is a singleton type (e.g., `None`, final classes)
# and a proper subtype of the supertype.
if isinstance(item, (CallableType, TypeType)) and item.is_singleton_type():
if is_proper_subtype(item, supertype):

Comment on lines +3117 to +3120
def is_singleton_type(self) -> bool:
return (isinstance(self.item, Instance) and self.item.type.is_final) or isinstance(
self.item, NoneType
)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This function determines if a TypeType is a singleton type. Why is NoneType considered a singleton type here?

Suggested change
def is_singleton_type(self) -> bool:
return (isinstance(self.item, Instance) and self.item.type.is_final) or isinstance(
self.item, NoneType
)
def is_singleton_type(self) -> bool:
# A TypeType is a singleton type if its item is a final instance or None.
return (isinstance(self.item, Instance) and self.item.type.is_final) or isinstance(
self.item, NoneType
)

Comment on lines +1441 to +1444
if x is Mark:
reveal_type(x) # N: Revealed type is "def () -> __main__.Mark"
else:
reveal_type(x) # N: Revealed type is "Any"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The revealed type here is def () -> __main__.Mark. Is this the intended behavior when x is Any and compared to a final class? Should it be Mark instead?

if x is Mark:
    reveal_type(x)  # N: Revealed type is "__main__.Mark"
else:
    reveal_type(x)  # N: Revealed type is "Any"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Narrow x is Sentinel where x is Union[type[Sentinel], ...] and Sentinel is @final
4 participants