Introduce InferContext (#14956)

## Summary

I'm currently on the fence about landing the #14760 PR because it's
unclear how we'd support tracking used and unused suppression comments
in a performant way:
* Salsa adds an "untracked" dependency to every query reading
accumulated values. This has the effect that the query re-runs on every
revision. For example, a possible future query
`unused_suppression_comments(db, file)` would re-run on every
incremental change and for every file. I don't expect the operation
itself to be expensive, but it all adds up in a project with 100k+ files
* Salsa collects the accumulated values by traversing the entire query
dependency graph. It can skip over sub-graphs if it is known that they
contain no accumulated values. This makes accumulators a great tool for
when they are rare; diagnostics are a good example. Unfortunately,
suppressions are more common, and they often appear in many different
files, making the "skip over subgraphs" optimization less effective.

Because of that, I want to wait to adopt salsa accumulators for type
check diagnostics (we could start using them for other diagnostics)
until we have very specific reasons that justify regressing incremental
check performance.

This PR does a "small" refactor that brings us closer to what I have in
#14760 but without using accumulators. To emit a diagnostic, a method
needs:

* Access to the db
* Access to the currently checked file

This PR introduces a new `InferContext` that holds on to the db, the
current file, and the reported diagnostics. It replaces the
`TypeCheckDiagnosticsBuilder`. We pass the `InferContext` instead of the
`db` to methods that *might* emit diagnostics. This simplifies some of
the `Outcome` methods, which can now be called with a context instead of
a `db` and the diagnostics builder. Having the `db` and the file on a
single type like this would also be useful when using accumulators.

This PR doesn't solve the issue that the `Outcome` types feel somewhat
complicated nor that it can be annoying when you need to report a
`Diagnostic,` but you don't have access to an `InferContext` (or the
file). However, I also believe that accumulators won't solve these
problems because:

* Even with accumulators, it's necessary to have a reference to the file
that's being checked. The struggle would be to get a reference to that
file rather than getting a reference to `InferContext`.
* Users of the `HasTy` trait (e.g., a linter) don't want to bother
getting the `File` when calling `Type::return_ty` because they aren't
interested in the created diagnostics. They just want to know what
calling the current expression would return (and if it even is a
callable). This is what the different methods of `Outcome` enable today.
I can ask for the return type without needing extra data that's only
relevant for emitting a diagnostic.

A shortcoming of this approach is that it is now a bit confusing when to
pass `db` and when an `InferContext`. An option is that we'd make the
`file` on `InferContext` optional (it won't collect any diagnostics if
`None`) and change all methods on `Type` to take `InferContext` as the
first argument instead of a `db`. I'm interested in your opinion on
this.

Accumulators are definitely harder to use incorrectly because they
remove the need to merge the diagnostics explicitly and there's no risk
that we accidentally merge the diagnostics twice, resulting in
duplicated diagnostics. I still value performance more over making our
life slightly easier.

Author: MichaReiser

Cargo.lock

@@ -26,6 +26,7 @@ bitflags = { workspace = true }
 camino = { workspace = true }
 compact_str = { workspace = true }
 countme = { workspace = true }
+drop_bomb = { workspace = true }
 indexmap = { workspace = true }
 itertools = { workspace = true }
 ordermap = { workspace = true }
@@ -58,4 +59,3 @@ serde = ["ruff_db/serde", "dep:serde"]
 
 [lints]
 workspace = true
-

crates/red_knot_python_semantic/Cargo.toml

@@ -0,0 +1,50 @@
+use salsa;
+
+use ruff_db::{files::File, parsed::comment_ranges, source::source_text};
+use ruff_index::{newtype_index, IndexVec};
+
+use crate::{lint::LintId, Db};
+
+#[salsa::tracked(return_ref)]
+pub(crate) fn suppressions(db: &dyn Db, file: File) -> IndexVec<SuppressionIndex, Suppression> {
+    let comments = comment_ranges(db.upcast(), file);
+    let source = source_text(db.upcast(), file);
+
+    let mut suppressions = IndexVec::default();
+
+    for range in comments {
+        let text = &source[range];
+
+        if text.starts_with("# type: ignore") {
+            suppressions.push(Suppression {
+                target: None,
+                kind: SuppressionKind::TypeIgnore,
+            });
+        } else if text.starts_with("# knot: ignore") {
+            suppressions.push(Suppression {
+                target: None,
+                kind: SuppressionKind::KnotIgnore,
+            });
+        }
+    }
+
+    suppressions
+}
+
+#[newtype_index]
+pub(crate) struct SuppressionIndex;
+
+#[derive(Clone, Debug, Eq, PartialEq, Hash)]
+pub(crate) struct Suppression {
+    target: Option<LintId>,
+    kind: SuppressionKind,
+}
+
+#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash)]
+pub(crate) enum SuppressionKind {
+    /// A `type: ignore` comment
+    TypeIgnore,
+
+    /// A `knot: ignore` comment
+    KnotIgnore,
+}

crates/red_knot_python_semantic/src/suppression.rs

@@ -1,5 +1,7 @@
 use std::hash::Hash;
 
+use context::InferContext;
+use diagnostic::{report_not_iterable, report_not_iterable_possibly_unbound};
 use indexmap::IndexSet;
 use itertools::Itertools;
 use ruff_db::diagnostic::Severity;
@@ -29,14 +31,15 @@ use crate::stdlib::{
 use crate::symbol::{Boundness, Symbol};
 use crate::types::call::{CallDunderResult, CallOutcome};
 use crate::types::class_base::ClassBase;
-use crate::types::diagnostic::{TypeCheckDiagnosticsBuilder, INVALID_TYPE_FORM};
+use crate::types::diagnostic::INVALID_TYPE_FORM;
 use crate::types::mro::{Mro, MroError, MroIterator};
 use crate::types::narrow::narrowing_constraint;
 use crate::{Db, FxOrderSet, Module, Program, PythonVersion};
 
 mod builder;
 mod call;
 mod class_base;
+mod context;
 mod diagnostic;
 mod display;
 mod infer;
@@ -2177,17 +2180,13 @@ pub struct InvalidTypeExpressionError<'db> {
 }
 
 impl<'db> InvalidTypeExpressionError<'db> {
-    fn into_fallback_type(
-        self,
-        diagnostics: &mut TypeCheckDiagnosticsBuilder,
-        node: &ast::Expr,
-    ) -> Type<'db> {
+    fn into_fallback_type(self, context: &InferContext, node: &ast::Expr) -> Type<'db> {
         let InvalidTypeExpressionError {
             fallback_type,
             invalid_expressions,
         } = self;
         for error in invalid_expressions {
-            diagnostics.add_lint(
+            context.report_lint(
                 &INVALID_TYPE_FORM,
                 node.into(),
                 format_args!("{}", error.reason()),
@@ -2827,20 +2826,20 @@ enum IterationOutcome<'db> {
 impl<'db> IterationOutcome<'db> {
     fn unwrap_with_diagnostic(
         self,
+        context: &InferContext<'db>,
         iterable_node: ast::AnyNodeRef,
-        diagnostics: &mut TypeCheckDiagnosticsBuilder<'db>,
     ) -> Type<'db> {
         match self {
             Self::Iterable { element_ty } => element_ty,
             Self::NotIterable { not_iterable_ty } => {
-                diagnostics.add_not_iterable(iterable_node, not_iterable_ty);
+                report_not_iterable(context, iterable_node, not_iterable_ty);
                 Type::Unknown
             }
             Self::PossiblyUnboundDunderIter {
                 iterable_ty,
                 element_ty,
             } => {
-                diagnostics.add_not_iterable_possibly_unbound(iterable_node, iterable_ty);
+                report_not_iterable_possibly_unbound(context, iterable_node, iterable_ty);
                 element_ty
             }
         }

crates/red_knot_python_semantic/src/types.rs

@@ -1,4 +1,5 @@
-use super::diagnostic::{TypeCheckDiagnosticsBuilder, CALL_NON_CALLABLE};
+use super::context::InferContext;
+use super::diagnostic::CALL_NON_CALLABLE;
 use super::{Severity, Type, TypeArrayDisplay, UnionBuilder};
 use crate::Db;
 use ruff_db::diagnostic::DiagnosticId;
@@ -86,24 +87,23 @@ impl<'db> CallOutcome<'db> {
     }
 
     /// Get the return type of the call, emitting default diagnostics if needed.
-    pub(super) fn unwrap_with_diagnostic<'a>(
+    pub(super) fn unwrap_with_diagnostic(
         &self,
-        db: &'db dyn Db,
+        context: &InferContext<'db>,
         node: ast::AnyNodeRef,
-        diagnostics: &'a mut TypeCheckDiagnosticsBuilder<'db>,
     ) -> Type<'db> {
-        match self.return_ty_result(db, node, diagnostics) {
+        match self.return_ty_result(context, node) {
             Ok(return_ty) => return_ty,
             Err(NotCallableError::Type {
                 not_callable_ty,
                 return_ty,
             }) => {
-                diagnostics.add_lint(
+                context.report_lint(
                     &CALL_NON_CALLABLE,
                     node,
                     format_args!(
                         "Object of type `{}` is not callable",
-                        not_callable_ty.display(db)
+                        not_callable_ty.display(context.db())
                     ),
                 );
                 return_ty
@@ -113,13 +113,13 @@ impl<'db> CallOutcome<'db> {
                 called_ty,
                 return_ty,
             }) => {
-                diagnostics.add_lint(
+                context.report_lint(
                     &CALL_NON_CALLABLE,
                     node,
                     format_args!(
                         "Object of type `{}` is not callable (due to union element `{}`)",
-                        called_ty.display(db),
-                        not_callable_ty.display(db),
+                        called_ty.display(context.db()),
+                        not_callable_ty.display(context.db()),
                     ),
                 );
                 return_ty
@@ -129,13 +129,13 @@ impl<'db> CallOutcome<'db> {
                 called_ty,
                 return_ty,
             }) => {
-                diagnostics.add_lint(
+                context.report_lint(
                     &CALL_NON_CALLABLE,
                     node,
                     format_args!(
                         "Object of type `{}` is not callable (due to union elements {})",
-                        called_ty.display(db),
-                        not_callable_tys.display(db),
+                        called_ty.display(context.db()),
+                        not_callable_tys.display(context.db()),
                     ),
                 );
                 return_ty
@@ -144,12 +144,12 @@ impl<'db> CallOutcome<'db> {
                 callable_ty: called_ty,
                 return_ty,
             }) => {
-                diagnostics.add_lint(
+                context.report_lint(
                     &CALL_NON_CALLABLE,
                     node,
                     format_args!(
                         "Object of type `{}` is not callable (possibly unbound `__call__` method)",
-                        called_ty.display(db)
+                        called_ty.display(context.db())
                     ),
                 );
                 return_ty
@@ -158,23 +158,22 @@ impl<'db> CallOutcome<'db> {
     }
 
     /// Get the return type of the call as a result.
-    pub(super) fn return_ty_result<'a>(
+    pub(super) fn return_ty_result(
         &self,
-        db: &'db dyn Db,
+        context: &InferContext<'db>,
         node: ast::AnyNodeRef,
-        diagnostics: &'a mut TypeCheckDiagnosticsBuilder<'db>,
     ) -> Result<Type<'db>, NotCallableError<'db>> {
         match self {
             Self::Callable { return_ty } => Ok(*return_ty),
             Self::RevealType {
                 return_ty,
                 revealed_ty,
             } => {
-                diagnostics.add(
+                context.report_diagnostic(
                     node,
                     DiagnosticId::RevealedType,
                     Severity::Info,
-                    format_args!("Revealed type is `{}`", revealed_ty.display(db)),
+                    format_args!("Revealed type is `{}`", revealed_ty.display(context.db())),
                 );
                 Ok(*return_ty)
             }
@@ -187,14 +186,16 @@ impl<'db> CallOutcome<'db> {
                 call_outcome,
             } => Err(NotCallableError::PossiblyUnboundDunderCall {
                 callable_ty: *called_ty,
-                return_ty: call_outcome.return_ty(db).unwrap_or(Type::Unknown),
+                return_ty: call_outcome
+                    .return_ty(context.db())
+                    .unwrap_or(Type::Unknown),
             }),
             Self::Union {
                 outcomes,
                 called_ty,
             } => {
                 let mut not_callable = vec![];
-                let mut union_builder = UnionBuilder::new(db);
+                let mut union_builder = UnionBuilder::new(context.db());
                 let mut revealed = false;
                 for outcome in outcomes {
                     let return_ty = match outcome {
@@ -210,10 +211,10 @@ impl<'db> CallOutcome<'db> {
                                 *return_ty
                             } else {
                                 revealed = true;
-                                outcome.unwrap_with_diagnostic(db, node, diagnostics)
+                                outcome.unwrap_with_diagnostic(context, node)
                             }
                         }
-                        _ => outcome.unwrap_with_diagnostic(db, node, diagnostics),
+                        _ => outcome.unwrap_with_diagnostic(context, node),
                     };
                     union_builder = union_builder.add(return_ty);
                 }