Bug: "Invalid string length" caused by allowDefaultProjectForFiles warning

[domarmstrong]

Before You File a Bug Report Please Confirm You Have Done The Following...

• I have tried restarting my IDE and the issue persists.
• I have updated to the latest version of the packages.
• I have searched for related issues and found none that matched my issue.
• I have read the FAQ and my problem is not listed.

Relevant Package

typescript-estree

Playground Link

No response

Repro Code

Can't provide a minimal sample as it requires a large code base

ESLint Config

No response

tsconfig

No response

Expected Result

Warnings but no error

Actual Result

ESLint: 8.57.0

RangeError: Invalid string length
    at Array.join (<anonymous>)
    at Object.stringify (/.yarn/cache/flatted-npm-3.2.9-0462256d3c-dc2b89e46a.zip/node_modules/flatted/cjs/index.js:78:23)
    at writeJSON (/.yarn/cache/flat-cache-npm-3.2.0-9a887f084e-02381c6ece.zip/node_modules/flat-cache/src/utils.js:42:40)
    at Object.save (/.yarn/cache/flat-cache-npm-3.2.0-9a887f084e-02381c6ece.zip/node_modules/flat-cache/src/cache.js:140:5)
    at Object.reconcile (/.yarn/cache/file-entry-cache-npm-6.0.1-31965cf0af-099bb9d4ab.zip/node_modules/file-entry-cache/cache.js:287:15)
    at LintResultCache.reconcile (/.yarn/cache/eslint-npm-8.57.0-4286e12a3a-00496e218b.zip/node_modules/eslint/lib/cli-engine/lint-result-cache.js:199:29)
    at CLIEngine.executeOnFiles (/.yarn/cache/eslint-npm-8.57.0-4286e12a3a-00496e218b.zip/node_modules/eslint/lib/cli-engine/cli-engine.js:861:29)
    at ESLint.lintFiles (/.yarn/cache/eslint-npm-8.57.0-4286e12a3a-00496e218b.zip/node_modules/eslint/lib/eslint/eslint.js:551:23)
    at Object.execute (/.yarn/cache/eslint-npm-8.57.0-4286e12a3a-00496e218b.zip/node_modules/eslint/lib/cli.js:421:36)
    at async main (/.yarn/cache/eslint-npm-8.57.0-4286e12a3a-00496e218b.zip/node_modules/eslint/bin/eslint.js:152:22)

Oops! Something went wrong! :(

ESLint: 8.57.0

RangeError: Invalid string length
    at stringify (/.yarn/cache/json-buffer-npm-3.0.1-f8f6d20603-8287615452.zip/node_modules/json-buffer/index.js:32:35)
    at Object.stringify [as serialize] (/.yarn/cache/json-buffer-npm-3.0.1-f8f6d20603-8287615452.zip/node_modules/json-buffer/index.js:32:37)
    at /.yarn/cache/keyv-npm-4.5.4-4c8e2cf7f7-167eb6ef64.zip/node_modules/keyv/src/index.js:202:22Typescript @5.4.3                    Typescript @5.3.3

Additional Info

We have not correctly configured EXPERIMENTAL_useProjectServicea.llowDefaultProjectForFiles. But because so many files match, and it produces so many instances of the warning, the result no longer serializes and eslint crashes.

Error: Too many files (>8) have matched the default project.

Having many files run with the default project is known to cause performance issues and slow down linting.

See https://typescript-eslint.io/troubleshooting/#allowdefaultprojectforfiles-glob-too-wide

Matching files:
...{1050 file paths}...

If you absolutely need more files included, set parserOptions.EXPERIMENTAL_useProjectService.maximumDefaultProjectFileMatchCount_THIS_WILL_SLOW_DOWN_LINTING to a larger value.

The warning should probably truncate the message like:

Error: Too many files (>8) have matched the default project.

Having many files run with the default project is known to cause performance issues and slow down linting.

See https://typescript-eslint.io/troubleshooting/#allowdefaultprojectforfiles-glob-too-wide

Matching files:
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
- file
... and 1030 additional files

If you absolutely need more files included, set parserOptions.EXPERIMENTAL_useProjectService.maximumDefaultProjectFileMatchCount_THIS_WILL_SLOW_DOWN_LINTING to a larger value.

Versions

package	version
@typescript-eslint/eslint-plugin	7.9.0
@typescript-eslint/parser	7.9.0
@typescript-eslint/rule-tester	7.9.0
@typescript-eslint/scope-manager	7.9.0
@typescript-eslint/typescript-estree	7.9.0
@typescript-eslint/type-utils	7.9.0
@typescript-eslint/utils	7.9.0
TypeScript	5.4.3
ESLint	8.57.0
node	20.11.1

[bradzacher]

Have you got an example of the config...?
It's going to be hard to action this without even a fragment of a reproduction.

[domarmstrong]

The config isn't really going to help much I think. The issue is here https://github.com/typescript-eslint/typescript-eslint/pull/8925/files#diff-b089b5bc90fd2a532136946c025806e6daa02b7deda1f704829d736dfdb4b3c8R86-R98. Essentially it you have a project with 10000 linted files and 1000 incorrectly matching the defaultProjectMatchedFiles, then you end up with 10k x warnings with 1000 file paths which ends up making the whole output too large to serialize, at least with the method eslint uses.

Something like the following should mitigate the problem.

defaultProjectMatchedFiles.add(filePathAbsolute);
if (defaultProjectMatchedFiles.size > maximumDefaultProjectFileMatchCount) {
  const matchedFiles = Array.from(defaultProjectMatchedFiles);
  const filePrintLimit = 20;
  const filesToPrint = matchedFiles.slice(0, filePrintLimit);
  const extraFiles = matchedFiles.length - filesToPrint.length;

  throw new Error(
    `Too many files (>${maximumDefaultProjectFileMatchCount}) have matched the default project.${DEFAULT_PROJECT_FILES_ERROR_EXPLANATION}
Matching files:
${filesToPrint.map((file) => `- ${file}`).join('\n')}
${extraFiles ? `...and ${extraFiles} more files` : ''}
If you absolutely need more files included, set parserOptions.EXPERIMENTAL_useProjectService.maximumDefaultProjectFileMatchCount_THIS_WILL_SLOW_DOWN_LINTING to a larger value.
`,
  );
}

[JoshuaKGoldberg]

Ha, I can reproduce this. typescript-eslint/examples@f55b9e7:

1. npm i
2. cd packages/flat-config-typed
3. node generate.js
4. npm run lint

Agreed that it makes sense to truncate the extra files. 👍 thanks for reporting!

[domarmstrong]

I've opened a PR #9127