Significant/breaking increase in memory usage in 1.19

[bmhatfield]

Using the release binary for amd64/linux, on 1.18 my linter set completes using 2364.4MB. On 1.19 it gets killed at 4GB by CircleCI. Running locally on OSX to experiment, it uses 8-9GB.

Updating to latest master on OSX to get this fix: #725 has no effect (no change in memory usage at all).

Note that this memory usage increase appears to be coming from megacheck:

v1.18.0 tag:

[bhatfield go-services]% GOGC=50; golangci-lint --concurrency 4 -v run --disable=govet
INFO [config_reader] Config search paths: [./ /Users/bhatfield/Documents/digits/go-services /Users/bhatfield/Documents/digits /Users/bhatfield/Documents /Users/bhatfield /Users /]
INFO [config_reader] Used config file .golangci.yml
INFO [lintersdb] Active 18 linters: [deadcode errcheck goconst gocritic gocyclo goimports gosimple ineffassign nakedret prealloc scopelint staticcheck structcheck stylecheck typecheck unconvert unused varcheck]
INFO [lintersdb] Optimized sublinters [staticcheck gosimple unused stylecheck] into metalinter megacheck
INFO [loader] Go packages loading at mode 991 (syntax|types|types_sizes|compiled_files|files|imports|deps|name|types_info) took 4.250424796s
INFO [loader] SSA repr building timing: packages building 81.21955ms, total 1.264090541s
INFO [runner] worker.4 took 1.711990296s with stages: gocritic: 1.635968444s, structcheck: 72.870967ms
INFO [runner] worker.3 took 1.756356889s with stages: ineffassign: 985.678416ms, goconst: 182.499071ms, varcheck: 173.635565ms, gocyclo: 110.043876ms, deadcode: 96.680908ms, nakedret: 62.802763ms, scopelint: 58.973492ms, unconvert: 32.255902ms, errcheck: 29.478845ms, prealloc: 24.099253ms, typecheck: 6.887µs
INFO [runner] worker.2 took 3.792684165s with stages: goimports: 3.792663762s
INFO [runner] worker.1 took 6.958488768s with stages: megacheck: 6.958478946s
INFO [runner] Workers idle times: #2: 3.164978969s, #3: 5.200812029s, #4: 5.242749804s
INFO [runner/skip dirs] Skipped dir services/internal-apis/accounting/models/vendor by pattern (^|/)vendor($|/)
INFO [runner] Issues before processing: 152, after processing: 0
INFO [runner] processing took 281.866956ms with stages: skip_dirs: 226.222899ms, autogenerated_exclude: 41.558168ms, cgo: 4.484415ms, exclude: 3.703539ms, filename_unadjuster: 3.0014ms, identifier_marker: 2.458822ms, path_prettifier: 363.902µs, exclude-rules: 62.329µs, diff: 1.785µs, nolint: 1.524µs, skip_files: 1.514µs, max_same_issues: 1.513µs, source_code: 1.312µs, max_from_linter: 1.171µs, path_shortener: 1.102µs, uniq_by_line: 799ns, max_per_file_from_linter: 762ns
INFO File cache stats: 0 entries of total size 0B
INFO Memory: 94 samples, avg is 1027.0MB, max is 2297.9MB
INFO Execution took 12.670883489s

master@9d1610b (1.19+a few more commits)

[bhatfield go-services]% GOGC=50; golangci-lint --concurrency 4 -v run --disable=govet
INFO [config_reader] Config search paths: [./ /Users/bhatfield/Documents/digits/go-services /Users/bhatfield/Documents/digits /Users/bhatfield/Documents /Users/bhatfield /Users /]
INFO [config_reader] Used config file .golangci.yml
INFO [lintersdb] Active 18 linters: [deadcode errcheck goconst gocritic gocyclo goimports gosimple ineffassign nakedret prealloc scopelint staticcheck structcheck stylecheck typecheck unconvert unused varcheck]
INFO [lintersdb] Optimized sublinters [staticcheck gosimple unused stylecheck] into metalinter megacheck
INFO [loader] Go packages loading at mode 991 (name|types|types_sizes|deps|files|imports|syntax|types_info|compiled_files) took 3.9507664s
INFO [runner] worker.2 took 2.136538439s with stages: gocritic: 1.657748278s, varcheck: 120.738108ms, unconvert: 116.189926ms, deadcode: 100.656412ms, structcheck: 86.42681ms, errcheck: 38.352101ms, nakedret: 16.333898ms, typecheck: 3.679µs
INFO [runner] worker.3 took 7.249678626s with stages: ineffassign: 7.099321821s, gocyclo: 58.865309ms, prealloc: 49.891697ms, scopelint: 22.223369ms, goconst: 19.326649ms
INFO [runner] worker.4 took 8.50693826s with stages: goimports: 8.506919202s
INFO [runner] worker.1 took 24.112767161s with stages: megacheck: 24.112757775s
INFO [runner] Workers idle times: #2: 21.974884465s, #3: 16.861314627s, #4: 15.602948006s
INFO [runner/skip dirs] Skipped 1 issues from dir services/internal-apis/file-relay/models/vendor by pattern (^|/)vendor($|/)
INFO [runner/skip dirs] Skipped 16 issues from dir services/internal-apis/accounting/models/vendor by pattern (^|/)vendor($|/)
INFO [runner/skip dirs] Skipped 1 issues from dir services/internal-apis/accounting/workers/vendor by pattern (^|/)vendor($|/)
INFO [runner] Issues before processing: 930, after processing: 0
INFO [runner] Processors filtering stat (out/in): cgo: 930/930, skip_dirs: 912/930, exclude: 77/785, exclude-rules: 0/77, filename_unadjuster: 930/930, path_prettifier: 930/930, skip_files: 930/930, autogenerated_exclude: 785/912, identifier_marker: 785/785
INFO [runner] processing took 185.187268ms with stages: autogenerated_exclude: 61.923794ms, exclude: 46.729878ms, identifier_marker: 41.010926ms, skip_dirs: 30.009804ms, path_prettifier: 4.363151ms, cgo: 609.941µs, filename_unadjuster: 482.862µs, exclude-rules: 43.167µs, max_same_issues: 2.111µs, nolint: 2.032µs, diff: 1.806µs, path_shortener: 1.578µs, source_code: 1.527µs, skip_files: 1.414µs, max_from_linter: 1.248µs, max_per_file_from_linter: 1.038µs, uniq_by_line: 991ns
INFO File cache stats: 0 entries of total size 0B
INFO Memory: 116 samples, avg is 4095.9MB, max is 8211.0MB
INFO Execution took 28.288715613s

[bmhatfield]

With some more experimentation I was able to determine the specific commit which jumped memory usage for me:

This is fine (completes @2230.2MB)

[bhatfield golangci-lint]% git checkout 0e0cd75
Previous HEAD position was 358ce7c Enable Go module support across all commands. (#717)
HEAD is now at 0e0cd75 Update to valyala/quicktemplate v1.2.0 (#704)
[bhatfield golangci-lint]% go install

This is not (completes @9486.8MB)

[bhatfield golangci-lint]% git checkout 6a979fb
Previous HEAD position was 0e0cd75 Update to valyala/quicktemplate v1.2.0 (#704)
HEAD is now at 6a979fb Update staticcheck and cache go/analysis facts (#699)
[bhatfield golangci-lint]% go install

And current master (completes @7808.8MB)

[bhatfield golangci-lint]% git checkout master
Previous HEAD position was 6a979fb Update staticcheck and cache go/analysis facts (#699)
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
[bhatfield golangci-lint]% go install

[alexsn]

Same issue here, had to pin version to 1.18.

[jirfag]

hi, thank you,
if you can add which open-source projects you analyze it will be helpful

[alexsn]

Hi, my project is not open source yet. Below is the config which I use from circleci:

run:
  deadline: 2m

linters-settings:
  govet:
    check-shadowing: false
  golint:
    min-confidence: 0
  goconst:
    min-len: 2
    min-occurrences: 2
  depguard:
    list-type: blacklist
  misspell:
    locale: US
  errcheck:
    ignore: :[Shutdown|Sync|HTML]

linters:
  enable-all: true
  disable:
    - lll
    - dupl
    - maligned
    - prealloc
    - gosec
    - goimports
    - gochecknoglobals
    - gochecknoinits
    - gocyclo
    - funlen

[bmhatfield]

Unfortunately the project I am experiencing this with is also not open-source. However, it's possibly notable that this project is "larger", containing about 200 packages plus dependencies, and we've had to manage how concurrent we allow the linker to be to control memory consumption.

In this particular case, based upon the commit message and the noticeable increase in megacheck running time, I suggest that the issue is either related to the staticcheck upgrade or the cache that was added.

[kolyshkin]

We (at Docker) are also interested in ways to (directly or indirectly) limit golangci-lint memory consumption; we are seeing a regression with 1.18.0 (compared to 1.17.1), I guess it's repeatable and Docker is open source; see moby/moby#39964 for more details

[jirfag]

ok, thank you, let's continue in #337, there are too many similar issues about memory