architecture.mdx

title
Architecture

import ResponsiveContainer from "components/ResponsiveContainer";

There are the following golangci-lint execution steps:

graph LR
    init[Init]
    loadPackages[Load packages]
    runLinters[Run linters]
    postprocess[Postprocess issues]
    print[Print issues]

    init --> loadPackages --> runLinters --> postprocess --> print

Loading

Init

The configuration is loaded from file and flags by config.Loader inside PersistentPreRun (or PreRun) of the commands that require configuration.

The linter database (linterdb.Manager) is fill based on the configuration:

• The linters ("internals" and plugins) are built by linterdb.LinterBuilder and linterdb.PluginBuilder builders.
• The configuration is validated by linterdb.Validator.

Load Packages

Loading packages is listing all packages and their recursive dependencies for analysis. Also, depending on the enabled linters set some parsing of the source code can be performed at this step.

Packages loading starts here:

func (cl *ContextLoader) Load(ctx context.Context, linters []*linter.Config) (*linter.Context, error) {
	loadMode := cl.findLoadMode(linters)
	pkgs, err := cl.loadPackages(ctx, loadMode)
	if err != nil {
		return nil, fmt.Errorf("failed to load packages: %w", err)
	}

	// ...
	ret := &linter.Context{
		// ...
	}
	return ret, nil
}

First, we find a load mode as union of load modes for all enabled linters. We use go/packages for packages loading and use it's enum packages.Need* for load modes. Load mode sets which data does a linter needs for execution.

A linter that works only with AST need minimum of information: only filenames and AST. There is no need for packages dependencies or type information. AST is built during go/analysis execution to reduce memory usage. Such AST-based linters are configured with the following code:

func (lc *Config) WithLoadFiles() *Config {
	lc.LoadMode |= packages.NeedName | packages.NeedFiles | packages.NeedCompiledGoFiles
	return lc
}

If a linter uses go/analysis and needs type information, we need to extract more data by go/packages:

func (lc *Config) WithLoadForGoAnalysis() *Config {
	lc = lc.WithLoadFiles()
	lc.LoadMode |= packages.NeedImports | packages.NeedDeps | packages.NeedExportFile | packages.NeedTypesSizes
	lc.IsSlow = true
	return lc
}

After finding a load mode, we run go/packages: the library get list of dirs (or ./... as the default value) as input and outputs list of packages and requested information about them: filenames, type information, AST, etc.

Run Linters

First, we need to find all enabled linters. All linters are registered here:

func (b LinterBuilder) Build(cfg *config.Config) []*linter.Config {
	// ...
	return []*linter.Config{
		// ...
		linter.NewConfig(golinters.NewBodyclose()).
			WithSince("v1.18.0").
			WithLoadForGoAnalysis().
			WithPresets(linter.PresetPerformance, linter.PresetBugs).
			WithURL("https://github.com/timakin/bodyclose"),
		// ...
		linter.NewConfig(golinters.NewGovet(govetCfg)).
			WithEnabledByDefault().
			WithSince("v1.0.0").
			WithLoadForGoAnalysis().
			WithPresets(linter.PresetBugs, linter.PresetMetaLinter).
			WithAlternativeNames("vet", "vetshadow").
			WithURL("https://pkg.go.dev/cmd/vet"),
		// ...
	}
}

We filter requested in config and command-line linters in EnabledSet:

func (m *Manager) GetEnabledLintersMap() (map[string]*linter.Config, error)

We merge enabled linters into one MetaLinter to improve execution time if we can:

// GetOptimizedLinters returns enabled linters after optimization (merging) of multiple linters into a fewer number of linters.
// E.g. some go/analysis linters can be optimized into one metalinter for data reuse and speed up.
func (m *Manager) GetOptimizedLinters() ([]*linter.Config, error) {
	// ...
	m.combineGoAnalysisLinters(resultLintersSet)
	// ...
}

The MetaLinter just stores all merged linters inside to run them at once:

type MetaLinter struct {
	linters              []*Linter
	analyzerToLinterName map[*analysis.Analyzer]string
}

Currently, all linters except unused can be merged into this meta linter. The unused isn't merged because it has high memory usage.

Linters execution starts in runAnalyzers. It's the most complex part of the golangci-lint. We use custom go/analysis runner there. It runs as much as it can in parallel. It lazy-loads as much as it can to reduce memory usage. Also, it sets all heavyweight data to nil as becomes unneeded to save memory.

We don't use existing multichecker because it doesn't use caching and doesn't have some important performance optimizations.

All found by linters issues are represented with result.Issue struct:

type Issue struct {
	FromLinter string
	Text       string

	Severity string

	// Source lines of a code with the issue to show
	SourceLines []string

	// If we know how to fix the issue we can provide replacement lines
	Replacement *Replacement

	// Pkg is needed for proper caching of linting results
	Pkg *packages.Package `json:"-"`

	LineRange *Range `json:",omitempty"`

	Pos token.Position

	// HunkPos is used only when golangci-lint is run over a diff
	HunkPos int `json:",omitempty"`

	// If we are expecting a nolint (because this is from nolintlint), record the expected linter
	ExpectNoLint         bool
	ExpectedNoLintLinter string
}

Postprocess Issues

We have an abstraction of result.Processor to postprocess found issues:

$ tree -L 1 ./pkg/result/processors/
./pkg/result/processors/
./pkg/result/processors/
├── autogenerated_exclude.go
├── autogenerated_exclude_test.go
├── base_rule.go
├── cgo.go
├── diff.go
├── exclude.go
├── exclude_rules.go
├── exclude_rules_test.go
├── exclude_test.go
├── filename_unadjuster.go
├── fixer.go
├── identifier_marker.go
├── identifier_marker_test.go
├── issues.go
├── max_from_linter.go
├── max_from_linter_test.go
├── max_per_file_from_linter.go
├── max_per_file_from_linter_test.go
├── max_same_issues.go
├── max_same_issues_test.go
├── nolint.go
├── nolint_test.go
├── path_prefixer.go
├── path_prefixer_test.go
├── path_prettifier.go
├── path_shortener.go
├── processor.go
├── processor_test.go
├── severity_rules.go
├── severity_rules_test.go
├── skip_dirs.go
├── skip_files.go
├── skip_files_test.go
├── sort_results.go
├── sort_results_test.go
├── source_code.go
├── testdata
├── uniq_by_line.go
└── uniq_by_line_test.go

The abstraction is simple:

type Processor interface {
	Process(issues []result.Issue) ([]result.Issue, error)
	Name() string
	Finish()
}

A processor can hide issues (nolint, exclude) or change issues (path_shortener).

Print Issues

We have an abstraction for printing found issues.

$ tree -L 1 ./pkg/printers/
./pkg/printers/
├── checkstyle.go
├── checkstyle_test.go
├── codeclimate.go
├── codeclimate_test.go
├── github.go
├── github_test.go
├── html.go
├── html_test.go
├── json.go
├── json_test.go
├── junitxml.go
├── junitxml_test.go
├── printer.go
├── tab.go
├── tab_test.go
├── teamcity.go
├── teamcity_test.go
├── text.go
└── text_test.go

Needed printer is selected by command line option --out-format.