Skip to content

dev: new commands system #4412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Feb 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
7f3900b
chore: clean config comments
ldez Feb 22, 2024
ec25f9e
feat: add cache command
ldez Feb 22, 2024
8cf5225
feat: add version command
ldez Feb 22, 2024
885878d
feat: add help command
ldez Feb 22, 2024
d2f0a8f
feat: add new configuration loader
ldez Feb 22, 2024
084ce0b
feat: add config command
ldez Feb 22, 2024
07953cd
feat: add new configuration flagset bindings
ldez Feb 22, 2024
817a775
feat: add linters command
ldez Feb 22, 2024
806eeaf
feat: add run command
ldez Feb 22, 2024
c306ef4
feat: add root command
ldez Feb 22, 2024
871c1c6
feat: use the new commands
ldez Feb 22, 2024
639b90b
chore: remove old cache command
ldez Feb 22, 2024
3870c56
chore: remove old version command
ldez Feb 22, 2024
ce6b24c
chore: remove old help command
ldez Feb 22, 2024
bad8c14
chore: remove old config command
ldez Feb 22, 2024
e24e294
chore: remove old linters command
ldez Feb 22, 2024
86aae64
chore: remove old run command
ldez Feb 22, 2024
d394533
chore: remove old root command
ldez Feb 22, 2024
22057e4
chore: death of the Executor
ldez Feb 22, 2024
a4cbde7
chore: remove config.Reader
ldez Feb 22, 2024
45f3109
chore: move configuration validation to config
ldez Feb 22, 2024
f529899
chore: clean Run structure
ldez Feb 22, 2024
b82dddf
chore: clean Output structure
ldez Feb 22, 2024
397e80d
docs: flags override file configuration even for slices
ldez Feb 23, 2024
0a88597
docs: remove old Init section
ldez Feb 23, 2024
5e89135
fix: remove tests on unsupported options in config
ldez Feb 23, 2024
e910424
fix: tests with slice option merge
ldez Feb 23, 2024
0c34280
chore: rename new package to commands
ldez Feb 23, 2024
f3bcb5c
chore: move the Go detection inside the configuration loader
ldez Feb 23, 2024
8497bc7
chore: clean Config structure
ldez Feb 23, 2024
fcee639
fix: override help command of the root command
ldez Feb 24, 2024
fb8d76b
chore(plugins): replace viper.ConfigFileUsed() by cfg.GetConfigDir()
ldez Feb 24, 2024
8e7beeb
chore: remove configuration fields for commands when a command doesn'…
ldez Feb 24, 2024
aba8d5b
chore: hack to append values on StringSlice
ldez Feb 25, 2024
3fa551e
chore: improve comment about Run.Args field
ldez Feb 25, 2024
b39613e
chore: remove missing unused configuration field
ldez Feb 25, 2024
8aa1b04
chore: clean configuration reference
ldez Feb 25, 2024
0e6009c
review: rename Vibra to AddFlagAndBind
ldez Feb 26, 2024
0ada04e
review: improve rootOptions comments
ldez Feb 26, 2024
c84d5cd
review: remove Loader receiver
ldez Feb 26, 2024
c8a133a
review: remove wh function and lll is stil not my friend
ldez Feb 26, 2024
740c5ca
Revert "chore: clean configuration reference"
ldez Feb 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions cmd/golangci-lint/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,8 @@ func main() {
Date: date,
}

e := commands.NewExecutor(info)

if err := e.Execute(); err != nil {
fmt.Fprintf(os.Stderr, "failed executing command with error %v\n", err)
if err := commands.Execute(info); err != nil {
_, _ = fmt.Fprintf(os.Stderr, "failed executing command with error %v\n", err)
os.Exit(exitcodes.Failure)
}
}
84 changes: 0 additions & 84 deletions docs/src/docs/contributing/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,90 +22,6 @@ graph LR

</ResponsiveContainer>

## Init

The execution starts here:

```go title=cmd/golangci-lint/main.go
func main() {
e := commands.NewExecutor(info)

if err := e.Execute(); err != nil {
fmt.Fprintf(os.Stderr, "failed executing command with error %v\n", err)
os.Exit(exitcodes.Failure)
}
}
```

The **executer** is our abstraction:

```go title=pkg/commands/executor.go
type Executor struct {
rootCmd *cobra.Command
runCmd *cobra.Command
lintersCmd *cobra.Command

exitCode int
buildInfo BuildInfo

cfg *config.Config
log logutils.Log
reportData report.Data
DBManager *lintersdb.Manager
EnabledLintersSet *lintersdb.EnabledSet
contextLoader *lint.ContextLoader
goenv *goutil.Env
fileCache *fsutils.FileCache
lineCache *fsutils.LineCache
pkgCache *pkgcache.Cache
debugf logutils.DebugFunc
sw *timeutils.Stopwatch

loadGuard *load.Guard
flock *flock.Flock
}
```

We use dependency injection and all root dependencies are stored in this executor.

In the function `NewExecutor` we do the following:

1. Initialize dependencies.
2. Initialize [cobra](https://github.com/spf13/cobra) commands.
3. Parse the config file using [viper](https://github.com/spf13/viper) and merge it with command line arguments.

The following execution is controlled by `cobra`. If a user executes `golangci-lint run`
then `cobra` executes `e.runCmd`.

Different `cobra` commands have different runners, e.g. a `run` command is configured in the following way:

```go title=pkg/commands/run.go
func (e *Executor) initRun() {
e.runCmd = &cobra.Command{
Use: "run",
Short: "Run the linters",
Run: e.executeRun,
PreRunE: func(_ *cobra.Command, _ []string) error {
if ok := e.acquireFileLock(); !ok {
return errors.New("parallel golangci-lint is running")
}
return nil
},
PostRun: func(_ *cobra.Command, _ []string) {
e.releaseFileLock()
},
}
e.rootCmd.AddCommand(e.runCmd)

e.runCmd.SetOut(logutils.StdOut) // use custom output to properly color it in Windows terminals
e.runCmd.SetErr(logutils.StdErr)

e.initRunConfiguration(e.runCmd)
}
```

The primary execution function of the `run` command is `executeRun`.

## Load Packages

Loading packages is listing all packages and their recursive dependencies for analysis.
Expand Down
122 changes: 30 additions & 92 deletions pkg/commands/cache.go
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
package commands

import (
"bytes"
"crypto/sha256"
"fmt"
"io"
"os"
"path/filepath"
"strings"

"github.com/spf13/cobra"
"gopkg.in/yaml.v3"

"github.com/golangci/golangci-lint/internal/cache"
"github.com/golangci/golangci-lint/pkg/config"
"github.com/golangci/golangci-lint/pkg/fsutils"
"github.com/golangci/golangci-lint/pkg/logutils"
)

func (e *Executor) initCache() {
type cacheCommand struct {
cmd *cobra.Command
}

func newCacheCommand() *cacheCommand {
c := &cacheCommand{}

cacheCmd := &cobra.Command{
Use: "cache",
Short: "Cache control and information",
Expand All @@ -28,42 +28,45 @@ func (e *Executor) initCache() {
},
}

cacheCmd.AddCommand(&cobra.Command{
Use: "clean",
Short: "Clean cache",
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
RunE: e.executeCacheClean,
})
cacheCmd.AddCommand(&cobra.Command{
Use: "status",
Short: "Show cache status",
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
Run: e.executeCacheStatus,
})
cacheCmd.AddCommand(
&cobra.Command{
Use: "clean",
Short: "Clean cache",
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
RunE: c.executeClean,
},
&cobra.Command{
Use: "status",
Short: "Show cache status",
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
Run: c.executeStatus,
},
)

// TODO: add trim command?
c.cmd = cacheCmd

e.rootCmd.AddCommand(cacheCmd)
return c
}

func (e *Executor) executeCacheClean(_ *cobra.Command, _ []string) error {
func (c *cacheCommand) executeClean(_ *cobra.Command, _ []string) error {
cacheDir := cache.DefaultDir()

if err := os.RemoveAll(cacheDir); err != nil {
return fmt.Errorf("failed to remove dir %s: %w", cacheDir, err)
}

return nil
}

func (e *Executor) executeCacheStatus(_ *cobra.Command, _ []string) {
func (c *cacheCommand) executeStatus(_ *cobra.Command, _ []string) {
cacheDir := cache.DefaultDir()
fmt.Fprintf(logutils.StdOut, "Dir: %s\n", cacheDir)
_, _ = fmt.Fprintf(logutils.StdOut, "Dir: %s\n", cacheDir)

cacheSizeBytes, err := dirSizeBytes(cacheDir)
if err == nil {
fmt.Fprintf(logutils.StdOut, "Size: %s\n", fsutils.PrettifyBytesCount(cacheSizeBytes))
_, _ = fmt.Fprintf(logutils.StdOut, "Size: %s\n", fsutils.PrettifyBytesCount(cacheSizeBytes))
}
}

Expand All @@ -77,68 +80,3 @@ func dirSizeBytes(path string) (int64, error) {
})
return size, err
}

// --- Related to cache but not used directly by the cache command.

func initHashSalt(version string, cfg *config.Config) error {
binSalt, err := computeBinarySalt(version)
if err != nil {
return fmt.Errorf("failed to calculate binary salt: %w", err)
}

configSalt, err := computeConfigSalt(cfg)
if err != nil {
return fmt.Errorf("failed to calculate config salt: %w", err)
}

b := bytes.NewBuffer(binSalt)
b.Write(configSalt)
cache.SetSalt(b.Bytes())
return nil
}

func computeBinarySalt(version string) ([]byte, error) {
if version != "" && version != "(devel)" {
return []byte(version), nil
}

if logutils.HaveDebugTag(logutils.DebugKeyBinSalt) {
return []byte("debug"), nil
}

p, err := os.Executable()
if err != nil {
return nil, err
}
f, err := os.Open(p)
if err != nil {
return nil, err
}
defer f.Close()
h := sha256.New()
if _, err := io.Copy(h, f); err != nil {
return nil, err
}
return h.Sum(nil), nil
}

// computeConfigSalt computes configuration hash.
// We don't hash all config fields to reduce meaningless cache invalidations.
// At least, it has a huge impact on tests speed.
// Fields: `LintersSettings` and `Run.BuildTags`.
func computeConfigSalt(cfg *config.Config) ([]byte, error) {
lintersSettingsBytes, err := yaml.Marshal(cfg.LintersSettings)
if err != nil {
return nil, fmt.Errorf("failed to json marshal config linter settings: %w", err)
}

configData := bytes.NewBufferString("linters-settings=")
configData.Write(lintersSettingsBytes)
configData.WriteString("\nbuild-tags=%s" + strings.Join(cfg.Run.BuildTags, ","))

h := sha256.New()
if _, err := h.Write(configData.Bytes()); err != nil {
return nil, err
}
return h.Sum(nil), nil
}
70 changes: 44 additions & 26 deletions pkg/commands/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,27 @@ import (
"os"

"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/spf13/viper"

"github.com/golangci/golangci-lint/pkg/config"
"github.com/golangci/golangci-lint/pkg/exitcodes"
"github.com/golangci/golangci-lint/pkg/fsutils"
"github.com/golangci/golangci-lint/pkg/logutils"
)

func (e *Executor) initConfig() {
type configCommand struct {
viper *viper.Viper
cmd *cobra.Command

log logutils.Log
}

func newConfigCommand(log logutils.Log) *configCommand {
c := &configCommand{
viper: viper.New(),
log: log,
}

configCmd := &cobra.Command{
Use: "config",
Short: "Config file information",
Expand All @@ -23,25 +35,38 @@ func (e *Executor) initConfig() {
},
}

pathCmd := &cobra.Command{
Use: "path",
Short: "Print used config path",
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
Run: e.executePath,
}
configCmd.AddCommand(
&cobra.Command{
Use: "path",
Short: "Print used config path",
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
Run: c.execute,
PreRunE: c.preRunE,
},
)

fs := pathCmd.Flags()
fs.SortFlags = false // sort them as they are defined here
c.cmd = configCmd

configCmd.AddCommand(pathCmd)
e.rootCmd.AddCommand(configCmd)
return c
}

func (e *Executor) executePath(_ *cobra.Command, _ []string) {
usedConfigFile := e.getUsedConfig()
func (c *configCommand) preRunE(cmd *cobra.Command, _ []string) error {
// The command doesn't depend on the real configuration.
// It only needs to know the path of the configuration file.
loader := config.NewLoader(c.log.Child(logutils.DebugKeyConfigReader), c.viper, cmd.Flags(), config.LoaderOptions{}, config.NewDefault())

if err := loader.Load(); err != nil {
return fmt.Errorf("can't load config: %w", err)
}

return nil
}

func (c *configCommand) execute(_ *cobra.Command, _ []string) {
usedConfigFile := c.getUsedConfig()
if usedConfigFile == "" {
e.log.Warnf("No config file detected")
c.log.Warnf("No config file detected")
os.Exit(exitcodes.NoConfigFileDetected)
}

Expand All @@ -50,24 +75,17 @@ func (e *Executor) executePath(_ *cobra.Command, _ []string) {

// getUsedConfig returns the resolved path to the golangci config file,
// or the empty string if no configuration could be found.
func (e *Executor) getUsedConfig() string {
usedConfigFile := viper.ConfigFileUsed()
func (c *configCommand) getUsedConfig() string {
usedConfigFile := c.viper.ConfigFileUsed()
if usedConfigFile == "" {
return ""
}

prettyUsedConfigFile, err := fsutils.ShortestRelPath(usedConfigFile, "")
if err != nil {
e.log.Warnf("Can't pretty print config file path: %s", err)
c.log.Warnf("Can't pretty print config file path: %s", err)
return usedConfigFile
}

return prettyUsedConfigFile
}

// --- Related to config but not used directly by the config command.

func initConfigFileFlagSet(fs *pflag.FlagSet, cfg *config.Run) {
fs.StringVarP(&cfg.Config, "config", "c", "", wh("Read config from file path `PATH`"))
fs.BoolVar(&cfg.NoConfig, "no-config", false, wh("Don't read config file"))
}
Loading