Adding component config proposal

Signed-off-by: Chris Hein <[email protected]>

Author: christopherhein

designs/component-config.md

@@ -0,0 +1,322 @@
+# ComponentConfig Controller Runtime Support
+Author: @christopherhein
+
+Last Updated on: 03/02/2020
+
+## Table of Contents
+
+<!--ts-->
+   * [ComponentConfig Controller Runtime Support](#componentconfig-controller-runtime-support)
+      * [Table of Contents](#table-of-contents)
+      * [Summary](#summary)
+      * [Motivation](#motivation)
+         * [Links to Open Issues](#links-to-open-issues)
+         * [Goals](#goals)
+         * [Non-Goals/Future Work](#non-goalsfuture-work)
+      * [Proposal](#proposal)
+            * [ComponentConfig Load Order](#componentconfig-load-order)
+            * [Default ComponentConfig Type](#default-componentconfig-type)
+               * [Caveats](#caveats)
+            * [Kubebuilder Scaffolding Example](#kubebuilder-scaffolding-example)
+      * [User Stories](#user-stories)
+         * [Controller Author with controller-runtime](#controller-author-with-controller-runtime)
+         * [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder)
+         * [Controller User without modifications to config](#controller-user-without-modifications-to-config)
+         * [Controller User with modifications to config](#controller-user-with-modifications-to-config)
+      * [Risks and Mitigations](#risks-and-mitigations)
+      * [Alternatives](#alternatives)
+      * [Implementation History](#implementation-history)
+
+<!--te-->
+
+## Summary
+
+Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hardcoding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized along the pattern of [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` patterns into `controller-runtime` to allow controller authors to make `go` types backed by `apimachinery` to unmarshal and configure the `ctrl.Manager` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args.
+
+
+## Motivation
+
+This change is important because: 
+- it will help make it easier for controllers to be configured by other machine processes
+- it will reduce the required flags required to start a controller
+- allow for more configuration types which flags don't natively support
+- allow using and upgrading older configurations avoiding breaking changes in flags
+
+### Links to Open Issues
+
+- [#518 Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518)
+- [#207 Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207)
+- [#722 Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722)
+
+### Goals
+
+- Provide an interface for pulling configuration data out of exposed `ComponentConfig` types (see below for implementation)
+- Provide a new `ctrl.NewFromComponentConfig()` function for initializing a manager
+- Provide a `DefaultControllerConfig` to make the switch easier
+
+### Non-Goals/Future Work
+
+- `kubebuilder` implementation and design in another PR
+- Changing the default `controller-runtime` implementation
+- Dynamically reloading ComponentConfig object
+- Providing `flags` interface and overrides
+
+## Proposal
+
+The `ctrl.Manager` _SHOULD_ support loading configurations from `ComponentConfig` like objects. 
+An interface for that object with getters for the specific configuration parameters is created to bridge existing patterns.
+
+Without breaking the current `ctrl.NewManager` which uses an exported `ctrl.Options{}` the `manager.go` can expose a new func, `NewFromComponentConfig()` this would be able to loop through the getters to hydrate an internal `ctrl.Options{}` and pass that into `New()`.
+
+File: _`pkg/manager/manager.go`_
+```golang
+// ManagerConfiguration defines what the ComponentConfig object for ControllerRuntime needs to support
+type ManagerConfiguration interface {
+	GetSyncPeriod() *time.Duration
+
+	GetLeaderElection() bool
+	GetLeaderElectionNamespace() string
+	GetLeaderElectionID() string
+
+	GetLeaseDuration() *time.Duration
+	GetRenewDeadline() *time.Duration
+	GetRetryPeriod() *time.Duration
+
+	GetNamespace() string
+	GetMetricsBindAddress() string
+	GetHealthProbeBindAddress() string
+
+	GetReadinessEndpointName() string
+	GetLivenessEndpointName() string
+
+	GetPort() int
+	GetHost() string
+
+	GetCertDir() string
+}
+
+func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme,filename string, managerconfig ManagerConfiguration) (Manager, error) {
+	codecs := serializer.NewCodecFactory(scheme)
+    if err := decodeComponentConfigFileInto(codecs, filename, componentconfig); err != nil {
+		
+	}
+	options := Options{}
+
+	if scheme != nil {
+		options.Scheme = scheme
+	}
+
+	// Loop through getters
+	if managerconfig.GetLeaderElection() {
+			managerconfig.GetLeaderElection()
+	}
+	// ...
+
+	return New(config, options)
+}
+```
+
+#### ComponentConfig Load Order
+
+![ComponentConfig Load Order](/designs/images/component-config-load.png)
+
+#### Default ComponentConfig Type
+
+To enable `controller-runtime` to have a default `ComponentConfig` struct which can be used instead of requiring each controller or extension to build it's own `ComponentConfig` type, we can create a `DefaultControllerConfiguration` type which can exist in `pkg/api/v1alpha1/types.go`. This will allow the controller authors to use this before needing to implement their own type with additional configs.
+
+```golang
+package v1alpha1
+
+import (
+	"time"
+
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	configv1alpha1 "sigs.k8s.io/component-base/config/v1alpha1"
+)
+
+// DefaultControllerConfigurationSpec defines the desired state of DefaultControllerConfiguration
+type DefaultControllerConfigurationSpec struct {
+	SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
+
+	LeaderElection configv1alpha1.LeaderElectionConfiguration `json:"leaderElection,omitempty"`
+
+	MetricsBindAddress string `json:"metricsBindAddress,omitempty"`
+
+	Health DefaultControllerConfigurationHealth `json:"health,omitempty"`
+
+	Port *int   `json:"port,omitempty"`
+	Host string `json:"host,omitempty"`
+
+	CertDir string `json:"certDir,omitempty"`
+}
+
+// DefaultControllerConfigurationHealth defines the health configs
+type DefaultControllerConfigurationHealth struct {
+	HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`
+
+	ReadinessEndpointName string `json:"readinessEndpointName,omitempty"`
+	LivenessEndpointName  string `json:"livenessEndpointName,omitempty"`
+}
+
+// DefaultControllerConfiguration is the Schema for the DefaultControllerConfigurations API
+type DefaultControllerConfiguration struct {
+	metav1.TypeMeta `json:",inline"`
+
+	Spec   DefaultControllerConfigurationSpec   `json:"spec,omitempty"`
+}
+```
+
+This would allow a controller author to use this struct with any config that supports the json/yaml structure. For example a controller author could define their `Kind` as `FoobarControllerConfiguration` and have it defined as the following.
+
+```yaml
+# config.yaml
+apiVersion: somedomain.io/v1alpha1
+kind: FoobarControllerConfiguration
+spec:
+  port: 9443
+  metricsBindAddress: ":8080"
+  leaderElection:
+    leaderElect: false
+```
+
+Given the following config and `DefaultControllerConfiguration` we'd be able to initialize the controller using the following.
+
+
+```golang
+	mgr, err := ctrl.NewManagerFromComponentConfig(ctrl.GetConfigOrDie(), scheme, configname, &defaultv1alpha1.DefaultControllerConfiguration{})
+	if err != nil {
+		// ...
+	}
+```
+
+The above example uses `configname` which is the name of the file to load the configuration from and uses `scheme` to get the specific serializer, eg `serializer.NewCodecFactory(scheme)`. This will allow the configuration to be unmarshalled into the `runtime.Object` type and passed into the 
+`ctrl.NewManagerFromComponentConfig()` as a `ManagerConfiguration` interface.
+
+##### Caveats
+
+> ⚠️ Using `DecodeComponentConfigFileInto` does not support the overrides for flags, this is something that is left up to the controller author since they could be using many different flagging interfaces. eg [`flag`](https://golang.org/pkg/flag/), [`pflag`](https://godoc.org/github.com/spf13/pflag), [`flagnum`](https://godoc.org/github.com/luci/luci-go/common/flag/flagenum) and `controller-runtime` should be agnostic to the CLI implementation.
+
+#### Kubebuilder Scaffolding Example
+
+Within a separate design (link once created) this will require controller authors to generate a type that implements the `ManagerConfiguration` interface. The following is a sample of what this looks like:
+
+```golang
+package config
+
+import (
+  "time"
+
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	configv1alpha1 "sigs.k8s.io/component-base/config/v1alpha1"
+) 
+
+type ControllerNameConfigurationSpec struct {
+	SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
+	
+	LeaderElection configv1alpha1.LeaderElectionConfiguration `json:"leaderElection,omitempty"`
+
+	MetricsBindAddress string `json:"metricsBindAddress,omitempty"`
+
+	Health ControllerNameConfigurationHealth
+
+	Port int `json:"port,omitempty"`
+	Host string `json:"host,omitempty"`
+
+	CertDir string `json:"certDir,omitempty"`
+}
+
+type ControllerNameConfigurationHealth struct {
+	HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`
+
+	ReadinessEndpointName string `json:"readinessEndpointName,omitempty"`
+	LivenessEndpointName string `json:"livenessEndpointName,omitempty"`
+}
+
+type ControllerNameConfiguration struct {
+  metav1.TypeMeta
+
+  Spec ControllerNameConfigurationSpec `json:"spec"`
+}
+```
+
+With the above `struct` we would need to implement the getters to satisfy the `ManagerConfiguration` interface, these could be automatically generated when you `init` or `create componentconfig` from the `kubebuilder` CLI. This could be something as simple as the following.
+
+```golang
+package config
+
+import (
+  "time"
+) 
+
+func (in *ControllerNameConfiguration) GetSyncPeriod() *time.Duration {}
+func (in *ControllerNameConfiguration) GetLeaderElection() bool {}
+func (in *ControllerNameConfiguration) GetLeaderElectionNamespace() string {}
+func (in *ControllerNameConfiguration) GetLeaderElectionID() string {}
+
+func (in *ControllerNameConfiguration) GetLeaseDuration() *time.Duration {}
+func (in *ControllerNameConfiguration) GetRenewDeadline() *time.Duration {}
+func (in *ControllerNameConfiguration) GetRetryPeriod() *time.Duration {}
+
+func (in *ControllerNameConfiguration) GetNamespace() string {}
+func (in *ControllerNameConfiguration) GetMetricsBindAddress() string {}
+func (in *ControllerNameConfiguration) GetHealthProbeBindAddress() string {}
+
+func (in *ControllerNameConfiguration) GetReadinessEndpointName() string {}
+func (in *ControllerNameConfiguration) GetLivenessEndpointName() string {}
+
+func (in *ControllerNameConfiguration) GetPort() int {}
+func (in *ControllerNameConfiguration) GetHost() string {}
+
+func (in *ControllerNameConfiguration) GetCertDir() string {}
+```
+
+Besides the implementation of the `ComponentConfig` The controller author as it stands would also need to implement the unmarshalling of the `ConfigMap` into the `ComponentConfig`, for this `controller-runtime` could expose helper methods to load a file from disk, unmarshal to the struct and pass the pointer into the `NewFromComponentConfig()` to return the `ctrl.Manager`
+
+## User Stories
+
+### Controller Author with `controller-runtime`
+
+- Implement `ComponentConfig` type
+- Implement `ManagerConfiguration` interface for `ComponentConfig` object
+- Set up `ConfigMap` unmarshalling into `ComponentConfig` type
+- Initialize `ctrl.Manager` with `NewFromComponentConfig`
+- Build custom controller as usual
+
+### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`)
+
+- Initialize `kubebuilder` project using `--component-config-name=XYZConfiguration`
+- Build custom controller as usual
+
+### Controller User without modifications to config
+
+_Provided that the controller provides manifests_
+
+- Apply the controller to the cluster
+- Deploy custom resources
+
+### Controller User with modifications to config
+
+- _Following from previous example without changes_
+- Create a new `ConfigMap` for changes
+- Modify the `controller-runtime` pod to use the new `ConfigMap`
+- Apply the controller to the cluster
+- Deploy custom resources
+
+
+## Risks and Mitigations
+
+- Given that this isn't changing the core Manager initialization for `controller-runtime` it's fairly low risk
+
+## Alternatives
+
+* `NewFromComponentConfig()` could load the object from disk based on the file name and hydrate the `ComponentConfig` type.
+
+## Implementation History
+
+- [x] 02/19/2020: Proposed idea in an issue or [community meeting]
+- [x] 02/24/2020: Proposal submitted to `controller-runtime`
+- [x] 03/02/2020: Updated with default `DefaultControllerConfiguration`
+
+
+<!-- Links -->
+[community meeting]: https://docs.google.com/document/d/1Ih-2cgg1bUrLwLVTB9tADlPcVdgnuMNBGbUl4D-0TIk