Adding component config proposal

Signed-off-by: Chris Hein <[email protected]>

Author: christopherhein

designs/component-config.md

@@ -0,0 +1,312 @@
+# ComponentConfig Controller Runtime Support
+Author: @christopherhein
+
+Last Updated on: 03/02/2020
+
+## Table of Contents
+
+<!--ts-->
+   * [ComponentConfig Controller Runtime Support](#componentconfig-controller-runtime-support)
+      * [Table of Contents](#table-of-contents)
+      * [Summary](#summary)
+      * [Motivation](#motivation)
+         * [Links to Open Issues](#links-to-open-issues)
+         * [Goals](#goals)
+         * [Non-Goals/Future Work](#non-goalsfuture-work)
+      * [Proposal](#proposal)
+            * [ComponentConfig Load Order](#componentconfig-load-order)
+            * [Embeddable ComponentConfig Type](#embeddable-componentconfig-type)
+            * [Default ComponentConfig Type](#default-componentconfig-type)
+            * [Using Flags w/ ComponentConfig](#using-flags-w-componentconfig)
+            * [Kubebuilder Scaffolding Example](#kubebuilder-scaffolding-example)
+      * [User Stories](#user-stories)
+         * [Controller Author with controller-runtime](#controller-author-with-controller-runtime)
+         * [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder)
+         * [Controller User without modifications to config](#controller-user-without-modifications-to-config)
+         * [Controller User with modifications to config](#controller-user-with-modifications-to-config)
+      * [Risks and Mitigations](#risks-and-mitigations)
+      * [Alternatives](#alternatives)
+      * [Implementation History](#implementation-history)
+
+<!--te-->
+
+## Summary
+
+Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hardcoding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized on [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` to `controller-runtime` to allow controller authors to make `go` types backed by `apimachinery` to unmarshal and configure the `ctrl.Manager{}` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args.
+
+
+## Motivation
+
+This change is important because: 
+- it will help make it easier for controllers to be configured by other machine processes
+- it will reduce the required flags required to start a controller
+- allow for more configuration types which flags don't natively support
+- allow using and upgrading older configurations avoiding breaking changes in flags
+
+### Links to Open Issues
+
+- [#518 Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518)
+- [#207 Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207)
+- [#722 Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722)
+
+### Goals
+
+- Provide an interface for pulling configuration data out of exposed `ComponentConfig` types (see below for implementation)
+- Provide a new `ctrl.NewFromComponentConfig()` function for initializing a manager
+- Provide an embeddable `RuntimeConfig` type for easily authoring `ComponentConfig` types
+- Provide an `DefaultControllerConfig` to make the switch easier
+
+### Non-Goals/Future Work
+
+- `kubebuilder` implementation and design in another PR
+- Changing the default `controller-runtime` implementation
+- Dynamically reloading `ComponentConfig` object
+- Providing `flags` interface and overrides
+
+## Proposal
+
+The `ctrl.Manager` _SHOULD_ support loading configurations from `ComponentConfig` like objects. 
+An interface for that object with getters for the specific configuration parameters is created to bridge existing patterns.
+
+Without breaking the current `ctrl.NewManager` which uses an exported `ctrl.Options{}` the `manager.go` can expose a new func, `NewFromComponentConfig()` this would be able to loop through the getters to hydrate an internal `ctrl.Options{}` and pass that into `New()`.
+
+```golang
+//pkg/manager/manager.go
+
+// ManagerConfiguration defines what the ComponentConfig object for ControllerRuntime needs to support
+type ManagerConfiguration interface {
+	GetSyncPeriod() *time.Duration
+
+	GetLeaderElection() bool
+	GetLeaderElectionNamespace() string
+	GetLeaderElectionID() string
+
+	GetLeaseDuration() *time.Duration
+	GetRenewDeadline() *time.Duration
+	GetRetryPeriod() *time.Duration
+
+	GetNamespace() string
+	GetMetricsBindAddress() string
+	GetHealthProbeBindAddress() string
+
+	GetReadinessEndpointName() string
+	GetLivenessEndpointName() string
+
+	GetPort() int
+	GetHost() string
+
+	GetCertDir() string
+}
+
+func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme, filename string, managerconfig ManagerConfiguration) (Manager, error) {
+	codecs := serializer.NewCodecFactory(scheme)
+    if err := decodeComponentConfigFileInto(codecs, filename, componentconfig); err != nil {
+		
+	}
+	options := Options{}
+
+	if scheme != nil {
+		options.Scheme = scheme
+	}
+
+	// Loop through getters
+	if managerconfig.GetLeaderElection() {
+			managerconfig.GetLeaderElection()
+	}
+	// ...
+
+	return New(config, options)
+}
+```
+
+#### ComponentConfig Load Order
+
+![ComponentConfig Load Order](/designs/images/component-config-load.png)
+
+#### Embeddable ComponentConfig Type
+
+To make this easier for Controller authors `controller-runtime` can expose a set of `meta.RuntimeConfig` type that can be embedded similar to the way that `k8s.io/apimachinery/pkg/apis/meta/v1` works for `TypeMeta` and `ObjectMeta` these could live in `pkg/api/meta/v1alpha1/types.go`. See the Default ComponentConfig type for and example implementation.
+
+```golang
+// pkg/api/meta/v1alpha1/types.go
+package v1alpha1
+
+import (
+	"time"
+
+	configv1alpha1 "k8s.io/component-base/config/v1alpha1"
+)
+
+// RuntimeConfig defines the embedded RuntimeConfiguration for controller-runtime clients.
+type RuntimeConfig struct {
+	SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
+
+	LeaderElection configv1alpha1.LeaderElectionConfiguration `json:"leaderElection,omitempty"`
+
+	MetricsBindAddress string `json:"metricsBindAddress,omitempty"`
+
+	Health DefaultControllerConfigurationHealth `json:"health,omitempty"`
+
+	Port *int   `json:"port,omitempty"`
+	Host string `json:"host,omitempty"`
+
+	CertDir string `json:"certDir,omitempty"`
+}
+
+// DefaultControllerConfigurationHealth defines the health configs
+type RuntimeConfigHealth struct {
+	HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`
+
+	ReadinessEndpointName string `json:"readinessEndpointName,omitempty"`
+	LivenessEndpointName  string `json:"livenessEndpointName,omitempty"`
+}
+```
+
+
+
+#### Default ComponentConfig Type
+
+To enable `controller-runtime` to have a default `ComponentConfig` struct which can be used instead of requiring each controller or extension to build it's own `ComponentConfig` type, we can create a `DefaultControllerConfiguration` type which can exist in `pkg/api/config/v1alpha1/types.go`. This will allow the controller authors to use this before needing to implement their own type with additional configs.
+
+```golang
+// pkg/api/config/v1alpha1/types.go
+package v1alpha1
+
+import (
+	"time"
+
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	metav1alpha1 "sigs.k8s.io/controller-runtime/pkg/apis/meta/v1alpha1"
+)
+
+// DefaultControllerConfiguration is the Schema for the DefaultControllerConfigurations API
+type DefaultControllerConfiguration struct {
+	metav1.TypeMeta `json:",inline"`
+
+	Spec   metav1alpha1.RuntimeConfig   `json:"spec,omitempty"`
+}
+```
+
+This would allow a controller author to use this struct with any config that supports the json/yaml structure. For example a controller author could define their `Kind` as `FoobarControllerConfiguration` and have it defined as the following.
+
+```yaml
+# config.yaml
+apiVersion: somedomain.io/v1alpha1
+kind: FoobarControllerConfiguration
+spec:
+  port: 9443
+  metricsBindAddress: ":8080"
+  leaderElection:
+    leaderElect: false
+```
+
+Given the following config and `DefaultControllerConfiguration` we'd be able to initialize the controller using the following.
+
+
+```golang
+mgr, err := ctrl.NewManagerFromComponentConfig(ctrl.GetConfigOrDie(), scheme, configname, &defaultv1alpha1.DefaultControllerConfiguration{})
+if err != nil {
+	// ...
+}
+```
+
+The above example uses `configname` which is the name of the file to load the configuration from and uses `scheme` to get the specific serializer, eg `serializer.NewCodecFactory(scheme)`. This will allow the configuration to be unmarshalled into the `runtime.Object` type and passed into the 
+`ctrl.NewManagerFromComponentConfig()` as a `ManagerConfiguration` interface.
+
+#### Using Flags w/ ComponentConfig
+
+Since this design still requires setting up the initial `ComponentConfig` type and passing in a pointer to `ctrl.NewFromComponentConfig()` if you want to allow for the use of flags your controller can use any of the different flagging interfaces. eg [`flag`](https://golang.org/pkg/flag/), [`pflag`](https://godoc.org/github.com/spf13/pflag), [`flagnum`](https://godoc.org/github.com/luci/luci-go/common/flag/flagenum) and set values on the `ComponentConfig` type prior to passing the pointer into the `ctrl.NewFromComponentConfig()`.
+
+```golang
+leaderElect := true
+
+config := &defaultv1alpha1.DefaultControllerConfiguration{
+	Spec: metav1alpha1.RuntimeConfig{
+		LeaderElection: configv1alpha1.LeaderElectionConfiguration{
+			LeaderElect: &leaderElect,
+		},
+	},
+}
+mgr, err := ctrl.NewManagerFromComponentConfig(ctrl.GetConfigOrDie(), scheme, configname, )
+if err != nil {
+	// ...
+}
+```
+
+#### Kubebuilder Scaffolding Example
+
+Within expanded in a separate design _(link once created)_ this will require controller authors to generate a type that implements the `ManagerConfiguration` interface. The following is a sample of what this looks like:
+
+```golang
+package config
+
+import (
+  "time"
+
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	configv1alpha1 "k8s.io/component-base/config/v1alpha1"
+	metav1alpha1 "sigs.k8s.io/controller-runtime/pkg/apis/meta/v1alpha1"
+) 
+
+type ControllerNameConfigurationSpec struct {
+	metav1alpha1.RuntimeConfig `json:",inline"`
+}
+
+type ControllerNameConfiguration struct {
+  metav1.TypeMeta
+
+  Spec ControllerNameConfigurationSpec `json:"spec"`
+}
+```
+
+Usage of this custom `ComponentConfig` type would require then changing the `ctrl.NewFromComponentConfig()` to use the new struct vs the `DefaultControllerConfiguration`.
+
+## User Stories
+
+### Controller Author with `controller-runtime`
+
+- Implement `ComponentConfig` type
+- Implement `ManagerConfiguration` interface for `ComponentConfig` object
+- Set up `ConfigMap` unmarshalling into `ComponentConfig` type
+- Initialize `ctrl.Manager` with `NewFromComponentConfig`
+- Build custom controller as usual
+
+### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`)
+
+- Initialize `kubebuilder` project using `--component-config-name=XYZConfiguration`
+- Build custom controller as usual
+
+### Controller User without modifications to config
+
+_Provided that the controller provides manifests_
+
+- Apply the controller to the cluster
+- Deploy custom resources
+
+### Controller User with modifications to config
+
+- _Following from previous example without changes_
+- Create a new `ConfigMap` for changes
+- Modify the `controller-runtime` pod to use the new `ConfigMap`
+- Apply the controller to the cluster
+- Deploy custom resources
+
+
+## Risks and Mitigations
+
+- Given that this isn't changing the core Manager initialization for `controller-runtime` it's fairly low risk
+
+## Alternatives
+
+* `NewFromComponentConfig()` could load the object from disk based on the file name and hydrate the `ComponentConfig` type.
+
+## Implementation History
+
+- [x] 02/19/2020: Proposed idea in an issue or [community meeting]
+- [x] 02/24/2020: Proposal submitted to `controller-runtime`
+- [x] 03/02/2020: Updated with default `DefaultControllerConfiguration`
+- [x] 03/04/2020: Updated with embeddable `RuntimeConfig`
+
+
+<!-- Links -->
+[community meeting]: https://docs.google.com/document/d/1Ih-2cgg1bUrLwLVTB9tADlPcVdgnuMNBGbUl4D-0TIk