Skip to content

Commit 1740c99

Browse files
christopherheinchristopherhein
committed
adding defaultcontrollercomponentconfig design
Signed-off-by: Chris Hein <[email protected]>
1 parent 4ec6da5 commit 1740c99

File tree

1 file changed

+113
-34
lines changed

1 file changed

+113
-34
lines changed

‎designs/component-config.md

Lines changed: 113 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# ComponentConfig Controller Runtime Support
2-
32
Author: @christopherhein
4-
Last Updated on: 02/24/2020
3+
4+
Last Updated on: 03/02/2020
55

66
## Table of Contents
77

@@ -15,46 +15,51 @@ Last Updated on: 02/24/2020
1515
* [Non-Goals/Future Work](#non-goalsfuture-work)
1616
* [Proposal](#proposal)
1717
* [ComponentConfig Load Order](#componentconfig-load-order)
18-
* [User Stories](#user-stories)
19-
* [Controller Author with controller-runtime](#controller-author-with-controller-runtime)
20-
* [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder)
21-
* [Controller User without modifications to config](#controller-user-without-modifications-to-config)
22-
* [Controller User with modifications to config](#controller-user-with-modifications-to-config)
23-
* [Risks and Mitigations](#risks-and-mitigations)
18+
* [Default ComponentConfig Type](#default-componentconfig-type)
19+
* [Caveats](#caveats)
20+
* [Kubebuilder Scaffolding Example](#kubebuilder-scaffolding-example)
21+
* [User Stories](#user-stories)
22+
* [Controller Author with controller-runtime](#controller-author-with-controller-runtime)
23+
* [Controller Author with kubebuilder (tbd proposal for kubebuilder)](#controller-author-with-kubebuilder-tbd-proposal-for-kubebuilder)
24+
* [Controller User without modifications to config](#controller-user-without-modifications-to-config)
25+
* [Controller User with modifications to config](#controller-user-with-modifications-to-config)
26+
* [Risks and Mitigations](#risks-and-mitigations)
2427
* [Alternatives](#alternatives)
2528
* [Implementation History](#implementation-history)
2629

2730
<!--te-->
2831

2932
## Summary
3033

31-
Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hard coding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized along the pattern of [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` patterns into `controller-runtime` to allow controller authors to make `go` types backed by apimachinery to unmarshal and configure the `ctrl.Manager` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args.
34+
Currently controllers that use `controller-runtime` need to configure the `ctrl.Manager` by using flags or hardcoding values into the initialization methods. Core Kubernetes has started to move away from using flags as a mechanism for configuring components and standardized along the pattern of [`ComponentConfig` or Versioned Component Configuration Files](https://docs.google.com/document/d/1FdaEJUEh091qf5B98HM6_8MS764iXrxxigNIdwHYW9c/edit). This proposal is to bring `ComponentConfig` patterns into `controller-runtime` to allow controller authors to make `go` types backed by `apimachinery` to unmarshal and configure the `ctrl.Manager` reducing the flags and allowing code based tools to easily configure controllers instead of requiring them to mutate CLI args.
3235

3336

3437
## Motivation
3538

3639
This change is important because:
3740
- it will help make it easier for controllers to be configured by other machine processes
38-
- it will reduce the upfront flags required to start a controller
41+
- it will reduce the required flags required to start a controller
3942
- allow for more configuration types which flags don't natively support
4043
- allow using and upgrading older configurations avoiding breaking changes in flags
4144

4245
### Links to Open Issues
4346

44-
- [Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518)
45-
- [Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207)
46-
- [Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722)
47+
- [#518 Provide a ComponentConfig to tweak the Manager](https://github.com/kubernetes-sigs/controller-runtime/issues/518)
48+
- [#207 Reduce command line flag boilerplate](https://github.com/kubernetes-sigs/controller-runtime/issues/207)
49+
- [#722 Implement ComponentConfig by default & stop using (most) flags](https://github.com/kubernetes-sigs/kubebuilder/issues/722)
4750

4851
### Goals
4952

5053
- Provide an interface for pulling configuration data out of exposed `ComponentConfig` types (see below for implementation)
5154
- Provide a new `ctrl.NewFromComponentConfig()` function for initializing a manager
55+
- Provide a `DefaultControllerConfig` to make the switch easier
5256

5357
### Non-Goals/Future Work
5458

5559
- `kubebuilder` implementation and design in another PR
5660
- Changing the default `controller-runtime` implementation
5761
- Dynamically reloading ComponentConfig object
62+
- Providing `flags` interface and overrides
5863

5964
## Proposal
6065

@@ -111,6 +116,89 @@ func NewFromComponentConfig(config *rest.Config, scheme *runtime.Scheme, manager
111116

112117
![ComponentConfig Load Order](/designs/images/component-config-load.png)
113118

119+
#### Default ComponentConfig Type
120+
121+
To enable `controller-runtime` to have a default `ComponentConfig` struct which can be used instead of requiring each controller or extension to build it's own `ComponentConfig` type, we can create a `DefaultControllerConfiguration` type which can exist in `pkg/api/v1alpha1/types.go`. This will allow the controller authors to use this before needing to implement their own type with additional configs.
122+
123+
```golang
124+
package v1alpha1
125+
126+
import (
127+
"time"
128+
129+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
130+
configv1alpha1 "sigs.k8s.io/component-base/config/v1alpha1"
131+
)
132+
133+
// DefaultControllerConfigurationSpec defines the desired state of DefaultControllerConfiguration
134+
type DefaultControllerConfigurationSpec struct {
135+
SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
136+
137+
LeaderElection configv1alpha1.LeaderElectionConfiguration `json:"leaderElection,omitempty"`
138+
139+
MetricsBindAddress string `json:"metricsBindAddress,omitempty"`
140+
141+
Health DefaultControllerConfigurationHealth `json:"health,omitempty"`
142+
143+
Port *int `json:"port,omitempty"`
144+
Host string `json:"host,omitempty"`
145+
146+
CertDir string `json:"certDir,omitempty"`
147+
}
148+
149+
// DefaultControllerConfigurationHealth defines the health configs
150+
type DefaultControllerConfigurationHealth struct {
151+
HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`
152+
153+
ReadinessEndpointName string `json:"readinessEndpointName,omitempty"`
154+
LivenessEndpointName string `json:"livenessEndpointName,omitempty"`
155+
}
156+
157+
// DefaultControllerConfiguration is the Schema for the DefaultControllerConfigurations API
158+
type DefaultControllerConfiguration struct {
159+
metav1.TypeMeta `json:",inline"`
160+
161+
Spec DefaultControllerConfigurationSpec `json:"spec,omitempty"`
162+
}
163+
```
164+
165+
This would allow a controller author to use this struct with any config that supports the json/yaml structure. For example a controller author could define their `Kind` as `FoobarControllerConfiguration` and have it defined as the following.
166+
167+
```yaml
168+
# config.yaml
169+
apiVersion: somedomain.io/v1alpha1
170+
kind: FoobarControllerConfiguration
171+
spec:
172+
port: 9443
173+
metricsBindAddress: ":8080"
174+
leaderElection:
175+
leaderElect: false
176+
```
177+
178+
Given the following config and `DefaultControllerConfiguration` we'd be able to initialize the controller using the following.
179+
180+
181+
```golang
182+
componentconfig := &defaultv1alpha1.DefaultControllerConfiguration{}
183+
if err := manager.DecodeComponentConfigFileInto(codecs, "config.yaml", componentconfig); err != nil {
184+
// ...
185+
}
186+
187+
mgr, err := ctrl.NewManagerFromComponentConfig(ctrl.GetConfigOrDie(), scheme, componentconfig)
188+
if err != nil {
189+
// ...
190+
}
191+
```
192+
193+
The above example uses `config.yaml` which is the name of the file to load the configuration bytes from and uses `codecs` to pass in the specific serializers, eg `serializer.NewCodecFactory(scheme)`. This will allow the configuration to be unmarshalled into the `runtime.Object` type and passed into the
194+
`ctrl.NewManagerFromComponentConfig()` as a `ManagerConfiguration` interface.
195+
196+
##### Caveats
197+
198+
> ⚠️ Using `DecodeComponentConfigFileInto` does not support the overrides for flags, this is something that is left up to the controller author since they could be using many different flagging interfaces. eg [`flag`](https://golang.org/pkg/flag/), [`pflag`](https://godoc.org/github.com/spf13/pflag), [`flagnum`](https://godoc.org/github.com/luci/luci-go/common/flag/flagenum) and `controller-runtime` should be agnostic to the CLI implementation.
199+
200+
#### Kubebuilder Scaffolding Example
201+
114202
Within a separate design (link once created) this will require controller authors to generate a type that implements the `ManagerConfiguration` interface. The following is a sample of what this looks like:
115203

116204
```golang
@@ -120,16 +208,13 @@ import (
120208
"time"
121209
122210
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
211+
configv1alpha1 "sigs.k8s.io/component-base/config/v1alpha1"
123212
)
124213
125214
type ControllerNameConfigurationSpec struct {
126-
LeaderElection ControllerNameConfigurationLeaderElection `json:"leaderElection,omitempty"`
127-
128-
LeaseDuration *time.Duration `json:"leaseDuration,omitempty"`
129-
RenewDeadline *time.Duration `json:"renewDeadline,omitempty"`
130-
RetryPeriod *time.Duration`json:"retryPeriod,omitempty"`
131-
132-
Namespace string `json:"namespace,omitempty"`
215+
SyncPeriod *time.Duration `json:"syncPeriod,omitempty"`
216+
217+
LeaderElection configv1alpha1.LeaderElectionConfiguration `json:"leaderElection,omitempty"`
133218

134219
MetricsBindAddress string `json:"metricsBindAddress,omitempty"`
135220

@@ -141,12 +226,6 @@ type ControllerNameConfigurationSpec struct {
141226
CertDir string `json:"certDir,omitempty"`
142227
}
143228

144-
type ControllerNameConfigurationLeaderElection struct {
145-
Enabled bool `json:"enabled,omitempty"`
146-
Namespace string `json:"namespace,omitempty"`
147-
ID string `json:"id,omitempty"`
148-
}
149-
150229
type ControllerNameConfigurationHealth struct {
151230
HealthProbeBindAddress string `json:"healthProbeBindAddress,omitempty"`
152231

@@ -194,29 +273,29 @@ func (in *ControllerNameConfiguration) GetCertDir() string {}
194273

195274
Besides the implementation of the `ComponentConfig` The controller author as it stands would also need to implement the unmarshalling of the `ConfigMap` into the `ComponentConfig`, for this `controller-runtime` could expose helper methods to load a file from disk, unmarshal to the struct and pass the pointer into the `NewFromComponentConfig()` to return the `ctrl.Manager`
196275

197-
### User Stories
276+
## User Stories
198277

199-
#### Controller Author with `controller-runtime`
278+
### Controller Author with `controller-runtime`
200279

201280
- Implement `ComponentConfig` type
202281
- Implement `ManagerConfiguration` interface for `ComponentConfig` object
203282
- Set up `ConfigMap` unmarshalling into `ComponentConfig` type
204283
- Initialize `ctrl.Manager` with `NewFromComponentConfig`
205284
- Build custom controller as usual
206285

207-
#### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`)
286+
### Controller Author with `kubebuilder` (tbd proposal for `kubebuilder`)
208287

209288
- Initialize `kubebuilder` project using `--component-config-name=XYZConfiguration`
210289
- Build custom controller as usual
211290

212-
#### Controller User without modifications to config
291+
### Controller User without modifications to config
213292

214293
_Provided that the controller provides manifests_
215294

216295
- Apply the controller to the cluster
217296
- Deploy custom resources
218297

219-
#### Controller User with modifications to config
298+
### Controller User with modifications to config
220299

221300
- _Following from previous example without changes_
222301
- Create a new `ConfigMap` for changes
@@ -225,19 +304,19 @@ _Provided that the controller provides manifests_
225304
- Deploy custom resources
226305

227306

228-
### Risks and Mitigations
307+
## Risks and Mitigations
229308

230309
- Given that this isn't changing the core Manager initialization for `controller-runtime` it's fairly low risk
231-
- If the underlaying `ctrl.Options{}`
232310

233311
## Alternatives
234312

235-
* `NewFromComponentConfig()` could load the object from disk and hydrate the `ComponentConfig` type
313+
* `NewFromComponentConfig()` could load the object from disk based on the file name and hydrate the `ComponentConfig` type.
236314

237315
## Implementation History
238316

239317
- [x] 02/19/2020: Proposed idea in an issue or [community meeting]
240318
- [x] 02/24/2020: Proposal submitted to `controller-runtime`
319+
- [x] 03/02/2020: Updated with default `DefaultControllerConfiguration`
241320

242321

243322
<!-- Links -->

0 commit comments

Comments
 (0)