You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**ACTION REQUIRED:** In order to merge code into a release, there must be an issue in [kubernetes/enhancements] referencing this KEP and targeting a release milestone **before [Enhancement Freeze](https://github.com/kubernetes/sig-release/tree/master/releases)
23
-
of the targeted release**.
24
-
25
-
For enhancements that make changes to code or processes/procedures in core Kubernetes i.e., [kubernetes/kubernetes], we require the following Release Signoff checklist to be completed.
26
-
27
-
Check these off as they are completed for the Release Team to track. These checklist items _must_ be updated for the enhancement to be released.
28
-
29
-
-[ ] kubernetes/enhancements issue in release milestone, which links to KEP (this should be a link to the KEP location in kubernetes/enhancements, not the initial KEP PR)
30
-
-[X] KEP approvers have set the KEP status to `implementable`
31
-
-[ ] Design details are appropriately documented
32
-
-[ ] Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
33
-
-[ ] Graduation criteria is in place
35
+
Items marked with (R) are required *prior to targeting to a milestone / release*.
36
+
37
+
-[ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
38
+
-[ ] (R) KEP approvers have approved the KEP status as `implementable`
39
+
-[ ] (R) Design details are appropriately documented
40
+
-[ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
41
+
-[ ] e2e Tests for all Beta API Operations (endpoints)
42
+
-[ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
43
+
-[ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
44
+
-[ ] (R) Graduation criteria is in place
45
+
-[ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
46
+
-[ ] (R) Production readiness review completed
47
+
-[ ] (R) Production readiness review approved
34
48
-[ ] "Implementation History" section is up-to-date for milestone
35
49
-[ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
36
-
-[ ] Supporting documentation e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
37
-
38
-
**Note:** Any PRs to move a KEP to `implementable` or significant changes once it is marked `implementable` should be approved by each of the KEP approvers. If any of those approvers is no longer appropriate than changes to that list should be approved by the remaining approvers and/or the owning SIG (or SIG-arch for cross cutting KEPs).
39
-
40
-
**Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone.
50
+
-[ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
@@ -51,68 +61,63 @@ This document is intended to propose a process and desired goals by which kube-p
51
61
## Motivation
52
62
53
63
kube-proxy is a component, that is present in almost all Kubernetes clusters in existence.
54
-
Historically speaking, kube-proxy's configuration was supplied by a set of command line flags. Over time, the number of flags grew and they became unwieldy to use and support. Thus, kube-proxy gained component config.
64
+
Historically speaking, kube-proxy's configuration was supplied by a set of command line flags. Over time, the number of flags grew, and they became unwieldy to use and support. Thus, kube-proxy gained component config.
55
65
Initially this was just a large flat object, that was representing the command line flags. However, over time new features were added to it, all while staying as v1alpha1.
56
66
57
67
This resulted in a configuration format, that had various different options grouped together in ways, that made them hard to specify and understand. For example:
58
68
59
-
- Instance local options (such as host name override, bind address, etc.) are in the same flat object as shared between instances options (such as the cluster CIDR, config sync period, etc.).
60
-
- Platform specific options are mixed together. For example, the IPTables rule sync fields are used by the Windows HNS backend for the same purpose.
61
-
- Again, the IPTables rule sync options are used for the Linux legacy user mode proxy, but not for the IPVS mode (where a set of identical options exist, despite the fact, that it too uses some other fields, designed for IPTables).
69
+
- Instance local options (such as hostnameOverride, bindAddress, etc.) are in the same flat object as shared between instances options (such as clusterCIDR, configSyncPeriod, etc.).
70
+
- Platform specific options are marked as generic options (eg. conntrack, oomScoreAdj).
71
+
- Backend agnostic options are marked as backed specific options (eg. syncPeriod, minSyncPeriod).
72
+
- Options specific to a backend are used by other backends (eg. masqueradeBit and masqueradeAll).
73
+
74
+
[kubernetes/issues/117909](https://github.com/kubernetes/kubernetes/issues/117909) captures all the mis configurations in details.
62
75
63
76
Clearly, this made the configuration both hard to use and to maintain. Therefore, a plan to restructure and stabilize the config format is needed.
64
77
65
78
### Goals
66
79
67
-
- To cleanup the existing config format.
80
+
- To clean up the existing config format.
68
81
- To provide config structure, that is easier for users to understand and use.
69
82
- To distinguish between instance local and shared settings.
70
83
- To allow for the persistence of settings for different platforms (such as Linux and Windows) in a manner that reduces confusion and the possibility of an error.
71
84
- To allow for easier introduction of new proxy backends.
72
-
- To provide users with flexibility, especially with regards to the config source.
85
+
- To provide users with flexibility, especially in regard to the config source.
73
86
74
87
### Non-Goals
75
88
76
89
- To change or implement additional features in kube-proxy.
77
90
- To deal with graduation of any other component of kube-proxy, other than its configuration.
78
-
- To remove most or even all of the command line flags, that have corresponding component config options.
91
+
- To remove most or even all the command line flags, that have corresponding component config options.
79
92
80
93
## Proposal
81
94
82
-
The idea is to conduct the process of graduation to beta in small steps in the span of at least one Kubernetes release cycle. This will be done by creating one or more alpha versions of the config with the last alpha version being copied as v1beta1 after the community is happy with it.
83
-
Each of the sub-sections below can result in a separate alpha version release, although it will be better for users to have no more than a couple of alpha versions past v1alpha1.
84
-
After each alpha version release, the community will gather around for new ideas on how to proceed in the graduation process. If there are viable proposals, this document is updated with an appropriate section(s) below and the new changes are introduced in the form of new alpha version(s).
85
-
The proposed process is similar to the already successfully used one for kubeadm.
86
-
87
-
### Re-encapsulate mode specific options
95
+
The idea is to conduct the process of graduation to beta in small steps in the span of at least one Kubernetes release cycle.
96
+
This will be done by creating one or more alpha versions of the config with the last alpha version being copied as v1beta1 after
97
+
the community is happy with it. Each of the subsections below can result in a separate alpha version release, although it will
98
+
be better for users to have no more than a couple of alpha versions past v1alpha1. After each alpha version release, the community
99
+
will gather around for new ideas on how to proceed in the graduation process. If there are viable proposals, this document is
100
+
updated with an appropriate section(s) below and the new changes are introduced in the form of new alpha version(s). The proposed
101
+
process is similar to the already successfully used one for kubeadm.
88
102
89
103
The current state of the config has proven that:
90
104
- Some options are deemed as mode specific, but are in fact shared between all modes.
91
105
- Some options are placed directly into KubeProxyConfiguration, but are in fact mode specific ones.
92
106
- There are options that are shared between some (but not all) modes. Specific features of the underlying implementation are common and this happens only within the boundaries of the platform (iptables and ipvs modes for example).
93
-
- Although legacy Linux and Windows user mode proxies are separate code bases, they have a common set of options.
94
107
95
108
With that in mind, the following measures are proposed:
96
-
- Mode specific structs are consolidated to not use fields from other mode specific structs.
97
-
- Introduce a single combined legacy user mode proxy struct for both Linux and Windows backends.
98
-
99
-
#### Example
100
-
101
-
```yaml
102
-
commonSetting1: ...
103
-
commonSetting2: ...
104
-
...
105
-
modeA: ...
106
-
modeB: ...
107
-
modeC: ...
108
-
```
109
+
- Create platform subsection for platform specific fields.
110
+
- Move backend-agnostic and platform-agnostic fields from backend section to root section.
111
+
- Move backend-agnostic and platform-specific fields from backend section to relevant platform section.
112
+
- Drop legacy/unused options.
109
113
110
114
### Risks and Mitigations
111
115
112
116
So far, the following risks have been identified:
113
117
- Deviation of the implementation guidelines and bad planning may have the undesired effect of producing bad alpha versions.
114
118
- Bad alpha versions will need good alpha versions to fix them. This will create too many iterations over the API and users may get confused.
115
-
- New and redesigned kube-proxy API versions may cause confusion among users who are used to the v1alpha1 relatively flat, single document design. In particular, multiple YAML documents and structured (as opposed to flat) objects can create confusion as to what option is placed where.
119
+
- New and redesigned kube-proxy API versions may cause confusion among users who are used to the v1alpha1 relatively flat, single document design.
120
+
In particular, multiple YAML documents and structured (as opposed to flat) objects can create confusion as to what option is placed where.
116
121
117
122
The mitigations to those risks:
118
123
- Strict following of the proposals in this document and planning ahead for a release and config cycle.
@@ -122,16 +127,192 @@ The mitigations to those risks:
122
127
123
128
## Design Details
124
129
130
+
125
131
### Test Plan
126
132
127
-
Existing test cases throughout the kube-proxy code base should be adapted to use the latest config version.
128
-
If required, new test cases should also be created.
133
+
[x] I/we understand the owners of the involved components may require updates to
134
+
existing tests to make this code solid enough prior to committing the changes necessary
135
+
to implement this enhancement.
136
+
137
+
##### Prerequisite testing updates
138
+
139
+
140
+
##### Unit tests
141
+
142
+
143
+
##### Integration tests
144
+
145
+
146
+
##### e2e tests
129
147
130
148
### Graduation Criteria
131
149
132
150
The config should be considered graduated to beta if it:
133
-
- is wellstructured with clear boundaries between different proxy mode settings.
151
+
- is well-structured with clear boundaries between different proxy mode settings.
134
152
- allows for easy multi-platform use with less probability of an error.
135
-
- allows for easy distinguishment between instance local and shared settings.
153
+
- allows for easy distinction between instance local and shared settings.
136
154
- is well covered by tests.
137
-
- is well documented. Especially with regards of migrating to it from older versions.
155
+
- is well documented. Especially in regard to migrating to it from older versions.
156
+
157
+
### Upgrade / Downgrade Strategy
158
+
159
+
160
+
### Version Skew Strategy
161
+
162
+
163
+
## Production Readiness Review Questionnaire
164
+
165
+
166
+
### Feature Enablement and Rollback
167
+
168
+
169
+
###### How can this feature be enabled / disabled in a live cluster?
170
+
171
+
<!--
172
+
Pick one of these and delete the rest.
173
+
174
+
Documentation is available on [feature gate lifecycle] and expectations, as
0 commit comments