Skip to content
This repository was archived by the owner on Apr 17, 2025. It is now read-only.

Commit 76a6c34

Browse files
zfrhvzfrhv
committed
adding hrq documentation
Signed-off-by: unknown <[email protected]>
1 parent fa104e8 commit 76a6c34

File tree

113 files changed

+14494
-2354
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

113 files changed

+14494
-2354
lines changed

‎docs/user-guide/concepts.md

Lines changed: 67 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ namespaces are, and _why_ they behave the way they do.
1313
* [Basic concepts](#basic)
1414
* [Parents, children, trees and forests](#basic-trees)
1515
* [Full namespaces and subnamespaces](#basic-subns)
16+
* [Hierarchical resource quotas (HRQs)](#hrq)
1617
* [Policy inheritance and object propagation](#basic-propagation)
1718
* [Tree labels and non-propagated policies](#basic-labels)
1819
* [Exceptions and propagation control](#basic-exceptions)
@@ -87,6 +88,8 @@ namespaces:
8788
as isolation units for their own services. However, namespace creation is a
8889
privileged cluster-level operation, and you typically want to control this
8990
privilege very closely.
91+
* You might want to give some amount of resources to a team (similar to `ResourceQuota`), and they can
92+
distribute those resources between their subnamespaces.
9093
* Finally, you might want to avoid having to find unique names for every
9194
namespace in the cluster.
9295

@@ -218,6 +221,68 @@ probably not a great idea.
218221
You can create a subnamespace from the command line via `kubectl hns create child
219222
-n parent`.
220223

224+
<a name="hrq">
225+
226+
### Hierarchical resource quotas (HRQs)
227+
228+
***Hierarchical resource quotas are beta in HNC v1.1***
229+
230+
When you want to give some amount of resources to `team-a`, and want them to be able to
231+
flexibly use resources in any of their subnamespaces, you create a `HierarchicalResourceQuota`
232+
in namespace `team-a`. The sum of all resources from all the subnamespaces of the
233+
members wont be over the amount of resources that is configured in
234+
`HierarchicalResourceQuota` of namespace `team-a`. All of the reasources of `team-a` are
235+
equally shared between the applications in their subnamespaces, which is very efficient.
236+
237+
In addition, you can let an org or team's admin create their own hierarchical
238+
quotas without violating the overall HRQ for their org or team. For example,
239+
if you start with the following structure:
240+
```
241+
company-a
242+
├── organization-a
243+
│ ├── org-a-team-1
244+
│ ├── org-a-team-2
245+
│ ...
246+
├── organization-b
247+
│ ├── org-b-team-1
248+
│ ├── org-b-team-2
249+
│ ...
250+
...
251+
```
252+
Instead of each team asking from the `cluster-admin` to modify their `ResourceQuota`,
253+
you can insert an additional "policy" namespace above each level to hold
254+
the policy objects (like hierarchical quota) that the sub-admin _cannot_
255+
change, while giving them permission to create their own quotas in the
256+
lower level namespaces, like this:
257+
```
258+
company-a-policy
259+
└── company-a
260+
```
261+
And put the resources HRQ in the `company-a-policy` namespace. This will restrict
262+
whole `company-a` to the amount of resources that they are paying for. Then `company-a`
263+
can do similar HRQ with their organizations:
264+
```
265+
company-a-policy (has HRQ)
266+
└── company-a
267+
├── org-a-policy (has HRQ)
268+
│ └── organization-a
269+
├── org-b-policy (has HRQ)
270+
│ └── organization-b
271+
...
272+
```
273+
Lower-level quotas cannot override more restrictive quotas from ancestor namespaces;
274+
the most restrictive quota always wins.
275+
This way each individual can fairly and securely distribute their resources across
276+
their members.
277+
278+
To implement hierarchical quotas, HNC automatically creates `ResourceQuota` objects in each
279+
affected namespace. This is a part of the internal implementation and shouldn't be modified or
280+
inspected. Use the `kubectl hns hrq` command to inspect hierarchical quotas,
281+
or look at the `HierarchicalResourceQuota` object in the ancestor
282+
namespaces.
283+
284+
Note: Decimal point values cannot be specified in HRQ (you can't do `cpu: 1.5` but you can do `cpu: "1.5"` or `cpu: 1500m`). See [#292](https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/292)
285+
221286
<a name="basic-propagation">
222287

223288
### Policy inheritance and object propagation
@@ -327,7 +392,7 @@ HNC typically propagates _all_ objects of a [specified type](how-to.md#admin-res
327392
from ancestor namespaces to descendant namespaces. However, sometimes this is
328393
too restrictive, and you need to create ***exceptions*** to certain policies. For example:
329394

330-
* A ResourceQuota was propagated to many children, but one child namespace now
395+
* A `ResourceQuota` was propagated to many children, but one child namespace now
331396
has higher requirements than the rest. Rather than getting rid of the quota in
332397
the parent namespace, or raising the limit for everyone, you can stop the
333398
quota in the parent from being propagated to that _one_ child namespace,
@@ -345,7 +410,7 @@ an object can also control how it is propagated to descendant namespaces.
345410
If you modify an exception - for example, by removing it - this could cause
346411
the object to be propagated to descendants from which it had previously been
347412
excluded. This could cause you to accidentally overwrite objects that were
348-
intended to be exceptions from higher-level policies, like the ResourceQuota
413+
intended to be exceptions from higher-level policies, like the `ResourceQuota`
349414
in the example above. To prevent this, if modifying an exception would cause
350415
HNC to overwrite another object, HNC’s admission controllers will prevent you
351416
from modifying the object, and will identify the objects that would have been

‎docs/user-guide/how-to.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ This document describes common tasks you might want to accomplish using HNC.
1010
* [Create a subnamespace](#use-subns-create)
1111
* [Inspect namespace hierarchies](#use-inspect)
1212
* [Propagating policies across namespaces](#use-propagate)
13+
* [Apply hierarchical resource quotas (HRQs)](#use-hrq)
1314
* [Select namespaces based on their hierarchies](#use-select)
1415
* [Delete a subnamespace](#use-subns-delete)
1516
* [Organize full namespaces into a hierarchy](#use-full)
@@ -186,6 +187,19 @@ permissions. To understand why an object is not being propagated to a namespace,
186187
use `kubectl hns describe <ns>`, where `<ns>` is either the source (ancestor) or
187188
destination (descendant) namespace.
188189

190+
<a name="use-hrq"/>
191+
192+
### Limit Resources over parent namespaces
193+
194+
***Hierarchical resource quotas are beta in HNC v1.1***
195+
196+
HNC has an object called `HierarchicalResourceQuota` which is a drop-in replacement for `ResourceQuota`
197+
but across all the namespaces in a hierarchy. It allows you to distribute your resources between
198+
teams, and those teams can distribute their resources between their subteams.
199+
[Learn how it works](concepts.md#hierarchical-resource-quota) or see an [quickstart example](quickstart.md#hrq)
200+
201+
Note: Decimal point values cannot be specified in HRQ (you can't do `cpu: 1.5` but you can do `cpu: "1.5"` or `cpu: 1500m`). See [#292](https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/292)
202+
189203
<a name="use-select"/>
190204

191205
### Select namespaces based on their hierarchies

‎docs/user-guide/quickstart.md

Lines changed: 100 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ also to all contributors since then.
2121
* [Basic functionality](#basic)
2222
* [Propagating different resources](#resources)
2323
* [Hierarchical network policy](#netpol)
24+
* [Hierarchical resource quotas](#hrq)
2425
* [Subnamespaces deep-dive](#subns)
2526
* [Keeping objects out of certain namespaces](#exceptions)
2627

@@ -341,7 +342,7 @@ Now we'll create a default network policy that blocks any ingress from other
341342
namespaces:
342343

343344
```bash
344-
cat << EOF | kubectl apply -f -
345+
kubectl apply -f - << EOF
345346
kind: NetworkPolicy
346347
apiVersion: networking.k8s.io/v1
347348
metadata:
@@ -394,7 +395,7 @@ other. We do this by creating a policy that selects namespaces based on the
394395
that is automatically added by the HNC.
395396

396397
```bash
397-
cat << EOF | kubectl apply -f -
398+
kubectl apply -f - << EOF
398399
kind: NetworkPolicy
399400
apiVersion: networking.k8s.io/v1
400401
metadata:
@@ -458,6 +459,102 @@ kubectl delete svc s2 -n service-2
458459
kubectl delete pods s2 -n service-2
459460
```
460461

462+
<a name="hrq"/>
463+
464+
### Hierarchical resource quotas
465+
466+
***Hierarchical resource quotas are beta in HNC v1.1***
467+
468+
_Will demonstrate: Create and delete [HierarchicalResourceQuota](concepts.md#hierarchical-resource-quota)._
469+
470+
Let's assume you own _acme-org_ from the previous example:
471+
```
472+
acme-org
473+
├── [s] team-a
474+
└── [s] team-b
475+
```
476+
477+
You can create a `HierarchicalResourceQuota` in namespace `acme-org`, and the sum of
478+
all subnamespaces resource usage cant go over what is configured in the hrq.
479+
480+
Creating the hrq:
481+
```bash
482+
kubectl apply -f - << EOF
483+
kind: HierarchicalResourceQuota
484+
apiVersion: hnc.x-k8s.io/v1alpha2
485+
metadata:
486+
name: acme-org-hrq
487+
namespace: acme-org
488+
spec:
489+
hard:
490+
requests.cpu: 1
491+
requests.memory: 2Gi
492+
limits.cpu: 2
493+
limits.memory: 4Gi
494+
services: 1
495+
EOF
496+
```
497+
498+
Lets create Service in namespace `team-a`:
499+
```bash
500+
kubectl apply -f - << EOF
501+
kind: Service
502+
apiVersion: v1
503+
metadata:
504+
name: team-a-svc
505+
namespace: team-a
506+
spec:
507+
selector:
508+
place: holder
509+
ports:
510+
- protocol: TCP
511+
port: 80
512+
targetport: 80
513+
EOF
514+
```
515+
516+
And when we try to create Service in namespace `team-b`:
517+
```bash
518+
kubectl apply -f - << EOF
519+
kind: Service
520+
apiVersion: v1
521+
metadata:
522+
name: team-b-svc
523+
namespace: team-b
524+
spec:
525+
selector:
526+
place: holder
527+
ports:
528+
- protocol: TCP
529+
port: 80
530+
targetport: 80
531+
EOF
532+
```
533+
We get an error:
534+
```
535+
Error from server (Forbidden):
536+
error when creating "STDIN":
537+
admission webhood "resourcesquotasstatus.hnc.x-k8s.io" denied the request:
538+
exceeded hierarchical quota in namespace "acme-org":
539+
"acme-org-hrq", requested: services=1, used: services=1, limited: services=1
540+
```
541+
542+
To view the HRQ usage, simply run:
543+
```bash
544+
kubectl hns hrq -n acme-org
545+
```
546+
you can also view in all namespace:
547+
```bash
548+
kubectl hns hrq --all-namespaces
549+
```
550+
551+
and you can delete the hrq via simply deleting the CR:
552+
```bash
553+
kubectl delete hrq acme-org-hrq -n acme-org
554+
```
555+
556+
Note: Decimal point values cannot be specified (you can't do `cpu: 1.5` but you can do `cpu: "1.5"` or `cpu: 1500m`). See [#292](https://github.com/kubernetes-sigs/hierarchical-namespaces/issues/292)
557+
461558
<a name="subns"/>
462559

463560
### Subnamespaces deep dive
@@ -672,7 +769,7 @@ Of course, the annotation can also be part of the object when you create it:
672769

673770
```bash
674771
kubectl delete secret my-secret -n acme-org
675-
cat << EOF | k create -f -
772+
kubectl create -f - << EOF
676773
apiVersion: v1
677774
kind: Secret
678775
metadata:

‎go.mod

Lines changed: 35 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ require (
2020
k8s.io/apimachinery v0.23.2
2121
k8s.io/cli-runtime v0.23.2
2222
k8s.io/client-go v0.23.2
23+
k8s.io/kubernetes v1.23.2
2324
sigs.k8s.io/controller-runtime v0.11.0
2425
sigs.k8s.io/controller-tools v0.8.0
2526
)
@@ -35,7 +36,7 @@ require (
3536
github.com/BurntSushi/toml v1.2.1 // indirect
3637
github.com/PuerkitoBio/purell v1.1.1 // indirect
3738
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 // indirect
38-
github.com/aws/aws-sdk-go v1.23.20 // indirect
39+
github.com/aws/aws-sdk-go v1.38.49 // indirect
3940
github.com/beorn7/perks v1.0.1 // indirect
4041
github.com/census-instrumentation/opencensus-proto v0.2.1 // indirect
4142
github.com/cespare/xxhash/v2 v2.1.1 // indirect
@@ -63,10 +64,10 @@ require (
6364
github.com/googleapis/gnostic v0.5.5 // indirect
6465
github.com/imdario/mergo v0.3.12 // indirect
6566
github.com/inconshreveable/mousetrap v1.0.0 // indirect
66-
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af // indirect
67+
github.com/jmespath/go-jmespath v0.4.0 // indirect
6768
github.com/josharian/intern v1.0.0 // indirect
6869
github.com/json-iterator/go v1.1.12 // indirect
69-
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect
70+
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
7071
github.com/mailru/easyjson v0.7.6 // indirect
7172
github.com/mattn/go-colorable v0.1.8 // indirect
7273
github.com/mattn/go-isatty v0.0.12 // indirect
@@ -99,9 +100,8 @@ require (
99100
golang.org/x/text v0.5.0
100101
golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect
101102
golang.org/x/tools v0.4.1-0.20221208213631-3f74d914ae6d // indirect
102-
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
103103
gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect
104-
google.golang.org/api v0.44.0 // indirect
104+
google.golang.org/api v0.46.0 // indirect
105105
google.golang.org/appengine v1.6.7 // indirect
106106
google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2 // indirect
107107
google.golang.org/grpc v1.40.0 // indirect
@@ -120,7 +120,34 @@ require (
120120
sigs.k8s.io/yaml v1.3.0 // indirect
121121
)
122122

123-
require (
124-
github.com/spf13/afero v1.6.0 // indirect
125-
sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20230208013708-22718275bffe // indirect
123+
require sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20230208013708-22718275bffe
124+
125+
require github.com/spf13/afero v1.6.0 // indirect
126+
127+
replace (
128+
k8s.io/api => k8s.io/api v0.23.2
129+
k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.23.2
130+
k8s.io/apimachinery => k8s.io/apimachinery v0.23.2
131+
k8s.io/apiserver => k8s.io/apiserver v0.23.2
132+
k8s.io/cli-runtime => k8s.io/cli-runtime v0.23.2
133+
k8s.io/client-go => k8s.io/client-go v0.23.2
134+
k8s.io/cloud-provider => k8s.io/cloud-provider v0.23.2
135+
k8s.io/cluster-bootstrap => k8s.io/cluster-bootstrap v0.23.2
136+
k8s.io/code-generator => k8s.io/code-generator v0.23.2
137+
k8s.io/component-base => k8s.io/component-base v0.23.2
138+
k8s.io/component-helpers => k8s.io/component-helpers v0.23.2
139+
k8s.io/controller-manager => k8s.io/controller-manager v0.23.2
140+
k8s.io/cri-api => k8s.io/cri-api v0.23.2
141+
k8s.io/csi-translation-lib => k8s.io/csi-translation-lib v0.23.2
142+
k8s.io/kube-aggregator => k8s.io/kube-aggregator v0.23.2
143+
k8s.io/kube-controller-manager => k8s.io/kube-controller-manager v0.23.2
144+
k8s.io/kube-proxy => k8s.io/kube-proxy v0.23.2
145+
k8s.io/kube-scheduler => k8s.io/kube-scheduler v0.23.2
146+
k8s.io/kubectl => k8s.io/kubectl v0.23.2
147+
k8s.io/kubelet => k8s.io/kubelet v0.23.2
148+
k8s.io/legacy-cloud-providers => k8s.io/legacy-cloud-providers v0.23.2
149+
k8s.io/metrics => k8s.io/metrics v0.23.2
150+
k8s.io/mount-utils => k8s.io/mount-utils v0.23.2
151+
k8s.io/pod-security-admission => k8s.io/pod-security-admission v0.23.2
152+
k8s.io/sample-apiserver => k8s.io/sample-apiserver v0.23.2
126153
)

0 commit comments

Comments
 (0)