Read-only mode implemented with read-only client

[erikgb]

The newly introduced read-only mode to make HNC more highly available is currently implemented with a ReadOnly field on all reconcilers, and every write operation must guarded with a check on the value of this field. This is not robust enough IMO, and should be improved. Contributors not aware of this feature could easily forget adding this guard, leaving it up to reviewers to catch it - creating a troublesome contributor process with a high risk of errors being introduced in HNC. As an example, I dumped into the following code today, which misses this guard:

https://github.com/kubernetes-sigs/hierarchical-namespaces/blob/c5662040fb67c72a22085f50653cf74a1317d5f2/internal/hierarchyconfig/reconciler.go#L141

I suggest to instead implement the read-only mode by supplying the reconcilers with a delegating client, where all writes are implemented as a no-op. A bit like controller-runtime does for their caching client.

[adrianludwin]

+1, I had a similar idea. This would be a great improvement.

[erikgb]

/assign