[Umbrella Issue] Java SDK utility tooling improvement plan

[yue9944882]

/cc @brendandburns
/help-wanted

Bringing more use helpful utilities to k-client/java will help polyglot kubernetes developers to extend more projects, operators in the open source world.

Listing a few items we might get start w/ in the foreseeable future:

• Informer Implementation based on Java generic types Java informer portal implementation #461
  • Test planning: Fixes watch conflict handling in informer and more test coverage #522
• Leader election utilities @wu8685 Feat: Structurelize patch object #587
  • basing on configmap api
  • basing on endpoints api
  • basing on coordination api
• Pagination tooling for LIST calls @kondapally1989 Adding pagination utility #527
• RESTMapper for resolving response from discovery API
  • @brendandburns: in favor of a better design
• Discovery utility as replacement of RESTMapper
• SMP Patch client utilities
  • @kondapally1989: another related bug in ExtensionsV1beta1Api.patchNamespacedDeployment throws 500 Internal server error because of wrong content type in header #127 should be solved Fixes patch format to support various patch formats #544
• Working queue for kube-styled controller @adohe add workqueue implementation for kube-style controller #525
• Impersonation header support
• Event recorder equivalence implementation @kondapally1989
• A recommended java kubernetes controller example

[brendandburns]

This generally looks good, but I don't think the RESTMapper stuff is needed/useful?

(and in general I think the RESTMapper stuff in the go client is pretty badly designed anyway, so let's look at the use cases, not just translating go client concepts to Java)

Also, I think that we should consider moving some of this into a different library/directory.

I'd like to ensure we have both a "simple" and an "advanced" client experience, which I think client-go didn't do very well...

[yue9944882]

Also, I think that we should consider moving some of this into a different library/directory.

what about a new java module under the root named client-java-extension or client-java-misc?

which I think client-go didn't do very well...

yeah, agree that we should list "provide better libraries than client-go" as a goal in the picture, especially making use of generic types.

[brendandburns]

That sounds good, how about client-java-extended? or client-java-enhanced?

[karthikkondapally]

@brendandburns <https://github.com/brendandburns> @yue9944882 <https://github.com/yue9944882> will it be okay, if i take a shot at Leader election utilities. A starting point would be of great help. is this similar to https://kubernetes.io/blog/2016/01/simple-leader-election-with-kubernetes/

…

On Fri, Mar 8, 2019 at 11:36 AM Brendan Burns ***@***.***> wrote: That sounds good, how about client-java-extended? or client-java-enhanced? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#523 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AIAJioywYAa_OJHrYVgdHXHDffECMXZ7ks5vUf3UgaJpZM4bihto> .

-- Thanks, * |< ∧ ɤ |_ || I |< *

[yue9944882]

will it be okay, if i take a shot at Leader election utilities.

thank you so much for helping w/ this. matter of fact, we already got an existing utility package for leader-electing from our internal project and it's ready to be turn'd into a pull soon. i don't really mind a duplicate of the work but how do you think about getting on with the pagination? it's an equivalence to one of the client-go's pkg https://github.com/kubernetes/client-go/tree/master/tools/pager. i believe we're actually in a compaign that encouraging developers to apply pagination for every LIST calls, which is critical to large-scale kubernetes cluster. ref: https://groups.google.com/forum/#!topic/kubernetes-sig-api-machinery/SfCIk5MccOc

[karthikkondapally]

@yue9944882 yup i will take up pagination.
i am a total noob in go.(i am trying to understand pager.go 😅)
i am trying to understand the understand requirement and TODO tasks.
list calls in this library already support pagination variables. for example list namespaced pods

V1PodList io.kubernetes.client.apis.CoreV1Api.listNamespacedPod(String namespace,
 Boolean includeUninitialized, String pretty, String _continue, 
String fieldSelector, String labelSelector, Integer limit, 
String resourceVersion, Integer timeoutSeconds, 
Boolean watch)

on type of such calls, do we want Pagination functionality on top of it ?

[brendandburns]

@kondapally1989

I would write an interface that is something like:

Iterator<T> listNamespacedWithPaging<T>(List<T> listFn(...), int pageSize) {
....
}

Basically, you want to transform one large list call into a series of small paginated list calls, and feed those items out as an iterator.

It's actually going to be tricky to do it right, because we don't have good base classes for everything so you will probably need to use reflection, or craft the right interface.

Hope that helps, let me know if there is additional questions.

[yue9944882]

i am trying to understand the understand requirement and TODO tasks. list calls in this library already support pagination variables.

yes. we're actually performing pagination over those list calls over limit and continue queries which is already supported by kube-apiserver or rather etcd. while just as @brendandburns said, what we want is a package that wraps those queries into sth like iterator so that user won't deal w/ the list chunks directly.

[yue9944882]

you will probably need to use reflection, or craft the right interface.

i'm slightly leaning on the latter one. sort of similar to CallGeneratorParams.java.

[yue9944882]

/cc @wu8685 for the leader-election

[karthikkondapally]

SMP Patch client utilities--
does this refer to wrapper to solve #127 and to support different kinds of patches?

[yue9944882]

@kondapally1989 oh this bug, sure we can start by fixing this as the first step. (btw in our internal fork, we workaround'd by hacking the parameter list of patch method to allow users to specify patch type, which i suppose is unacceptable in the community version?) while the item in this issue is about a utility library which helps dynamically construct patch body instead of formatting. sth like:

patch = diff(objectA, objectB)

btw server-side apply is landing k/k in 1.14, that would be a fine replacement for SMP in the future. but am not sure how will that feature work w/ openapi doc :P

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.