29
29
openAPIV3Schema :
30
30
description : |-
31
31
InferenceModel is the Schema for the InferenceModels API.
32
- The InferenceModel is intended to represent a model workload within Kubernetes.
32
+ The InferenceModel is intended to represent a model workload (also referred to as a model use case) within Kubernetes.
33
+ The management of the model server is not done by the InferenceModel. Instead, the
34
+ focus of the InferenceModel is to provide the tools needed to effectively manage multiple models
35
+ that share the same base model (currently the focus is LoRA adapters). Fields such as TargetModel
36
+ are intended to simplify A/B testing and version rollout of adapters. While Criticality assists with
37
+ governance of multiplexing many usecases over shared hardware.
33
38
properties :
34
39
apiVersion :
35
40
description : |-
@@ -50,15 +55,16 @@ spec:
50
55
type : object
51
56
spec :
52
57
description : |-
53
- InferenceModelSpec represents the desired state of a specific model use case . This resource is
58
+ InferenceModelSpec represents the desired state of an InferenceModel . This resource is
54
59
managed by the "Inference Workload Owner" persona.
55
60
56
61
The Inference Workload Owner persona is someone that trains, verifies, and
57
- leverages a large language model from a model frontend, drives the lifecycle
58
- and rollout of new versions of those models, and defines the specific
62
+ leverages a large language model focusing on model fidelity performance, and
63
+ less on inference performance (which is managed by the Inference Platform Admin).
64
+ They also drive the lifecycle and rollout of new versions of those models, and defines the specific
59
65
performance and latency goals for the model. These workloads are
60
66
expected to operate within an InferencePool sharing compute capacity with other
61
- InferenceModels, defined by the Inference Platform Admin.
67
+ InferenceModels, with specific governance defined by the Inference Platform Admin.
62
68
properties :
63
69
criticality :
64
70
description : |-
0 commit comments