Skip to content

Commit e1ba762

Browse files
authored
Removing Obsolete Portion of Metrics Guide (#608)
1 parent cb98e2f commit e1ba762

File tree

1 file changed

+2
-21
lines changed

1 file changed

+2
-21
lines changed

site-src/guides/metrics.md

+2-21
Original file line numberDiff line numberDiff line change
@@ -4,26 +4,7 @@ This guide describes the current state of exposed metrics and how to scrape them
44

55
## Requirements
66

7-
To have response metrics, set the body mode to `Buffered` or `Streamed`:
8-
```
9-
apiVersion: gateway.envoyproxy.io/v1alpha1
10-
kind: EnvoyExtensionPolicy
11-
metadata:
12-
name: ext-proc-policy
13-
namespace: default
14-
spec:
15-
extProc:
16-
- backendRefs:
17-
- group: ""
18-
kind: Service
19-
name: inference-gateway-ext-proc
20-
port: 9002
21-
processingMode:
22-
request:
23-
body: Buffered
24-
response:
25-
body: Buffered
26-
```
7+
To have response metrics, ensure the body mode is set to `Buffered` or `Streamed` (this should be the default behavior for all implementations).
278

289
If you want to include usage metrics for vLLM model server streaming request, send the request with `include_usage`:
2910

@@ -40,7 +21,7 @@ curl -i ${IP}:${PORT}/v1/completions -H 'Content-Type: application/json' -d '{
4021

4122
## Exposed metrics
4223

43-
| **Metric name** | **Metric Type** | <div style="width:200px">**Description**</div> | <div style="width:250px">**Labels**</div> | **Status** |
24+
| **Metric name** | **Metric Type** | <div style="width:200px">**Description**</div> | <div style="width:250px">**Labels**</div> | **Status** |
4425
|:---------------------------------------------|:-----------------|:------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:------------|
4526
| inference_model_request_total | Counter | The counter of requests broken out for each model. | `model_name`=&lt;model-name&gt; <br> `target_model_name`=&lt;target-model-name&gt; | ALPHA |
4627
| inference_model_request_error_total | Counter | The counter of requests errors broken out for each model. | `model_name`=&lt;model-name&gt; <br> `target_model_name`=&lt;target-model-name&gt; | ALPHA |

0 commit comments

Comments
 (0)