Skip to content

Separates EnvoyExtensionPolicy from Ext Proc #200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 25, 2025
Merged

Separates EnvoyExtensionPolicy from Ext Proc #200

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f53749a
Separates EnvoyExtensionPolicy from Ext Proc
danehans Jan 15, 2025
8ebe6ed
Removes dplicative EnvoyExtensionPolicy from manifests
danehans Jan 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion pkg/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ The current manifests rely on Envoy Gateway [v1.2.1](https://gateway.envoyproxy.
```
Additionally, if you would like to enable the admin interface, you can uncomment the admin lines and run this again.


1. **Deploy Gateway**

```bash
Expand All @@ -41,6 +40,12 @@ The current manifests rely on Envoy Gateway [v1.2.1](https://gateway.envoyproxy.

```bash
kubectl apply -f ./manifests/ext_proc.yaml
```

1. **Deploy Envoy Gateway Custom Policies**

```bash
kubectl apply -f ./manifests/extension_policy.yaml
kubectl apply -f ./manifests/patch_policy.yaml
```

Expand Down
31 changes: 31 additions & 0 deletions pkg/manifests/extension_policy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
apiVersion: gateway.envoyproxy.io/v1alpha1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still exists in ext_proc.yaml, do we need it in both places?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not required for EPP, it's an Envoy Gateway requirement. I forgot to remove it so thanks for the heads-up. Commit 8ebe6ed removes the duplicate EnvoyExtensionPolicy, PTAL.

kind: EnvoyExtensionPolicy
metadata:
name: ext-proc-policy
namespace: default
spec:
extProc:
- backendRefs:
- group: ""
kind: Service
name: inference-gateway-ext-proc
port: 9002
processingMode:
request:
body: Buffered
response:
# The timeouts are likely not needed here. We can experiment with removing/tuning them slowly.
# The connection limits are more important and will cause the opaque: ext_proc_gRPC_error_14 error in Envoy GW if not configured correctly.
messageTimeout: 1000s
backendSettings:
circuitBreaker:
maxConnections: 40000
maxPendingRequests: 40000
maxParallelRequests: 40000
timeout:
tcp:
connectTimeout: 24h
targetRef:
group: gateway.networking.k8s.io
kind: HTTPRoute
name: llm-route