Code

Promote Dev#122

Code

Open

vMaroon wants to merge 48 commits intomainneuralmagic/gateway-api-inference-extension:mainfrom

devneuralmagic/gateway-api-inference-extension:devCopy head branch name to clipboard

Commits

Commits on Apr 30, 2025

• - implemented kvcache-aware-scorer

  Show description for dc174c2
  vMaroon

  committed
  dc174c2

  Copy full SHA for dc174c2

• undo gofumpt

  Show description for 3476f59
  vMaroon

  committed
  3476f59

  Copy full SHA for 3476f59

• added scorer initialization debug msg

  Show description for e6ca553
  vMaroon

  committed
  e6ca553

  Copy full SHA for e6ca553

• - added debug logging

  Show description for 388a7db
  vMaroon

  committed
  388a7db

  Copy full SHA for 388a7db

• updated KVCacheAwareScorer comments

  Show description for 01f019d
  vMaroon

  committed
  01f019d

  Copy full SHA for 01f019d

• reused envutils (review comment)

  Show description for bc2fee3
  vMaroon

  committed
  bc2fee3

  Copy full SHA for bc2fee3

• Merge pull request #34 from vMaroon/kvcache-aware

  Show description for d08e32f
  vMaroon

  authored
  d08e32f

  Copy full SHA for d08e32f

• testing new lint config to diff: false

  clubanderson

  committed
  e7d8837

  Copy full SHA for e7d8837

Commits on May 1, 2025

• fix lint

  clubanderson

  committed
  fe5168f

  Copy full SHA for fe5168f

Commits on May 7, 2025

• fix lint issues

  clubanderson

  committed
  00eaf5e

  Copy full SHA for 00eaf5e

• add comments to working golang file

  clubanderson

  committed
  ea6b8b0

  Copy full SHA for ea6b8b0

• Provide a way to enable the PDFilter

  

  lionelvillard

  authored and

  clubanderson

  committed
  d1dda06

  Copy full SHA for d1dda06

• update readme

  

  lionelvillard

  authored and

  clubanderson

  committed
  7ba6dae

  Copy full SHA for 7ba6dae

• Update pod labels to match ModelService

  Show description for cdef3f2
  

  jgchn

  authored and

  clubanderson

  committed
  cdef3f2

  Copy full SHA for cdef3f2

• add log lines

  

  lionelvillard

  authored and

  clubanderson

  committed
  e38d80b

  Copy full SHA for e38d80b

• address review comments

  

  lionelvillard

  authored and

  clubanderson

  committed
  82f2edd

  Copy full SHA for 82f2edd

• fix build:

  Show description for ec112f1
  

  vMaroon

  authored and

  clubanderson

  committed
  ec112f1

  Copy full SHA for ec112f1

• Fixed scorer tests

  

  shmuelk

  authored and

  clubanderson

  committed
  4f19b66

  Copy full SHA for 4f19b66

• Added PostResponse to scheduler config

  

  shmuelk

  authored and

  clubanderson

  committed
  ec440d6

  Copy full SHA for ec440d6

• Use an init() function instead of modifying the scheduler code to inject our config

  

  shmuelk

  authored and

  clubanderson

  committed
  4262afc

  Copy full SHA for 4262afc

• Added code to scheduler to enable running the PostResponse plugins

  

  shmuelk

  authored and

  clubanderson

  committed
  6974c4f

  Copy full SHA for 6974c4f

• Invoke the PostResponse handlers and send any added headers to the user

  

  shmuelk

  authored and

  clubanderson

  committed
  cae1fb4

  Copy full SHA for cae1fb4

• Added a simple unit test for the PostResponse plugin invocation

  

  shmuelk

  authored and

  clubanderson

  committed
  012e2b3

  Copy full SHA for 012e2b3

• [build]: Updating vllm deployment to the latest image and scorers (#112)

  Show description for 5c06734
  

  kfirtoledo

  authored and

  clubanderson

  committed
  5c06734

  Copy full SHA for 5c06734

• Add P/D scheduler (#115)

  Show description for 47ea097
  

  mayabar

  authored and

  clubanderson

  committed
  47ea097

  Copy full SHA for 47ea097

• Add decode filter to the default filters list in case pd is enabled (#119)

  

  mayabar

  authored and

  clubanderson

  committed
  eb42ef9

  Copy full SHA for eb42ef9

• cherry-picked prefix_score

  

  oglok

  authored and

  clubanderson

  committed
  1f73e44

  Copy full SHA for 1f73e44

• Add prefix store functionality

  Show description for 4f30a01
  

  oglok

  authored and

  clubanderson

  committed
  4f30a01

  Copy full SHA for 4f30a01

• Prefix Aware Scorer

  Show description for d271085
  

  oglok

  authored and

  clubanderson

  committed
  d271085

  Copy full SHA for d271085

• Add unit tests for prefix store

  Show description for 8258b52
  

  oglok

  authored and

  clubanderson

  committed
  8258b52

  Copy full SHA for 8258b52

• Add unit tests for prefix aware scorer

  Show description for d2a20d9
  

  oglok

  authored and

  clubanderson

  committed
  d2a20d9

  Copy full SHA for d2a20d9

• implemented PrefixAwareScorer based on Ricardo's work

  

  vMaroon

  authored and

  clubanderson

  committed
  24c51c0

  Copy full SHA for 24c51c0

• Remove KVcache scorer changes for traceability

  Show description for 749e29c
  

  oglok

  authored and

  clubanderson

  committed
  749e29c

  Copy full SHA for 749e29c

• addressed review comments

  Show description for 9685f3c
  

  vMaroon

  authored and

  clubanderson

  committed
  9685f3c

  Copy full SHA for 9685f3c

• Session affinity scorer (#117)

  Show description for 15356da
  

  2 peopleauthored and

  clubanderson

  committed
  15356da

  Copy full SHA for 15356da

• [docs]: Add prefix flags to README

  Show description for 249e745
  

  kfirtoledo

  authored and

  clubanderson

  committed
  249e745

  Copy full SHA for 249e745

• switch prefix_scorer updates to post-schedule temporarily

  Show description for 34ef75d
  

  vMaroon

  authored and

  clubanderson

  committed
  34ef75d

  Copy full SHA for 34ef75d

• chore: update istio-control-plane release

  Show description for 30ba3cd
  

  shaneutt

  authored and

  clubanderson

  committed
  30ba3cd

  Copy full SHA for 30ba3cd

• fix: epp image

  Show description for 3a2701e
  

  shaneutt

  authored and

  clubanderson

  committed
  3a2701e

  Copy full SHA for 3a2701e

• fix: kind env handling for gw impl choice

  Show description for 72437fe
  

  shaneutt

  authored and

  clubanderson

  committed
  72437fe

  Copy full SHA for 72437fe

• docs: update DEVELOPMENT.md

  Show description for a80bed9
  

  shaneutt

  authored and

  clubanderson

  committed
  a80bed9

  Copy full SHA for a80bed9

• fix: target for image override in Istio k8s env

  Show description for e5bd2ed
  

  shaneutt

  authored and

  clubanderson

  committed
  e5bd2ed

  Copy full SHA for e5bd2ed

• fix PD scorers configurations (#133)

  Show description for 5ee8225
  

  vMaroon

  authored and

  clubanderson

  committed
  5ee8225

  Copy full SHA for 5ee8225

• chat-completions for prefix-aware

  Show description for 4ddfc9f
  vMaroon

  committed
  4ddfc9f

  Copy full SHA for 4ddfc9f

• Merge pull request #137 from vMaroon/chat-completions

  Show description for 47e32bf
  vMaroon

  authored
  47e32bf

  Copy full SHA for 47e32bf

Commits on May 8, 2025

• - improved prefix-store (performance & hit rate)

  Show description for 14a2d2d
  vMaroon

  committed
  14a2d2d

  Copy full SHA for 14a2d2d

• error on hashing fail in PrefixStore::AddEntry

  vMaroon

  committed
  105558c

  Copy full SHA for 105558c

• Merge pull request #140 from vMaroon/dev

  Show description for 0755bb3
  vMaroon

  authored
  0755bb3

  Copy full SHA for 0755bb3