"tools" not considered in prefix matching logic

[liu-cong]

What happened:

In the discussion #2133 (comment), we found that "tools" in the chat completion API are not considered in the prefix matching.

What you expected to happen:

"tools" should be considered to match vllm's logic. "tools" are often considered the first part of the chat template, example. so for the prefix scorer, we should merge "tools" field as first part of the prefix string.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• Inference extension version (use git describe --tags --dirty --always):
• Cloud provider or hardware configuration:
• Install tools:
• Others:

[srampal]

Note: A first pass of adding "tools" into the responses api was included as part of the PR #2133
Also an initial version of support for the prefix plugin was added as part of the same PR.
This PR is to track if any further updates are needed (not just for "tools" but with any additional updates needed to the prefix plugin for optimal scheduling performance in case of the responses api)

The chat completions api was left untouched for "tools" since it was prior code and this can also be investigated for potential updates needed.

This PR is for the prefix plugin. Created a similar one for the SLO/ latency predictor plugin, the work could potentially be combined.
#2219