NVIDIA TensorRT-LLM: Concurrent requests race inside TensorRT-LLM and reach data tampering with a changed scope. On a

Impact

Concurrent requests race inside TensorRT-LLM and reach data tampering with a changed scope. On a shared LLM serving tier, a race between concurrent requests means one tenant's request can affect another's - the mechanism by which response bleed-through happens.

Who can reach it

Network, low privileges. Any client able to issue concurrent requests to the serving endpoint, which is every client.

What to do

Upgrade TensorRT-LLM per bulletin 5805 and roll the serving deployment. Cost: rolling restart. Until patched, the compensating control is reducing concurrency or dedicating an engine per tenant, both of which cost throughput.

References