NVIDIA TAO Toolkit: An uncontrolled search path loads an attacker-planted resource, reaching privilege escalation

Impact

An uncontrolled search path loads an attacker-planted resource, reaching privilege escalation and code execution. In an AI datacenter this is the model-and-data supply chain problem: the code runs with whatever the training or inference job holds, which is usually a GPU, a service account, and mounted object storage credentials.

Who can reach it

Requires the job to load an attacker-influenced artifact - a checkpoint, .nemo file, config, tokenizer or dataset. Any pipeline that pulls from a public model hub, a customer bucket, or a tenant-supplied path is in scope.

What to do

Bump the package to the fixed version in bulletin 5730 and rebuild every training/inference image that embeds it. Cost: image rebuild and job restart; no host driver or firmware change. The durable control is refusing to deserialize untrusted checkpoints at all - prefer safetensors-style formats and treat pickle-bearing artifacts as executable code.

References