Linux memory management (NUMA balancing) as used by Gaudi accelerators: Automatic NUMA balancing migrated copy-on-write

Impact

Automatic NUMA balancing migrated copy-on-write pages that the Gaudi accelerator still had pinned, silently corrupting data in flight to the device. This is a correctness and integrity bug rather than an access-control one, but on a training cluster silent tensor corruption is worse than a crash because it poisons checkpoints before anyone notices.

Who can reach it

No attacker needed - it fires on normal accelerator workloads whenever automatic NUMA balancing is enabled on the node.

What to do

Take the kernel fix and reboot. As an interim mitigation, disable automatic NUMA balancing (numa_balancing=disable) on accelerator nodes, which is a boot-parameter change and therefore still needs a drain and reboot. No firmware component.

References