I think the graphics association comes from having written the implementation in HLSL. HLSL compute shaders are general-purpose (like CUDA) but often used in the graphics context, so graphics programmers might feel more comfortable with it?

Tbh I don't really know, I've done extensive ML and graphics work and I'd just pick the right tool for the job (PyTorch and just drop down to Triton/CUDA/whatever when you need custom GPU code).