https://probablymarcus.com/blocks/2023/10/19/vectorizing-wide-pytorch-expressions.html
What happens when you vectorize wide PyTorch expressions?
Marcus Lewis
19 October 2023
“So we see that vectorization has three effects:
It lets us keep the GPU busy even when inputs are small.
It frees up the CPU to do other work.
It can slightly change the total amount of GPU work, sometimes for the worse.”