PyTorch Einsum Notation
We were recently introduced to the Einsum notation in PyTorch in a recent post on coding up Transformers in PyTorch. And i believe i ran into it in one of Alfredo's practicum lectures in the NYU Deep Learning 2020 lectures as well (the transformer one). And it seems pretty cool. So let's dive in.
Here's a link to the blog post he mentioned that gave him Einsum notation inspiration.
And of course there's the official PyTorch documentation here.
What i love about PyTorch code is that it can be extremely elegant. That's a fancy way of saying real easy to understand when you look at it.
And it appears that the Einsum function is indeed a 'swiss army knife for all kinds of tensor operations'.