The first thing to recognize is that this is a permutation matrix where each row and column contains a single $1$ entry with the remaining entries being $0$. This follows from the fact that $x$ and $y$ represent classical bit strings and $f$ is a classical boolean function. Let's
Let's take the basic case where $x$ is from the space of $n$ bit strings, $y$ is a single bit, and $\mathcal{f}$ maps $\{0,1\}^n$ to $\{0,1\}$. The operation then functions on $n+1$ bits and is represented by a $2^{n+1} \times 2^{n+1}$ matrix. If you let the binary number $xy$ index the matrix columns, then for column $xy$ set row $x(f(x) \oplus y)$ to 1 and all other rows to 0.
Let's look at an example from Deutsch's algorithm. Let $x$ be from $\{0,1\}$ and $f$ be bit negation, i.e. $f(0) = 1$ and $f(1) = 0$. From here we compute, for each row, the column number containing the 1.
$\begin{array}{cc} \mbox{column} & \mbox{row with 1} \\ 00 & 0(0 \oplus 1) = 01 \\ 01 & 0(1 \oplus 1) = 00 \\ 10 & 1(1 \oplus 1) = 10 \\ 11 & 1(1 \oplus 0) = 11 \end{array} $
This gives us the matrix
$ \left( \begin{array}{cc} 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{array} \right) $
This generalizes to when $y$ is more than a single bit.