I am getting confused by the regular expression $(a\mid b)^*$ as it for sure matches $aab$ and $ab$.
Does $(a\mid b)^*$ also match strings like $aa$, $aaaa$, $bb$ or $bbb$, that is those that use only $a$ or $b$?
Yes: $(a\mid b)^*$ is the concatenation of any number of $a$'s and $b$'s, so the expression matches $a^n$ and $b^n$.
As you can read more about on Wikipedia, the vertical bar $\mid$ is the boolean "or" relation (one or the other must be included in the regular expression), while the asterisk $^*$ indicates that the preceding element is repeated zero or more times. Hence:
$(a\mid b)^*$ is equivalent to $(a\;|\;b)(a\;|\;b)(a\;|\;b)\cdots$ repeated any number of times (even zero).
For each term, you can select either $a$ or $b$, so strings like $aa$, $aaaa$, $bb$ and $bbb$ are valid. In particular, this expression matches the empty string $\epsilon$, which has zero $a$'s and zero $b$'s.