There are two possible ways of defining what the length or size of the formula is:
According to the size of a reasonable (and fixed!) binary encoding.
According to the size of a reasonable (and fixed!) encoding over an infinite alphabet which includes symbols for all possible variables.
The definition you mention, for example, is what you get if you apply the second type of definition with respect to the prefix or postfix encoding of a formula.
Note that these two "approaches" don't by themselves defined a concrete measure of size. In cases where it matters, a concrete definition will appear, for example the one you quote. In many cases, however, the exact definition makes no difference since all definitions are "polynomially related", that is, the size of a formula varies at most polynomially between the different definitions. Your quote, for example, cares only about the size of the formulas being polynomial, and so an exact definition isn't important.
In other places, however, it is mentioned that some formula has size $O(n)$. Here the exact encoding still doesn't matter, but there is a real difference between the two options mentioned above. Consider for example the formula $x_1 \lor \cdots \lor x_n$. Under the second option, such a formula has size $\Theta(n)$. However, any reasonable binary encoding of this formula will have size $\Theta(n\log n)$, since storing the indices also takes space. From the author's statement it follows that he uses a definition of the second kind.