The study of models of computation extremely broadly construed is called "generalized recursion theory" (or "higher recursion theory," or "generalized/higher computability theory," or etc.). We can indeed rigorously define and study (from a theoretical standpoint of course) models of computation much stronger than what is "physically implementable" in even ...
Les provides a concise and correct answer:
mathematical definitions are as concise as possible,
and explicitly including an infinite tape into a definition of a Turing machine would make its definition much less concise, so we don't.
This doesn't answer the question: why?
How can the definition exclude the infinite tape when we need one?
The answer: we don'...