You are correct; time complexity in theoretical computer science is usually measured in terms of the size of the input1.
This imposes a wrinkle for working programmers using time complexity to reason about their algorithms' performance. As you note, a lot of operations on numbers have higher complexity than a programmer would expect because numbers of high magnitude can be represented in a small amount of space and CPUs implement basic operations as single instructions. To a programmer, comparing two integers for equality is a (very cheap) constant time operation. To a computer scientist it requires linear time (in the size of the representation of the integers, not in their magnitude)!
But the basic idea of using big-oh complexity to measure running time fundamentally relies on inputs being arbitrarily large, and even the basic theoretical notions of whether problems are computable or not relies on the input space being infinite. Whereas the way primitive numeric types work in actual CPUs relies on them being fixed-size units that therefore have a bounded size and a finite number of possible values.
In big-oh complexity analysis we're giving an expression that will bound the runtime as inputs grow arbitrarily large. If the input is a single number expressed in a primitive machine type, the largest possible input is a constant, and thus the largest possible runtime is also a constant. Many problems would thus have "degenerate" $O(1)$ runtime if analysed this way. Additionally every problem that has only a finite number of cases is decidable, because a big (but finite) mapping table would decide it. The halting problem where the input is a positive integer that identifies a Turing machine, is undecidable. The variant halting problem where the input is a positive integer that fits in 64-bits is decidable and can be implemented in $O(1)$ runtime; but this is a useless result that doesn't help anyone! (because the mapping table that would decide it is impractically vast, even though technically finite, and actually figuring out all the entries that should go in the table is probably impossible in the real world)
So basically, the standard tools of this sort of computer science theory just aren't useful to analyse algorithms where the input can only be a primitive machine integer. They don't usefully divide problems & algorithms into different classes unless we are considering problems that have an infinite space of inputs that can be of unbounded size. And primitive numeric types as implemented in actual CPUs and programming language don't meet this criteria. It is fundamentally impossible to do constant-time comparisons and basic arithmetic on numbers of unbounded size, CPUs can only do it because they work with bounded numbers.
For example, taking a number $n$ and printing the numbers from $1$ to $n$ is not $O(n)$ if we're being fully general. Simply incrementing a number that is of unbounded size (and might require many many more than 64 bits to represent) in the worst case requires us to traverse all of the digits. Even if we assume each 64-bit chunk of bits can be handled all at once in a single operation, the number of 64-bit chunks is linear in the representation size of $n$ (which is logarithmic in $n$ itself). But if we constrain our program to only take an $n$ that can fit in a single 64-bit word, then the worst case runtime is $O(2^{64})$ which is also $O(1)$ since constant factors aren't relevant to big-oh analysis.
For a real algorithm that takes a number as input to match up with the this kind of theoretical analysis, the input needs to be not a primitive type like a 64-bit integer, but rather an arbitrary size (and/or precision) number; a lot of languages have "bignum" types and libraries that support this. These effectively use primitive machine types as digits in a base $2^{64}$ (or whatever) number system; numbers are variable length sequences of symbols rather than single symbols, and operations like arithmetic and comparisons take time proportional to the number of "digits" rather than being constant time.
Most of the time real programs simply don't need to ever be given inputs large enough to require arbitrary-precision numbers. For example your "print every number from $1$ to $n$" program is already getting up to the scale of the age of the universe to run just with the largest possible 64-bit numbers (depending on exactly how many we can print a second). So programmers take the shortcut of just assuming they'll only ever need a number that is a single "digit". That shortcut makes lots of algorithms much "faster" than we would get from analysing them as if they were Turing machines reading numbers encoded in an unbounded number of symbols, but technically this is only because they don't actually solve the infinite problem such a Turing machine would solve, only the finite number of cases that we care about in practice.
If we're being very pedantic this issue applies with almost all programs, not just ones involving numbers. Real programs run on an actual machine always have a finite bound on the amount of memory they can use, which stops them from actually being able to solve problems of arbitrary size, and thus technically makes their "real" complexity degenerate. But analysing a string-processing algorithm as-if the strings could be unbounded still produces useful knowledge because programmers are simply used to thinking about strings as variable-sized with operations on them taking time dependent on the size of the string. The point where the big-oh bound takes over and dominates any lower-order term (that can be larger for small inputs) tends to in practice occur at small enough magnitudes that it's still well within the scale of inputs that actually occur, and so it produces useful results to take the classifications we get from analysing idealised infinite-capacity algorithms and applying them to algorithms that will run on real machines implemented in real programming languages.
But if you're processing numbers and only accepting numbers with a finite bound, you need to do a different analysis with different assumptions than standard basic complexity analysis. Giving your runtime bound in terms of the magnitude of the input rather than the size of its representation and assuming basic operations are constant time can produce quite reasonable results for a working programmer; they would be technically wrong as general statements for infinite problems requiring unbounded input, but if that's not what you actually want to analyse it doesn't necessarily matter much.
1 If we're being properly rigorous we really should, as Caleb's answer points out, explicitly consider and state what variable we're actually using in the expression we're using to bound the input. It doesn't have to be the size of the input but that's usually the "default", especially in informal discussions like those that tend to be made by programmers rather than academics.
There's also some subtleties in what quantity is actually being bounded by the big-oh expression too. For example most programmers with any exposure to complexity theory have heard of the idea that an optimal comparison-based sorting algorithm has time complexity $O(n \cdot \log n)$. But what is often missed is that this bound is abstracting over the actual size of the items being sorted and the time it takes to compare them. This is not a bound on the total number of operations in running the sorting algorithm. $n$ in this expression is the number of items to be sorted, and the quantity being bounded is the number of comparisions that will be needed (regardless of the number of operations it takes to compare two items).