Indeed, the theoretical undecidability of the halting problem does not apply to any concrete computer with finite memory. That's a finite system, so for any program, it's mathematically possible to determine whether it halts on a particular computer.
The thing is, this mathematical property is useless in practice.
The main reason for that is that while it's mathematically possible to determine whether a program halts on a computer with finite memory, it may not be physically possible. A computer with $n$ bits of memory has $2^n$ states. The naive way to determine whether a program halts on a finite-state computer is to run it and see if the states repeats before the program halts. But the number of states is so large than this is physically impossible even for relatively small values of $n$. Not only is there not enough time in a human life, there isn't enough energy to do it! Landauer's principle gives a lower limit for the amount of energy consumed by a computer (with any technology that we currently recognize as a computer), and the Solar system only has about enough energy for a little over 200 bits (see Amount of simple operations that is safely out of reach for all humanity?). That's billions of time fewer bits than a typical PC. The whole universe has enough energy, but if you're working on that scale you're probably going to want to analyze a bigger computer, and the ratio between doing and analyzing gets exponentially worse.
In addition, even if you lucked out and found some tricks to analyze the program faster (which isn't always possible — it would effectively mean that you could systematically compress a program — but it might be possible for “interesting” programs), the analysis would only tell you that the program halts, or doesn't halt, on a computer with $n$ bits of memory. It wouldn't tell you whether the program halts on a computer with $n+1$ bits of memory.
In some applications (e.g. a heart monitor or a brake controller), it's very important to know not only that a program will run in a certain fixed amount of memory, but also that it will run in a certain amount of time. For this kind of situation, the halting problem isn't relevant. There is a cost to doing such precise analysis: you need to keep the programs simple enough to stay very, very, very, very far from the theoretical maximum complexity.
In other applications (e.g. a server operating system or a compiler), the exact amount of memory and the exact time are not so important, but we want to be able to analyze complex programs. In this model, removing the time taken and the memory consumption from the analysis makes things a lot easier. It's a well-known phenomenon in mathematics that it's often easier to reason in infinite space than in large finite space, because in large finite space you have to constantly check whether you're bumping into some limit. Here the halting problem corresponds to a useful question to ask in practice.