I will answer the question "is there a language which is countable and contains a string of infinite length?"
The answer is yes. Consider the symbols $\{0, 1\}$ and the language consisting of strings which do not contain the symbol $1$. The string of infinitely many $0$s and no $1$s is in the language, but there are still countably many strings in the language (match the empty string with the natural number 0, match the string of infinitely many $0$s with the natural number 1, match the string $0$ with 2, the string $00$ with 3, $000$ with 4, and so on.)
Now, given an infinite string over this set of symbols it is only semi-decidable if the string belongs to this language. That is, there is no TM which will tell you if an infinite string contains no $1$s, but a TM can tell you if a string does contain a $1$.
Since your question seems to indicate that you view the cardinality of a set and the decidability of set membership as being related, you may be interested in apartness relations and, more generally, in constructive mathematics: https://en.wikipedia.org/wiki/Apartness_relation