Answering knowingly this question would require experiments. I am sure
there is some data for natural languages, where it is a common
problem. I recall from memory that one study gave ridiculously small
figures for natural language, which is not too surprising. If you take
5 consecutive word in a sentence (the figure I recall, without being
sure), there is a good chance that one of them belongs to a single
language, and even more that the fragment can syntactically belong to
only one language (parsing fragments without context is possible with
existing technology). To me the problem is not so much the size of the
input as the size of the recognition program and its data. There is a
compromise there. Actually my guess is that keeping all the relevant
data is far too costly, and that the actual techniques are statistical
ones, such as checking n-grams of letters. (see Wikipedia), which are
Regarding programming languages, the problem is a bit different. The
size of the vocabulary is ridiculously small, and identifiers do not
give any indication (or very little with the allowed mophology: a
language could forbid to use dask inside identifiers, for example).
Furthermore, what fixed vocabulary there is (keywords) is often the
same in many programming languages. However, programming languages
have a very strict syntax, which will certainly distinguish them
rather quickly. It is not so much how long a fragment as the kind of
fragment. A long succession of assignments might look the same in many
programming languages. Buit I would not venture any figure, and I am
not even sure statistics would make sense.
Then there is the issue of parenthood. A fragment of Pascal may look
very much like a fragment of Algol 60 or Simula 67. Is American
English to be distinguished from British or Autralian English?
To conclude, without any hard knowledge on facts:
The problem should be stated with a word regarding the space cost (and
possibly time-cost) of the identification program.
Identification for natural language is essentially morphology or
lexically based, and will use statistical techniques if space costs
are to be acceptable. They can recognize fairly short sequences (a few
words as I recall) with good accuracy.
Identification for programming languages is essentially syntax based,
and probably needs larger fragments in number of tokens, in order to
have enough syntax substance, despite the intentional similarities
between programming languages. But it can probably be 100% accurate,
without excessive size of the identification program. I would however
be more confident if I had actual data to back my guesswork. I do not know of any work on this topic.
Considering only fragments is not an issue. It is obviously not an
issue when only lexical information is used. It is not an issue either
when syntactic information dominates, as the technology to parse
fragments is working well.
Afterthoughts, after the question was completed.
One minor remark concerns the concept of substring size: is it measured in
characters, in bytes, in lexical elements? Size of character encoding
is variable. Characters take diacritical marks. Lexical elements have
different average size in natural and programming languages.
A more important remark concerns the mode of measurement. Natural
language will use statistical methods to avoid the problem of natural
language huge specification. Hence the answer is accordingly accurate
with some probability that may depend on substring length (different
techniques produce different types of mis-detection).
In the case of programming languages, the specifications are small
enough that they can probably be used exactly. Hence the answer could
possibly be always 100% exact at acceptable cost. The problem would
not be with the detection procedure inaccuracies as in natural
language, but in the question itself. If a string does not
discriminate two languages because it can belong to both, no amount of
technology will help you solve the problem. In such a case, the
detection software should not guess, which would be meaningless, but it should just list, with a 100% accuracy, all the
programming languages the substring can belong to.
In other words, the case of natural language and the case of
programming languages are very different technically. I am not sure it
makes sense to compare them.