Skip to main content
3 of 6
added 26 characters in body
Stef
  • 570
  • 2
  • 10

This can be summed up in one sentence that you should memorise:

In complexity theory, what we call a "problem" is never one question, but instead an infinite family of questions that depend on an input.

For each input, there is a question.

An algorithm is said to solve the problem if for any input, the algorithm is able to find the answer to the question. Upper bounds and lower bounds on the complexity of an algorithm that solves a problem are expressed as a function of the size of the input.

With these definitions in mind, a problem that doesn't depend on an input can be solved in O(1) by definition, and a problem whose input is artificially limited in size can be solved in O(1) by definition. Only problems whose input can grow arbitrarily large are "interesting" from the point of view of complexity theory.

Constant Connect-Four vs. unbounded Connect-Four

For instance, the problem "Given a position of 7x6 Connect-Four, does the first player have a winning strategy?" is a boring O(1) problem from the theoretical point of view of complexity theory. You can imagine many algorithms to solve this problem, and all of them will have O(1) complexity by definition, which tells us nothing. For instance, one possible algorithm is to hardcode all $3^{42}$ possible positions and their answers in a lookup table. Technically this algorithm only uses O(1) space, even though $3^{42}$ is a huge number (about two hundred million million millions).

This is of course a disappointing answer, and certainly we want to say that some algorithms are more efficient than others. But complexity theory only studies the growth of complexity as input size grows, so if you don't allow input size to grow, then by definition the theory cannot do anything.

So instead of studying "classic 7x6 Connect-Four", you can study the more general problem "cxr Connect-Four", a generalisation of classic Connect-Four played on a board with c columns and r rows. Now you can't hardcode all the answers in a lookup table, because the problem is an infinite family of questions, for arbitrarily-large board sizes. And now you can express upper bounds and lower bounds on the complexity of an algorithm as a function of c and r, or as a function of n=cr to simplify.

Searching for a substring of size k in a string of size n

Sometimes instead of just one number n, there can be several numbers that depend on the input and that are relevant to algorithms' complexity. For instance, the problem "Find the index of a substring in a string" can be solved by a naive algorithm in time $O(nk)$, where $n$ is the length of the big string ("the haystack") and $k$ is the length of the substring ("the needle").

If the input is just the haystack and the needle, and there is no constraint on $k$ and $n$, then the size of the input is $N = k + n$ and if we want to express the complexity of the naive algorithm with respect to $N$, all we can say is that this algorithm has quadratic complexity, ie in the worst case it grows proportionally to $N^2$. This worst-case is realised for instance whenever $k = \frac{1}{2}n = \frac{1}{3}N$, because in those cases $nk = \frac{2}{9}N^2 = Θ(N^2)$.

However, a variant of this problem is the similar problem, but with $k$ bounded by a constant. In this case, the naive algorithm has linear complexity, because $kn ≈ kN = Θ(N)$.

Studying this "k is bounded" variant makes sense; it's a subproblem where we limit ourselves to searching small needles in arbitrarily large haystacks, so we don't want to be pessimistic and calculate worst-case complexities that allow $k$ to grow as large as $N/2$.

Stef
  • 570
  • 2
  • 10