How to connect the math of recurrence relations to daily programming concepts

Question

What exactly are we doing from a CS perspective when we solve a recurrence relation and find a resulting formula for a sequence given a set of initial conditions? I just went through the "linear homogeneous recurrence relations of degree k with constant coefficients" bit in discrete math and basically understand the math part and have a simple process for solving them mechanically.

What I haven't seen yet is an explanation of what this correlates to in a CS sense. I understand we will encounter recurrence relations in algorithms which we haven't reached yet (next class) but I'm wondering what exactly do the initial conditions and the sequence represent?

For a trivial programming example, if we were to write a recursive function to process a directory and all its subdirectories, I understand conceptually that could be modeled as a recurrence relation because it is recursive, but I don't know how, and I don't know what the initial conditions and final formula for the sequence would represent in such a scenario.

Here's an example of the type of relation I'm talking about. So we solve these by taking the relation down into its characteristic equation, finding the roots, and then building a system of equations from the initial conditions $a_0, ..., a_j$ that we solve to find the constants. Finding the constants gives a closed formula for that particular sequence defined by those initial conditions.

My question is, from a CS/programming/software engineering perspective what would we model using recurrence relations like this other than algorithms, and what would the initial conditions represent in those models?

Answer 1

In computer science, recurrence relations are often useful for analyzing the running time of algorithms. For instance, we might define a function $T(\cdot)$, so that $T(n)$ is the running time when running the algorithm on an input of size $n$. Often, for many kinds of algorithms, we can write a recurrence relation that expresses $T(n)$ in terms of smaller values, such as

$$T(n) = 2T(n/2) + O(n).$$

This would be saying that the running time to solve a problem of size $n$, is twice the time to solve a problem of half that size (perhaps because we recursively solve the first half and the second half using the same algorithm), plus $O(n)$ more (perhaps to combine/merge their two solutions).

So, the numbers represent running times -- the number of operations the algorithm takes to solve the problem. You might like to take a look at Reference answers to frequently asked questions and How to come up with the runtime of algorithms?. Also take a look at algorithms textbooks, especially their chapters on running time analysis and divide-and-conquer algorithms.

The initial condition of a recurrence (e.g., the value of $T(1)$) depicts the running time of the algorithm on a small instance. Often, for a recursive algorithm, this is the running time of the base case.

In computer science, the sequence we are considering is

$$T(1),T(2),T(3),T(4),T(5),\dots,$$

i.e., the sequence of running times as the input size increases. If you were to write down the sequence of running times for some algorithm, you might spot a pattern, where each term can be computed by a simple formula of a few previous terms (namely, the pattern indicated by the recurrence relation). That said, honestly, thinking about this as a sequence might not be the most helpful or intuitive viewpoint (at least, I don't find it to be so); I find it more intuitive to think of $T(\cdot)$ as a function rather than a sequence. The same mathematics still applies.

Another way to get some intuition is to work through a few examples where recurrences are used to analyze the running time of some simple algorithms. I think that will also help you get a better intuition of what the mathematical expressions mean and how they relate to the algorithm.

Answer 2

The following is an answer to your first question “What exactly are we doing from a CS perspective when we solve a recurrence relation and find a resulting formula for a sequence given a set of initial conditions?”

Two motivations or reasons for finding a corresponding formula for a given recurrence relation and set of initial condition are to reduce the amount of memory and the time needed to compute the solution or the answer.

An “easy” (in terms of human effort) way to find the solution to a recurrence relation would be to use a recursive function: run it and hope for two things to happen – not to encounter a stack overflow and get an answer soon. This is “easy” for a human being because he or she does three basic things: translates the recurrence relation into a recursive function, run the recursive function, and wait for a response from the computer. It becomes a “challenge” if he or she cannot wait any longer for a response or the recursive function takes too long to find the answer or if the recursive function runs into a stack overflow – in other words, out of memory error. It is very tempting to use a recursive function because it is easy to translate a recurrence relation to a recursive function, and it works for “small problems” in a learning environment.