96
$\begingroup$

Many computer science programs require two or three calculus classes.

I'm wondering, how and when is calculus used in computer science? The CS content of a degree in computer science tends to focus on algorithms, operating systems, data structures, artificial intelligence, software engineering, etc. Are there times when Calculus is useful in these or other areas of Computer Science?

$\endgroup$

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

  • 6
    $\begingroup$ We don't have a strict policy for list questions, but there is a general dislike. Please note also this and this discussion; you might want to improve your question as to avoid the problems explained there. If you are not sure how to improve your question maybe we can help you in Computer Science Chat? $\endgroup$ – Raphael Apr 5 '16 at 12:14
  • 44
    $\begingroup$ You seem to make the common mistake of assuming that the content of every course has to be relevant (for every career path). Sometimes is just about training you how to think in certain ways. $\endgroup$ – Raphael Apr 5 '16 at 12:15
  • 8
    $\begingroup$ It would probably be helpful if this question could clarify whether you're only asking about uses within Computer Science proper or all of the uses for the students who are getting a Comp Sci degree. At least here in the U.S., a very large percentage of Comp Sci graduates become software engineers spread out over just about every engineering domain imaginable. Many of these domains require understanding Calculus for various purposes. Not all CS grads will use Calculus in their jobs, but many certainly will (probably more than who think they will during their Freshman year.) $\endgroup$ – reirab Apr 5 '16 at 15:56
  • 1
    $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – Raphael Apr 8 '16 at 6:42
  • $\begingroup$ For me it was very useful for the feeling of "appreciation and relief later when learning about numerical methods". When working with discrete data coming from real measurements, instead of continuous functions from theoretical mathematics, it was such a relief that integration is just a sum and the derivative is just a subtraction, instead of the overly complicated formulas we needed so much time and effort to learn in our calculus classes before! $\endgroup$ – vsz Apr 8 '16 at 16:48
111
$\begingroup$

I can think of a few courses that would need Calculus, directly. I have used bold face for the usually obligatory disciplines for a Computer Science degree, and italics for the usually optional ones.

  • Computer Graphics/Image Processing, and here you will also need Analytic Geometry and Linear Algebra, heavily! If you go down this path, you may also want to study some Differential Geometry (which has multivariate Calculus as a minimum prerequisite). But you'll need Calculus here even for very basic things: try searching for "Fourier Transform" or "Wavelets", for example -- these are two very fundamental tools for people working with images.
  • Optimization, non-linear mostly, where multivariate Calculus is the fundamental language used to develop everything. But even linear optimization benefits from Calculus (the derivative of the objective function is absolutely important)
  • Probability/Statistics. These cannot be seriously studied without multivariate Calculus.
  • Machine Learning, which makes heavy use of Statistics (and consequently, multivariate Calculus)
  • Data Mining and related subjects, which also use lots of Statistics;
  • Robotics, where you will need to model physical movements of a robot, so you will need to know partial derivatives and gradients.
  • Discrete Math and Combinatorics (yes!, you may need Calculus for discrete counting!) -- if you get serious enough about generating functions, you'll need to know how to integrate and derivate certain formulas. And that is useful for Analysis of Algorithms (see the book by Sedgewick and Flajolet, "Analysis of Algorithms"). Similarly, Taylor Series and calculus can be useful in solving certain kinds of recurrence relations, which are used in algorithm analysis.
  • Analysis of Algorithms, where you use the notion of limit right from the start (see Landau notation, "little $o$" -- it's defined using a limit)

There may be others -- this is just off the top of my head.

And, besides that, one benefits indirectly from a Calculus course by learning how to reason and explain arguments with technical rigor. This is more valuable than students usually think.

Finally -- you will need Calculus in order to, well, interact with people from other Exact Sciences and Engineering. And it's not uncommon that a Computer Scientist needs to not only talk but also work together with a Physicist or an Engineer.

$\endgroup$
  • 35
    $\begingroup$ Perhaps you had a different experience, but I found calculus pretty useless for learning how to reason and explain arguments rigorously. It was taught by rote and pattern matching pretty much like high school algebra and geometry. On the other hand, it was the prerequisite to several higher math classes that did teach these skills, so I suppose it wasn't entirely useless. $\endgroup$ – tsleyson Apr 5 '16 at 6:17
  • 6
    $\begingroup$ I can totally relate to the very last point (indirect benefits). Working on programming languages theory, I rarely used calculus directly. Perhaps the most direct application was in probabilistic computational models (e.g. Plotkin&Jones probabilistic powerdomains). Yet, my calculus course was mostly about proving things, and this was very, very valuable. One or two calculus courses are IMHO needed in every serious CS program, along some more math (discrete math, logic, linear algebra, numerical analysis, ... and possibly categories, topology, algebra, ... ). $\endgroup$ – chi Apr 5 '16 at 10:51
  • 3
    $\begingroup$ Here's an example of how I needed calculus in computer graphics: Smooth interpolation functions will basically all be of the form f(0) = 0, f(1) = 1, f'(0) = f'(1) = 0, and you can add any other constraints you care about, for instance f'(0.5) = 1. A little while ago I used this to derive some different interpolation polynomials for smoothing images. $\endgroup$ – porglezomp Apr 6 '16 at 16:24
  • 3
    $\begingroup$ Robotics can probably be expanded to any kind of physics modelling (which I guess also covers CG, in terms of lighting, so let's call it kinetic physics modelling). This includes acceleration/velocity, bounces/springs/deformation, PID controllers, acoustics, gravitation... $\endgroup$ – metao Apr 7 '16 at 7:37
  • 2
    $\begingroup$ I'll champion the indirectly point this way: better than any class they take before, Calculus teaches students they can't simply count the number of problems and estimate how much work is going to be involved. $\endgroup$ – candied_orange Apr 10 '16 at 21:33
20
$\begingroup$

This is somewhat obscure, but calculus turns up in algebraic data types. For any given type, the type of its one-hole contexts is the derivative of that type. See this excellent talk for an overview of the whole subject. This is very technical terminology, so let's explain.

Algebraic Data Types

You may have come across tuples being referred to as product types (if not, it's because they are the cartesian product of two types). We're going to take this literally and use the notation:

$$a * b$$

To represent a tuple, where $a$ and $b$ are both types. Next, you may have come across sum types these are types which can be either one type, or another (known as unions, variants, or as the Either type (kinda) in Haskell). We're also going to take this one literally and use the notation:

$$a + b$$

These are named as they are because if a type $a$ has $N_a$ values and a type $b$ has $N_b$ values, then the type $a + b$ has $N_a + N_b$ values.

These types look like normal algebraic expressions and we can, in fact, manipulate them as such (to a point).

An Example

In functional languages a common definition of a list (given in Haskell here) is this:

data List a = Empty 
            | Cons a List

This says that a list is either empty or a tuple of a value and another list. Transforming that to algebraic notation, we get:

$$L(a) = 1 + a * L(a)$$

Where $1$ represents a type with one value (aka the unit type). By repeatedly inserting, we can evaluate this to get a definition for $L(a)$:

$$L(a) = 1 + a * L(a)$$ $$L(a) = 1 + a * (1 + a * L(a))$$ $$L(a) = 1 + a + a^2 * (1 + a * L(a))$$ $$L(a) = 1 + a + a^2 + a^3 * (1 + a * L(a))$$ $$L(a) = 1 + a + a^2 + a^3 + a^4 + a^5...$$

(Where $x^n$ is meant in the sense of repeated multiplication.)

This definition says then that a list is either unit, or a tuple of one item, or a tuple of two items, or of three etc, which is the definition of a list!

One-hole Contexts

Now on to one-hole contexts: a one-hole context is what you get when you 'take a value out' of a product type. Let's give an example:

For a simple 2-tuple which is homogeneous, $a^2$, if we take a value out, we just get a 1-tuple, $a$. But there are two different one-hole contexts of this type: namely the first and second values of the tuple. So since it is either of these we could write that it is $a + a$, which is, of course, $2a$. This is where the differentiation comes in to play. Let's confirm this with another example:

Taking a value out of a 3-tuple gives a 2-tuple, but there three different variants:

$$(a, a, \_)$$ $$(a, \_, a)$$ $$(\_, a, a)$$

Depending on where we put the hole. This gives us $3a^2$ which is indeed the derivative of $a^3$. There is a proof of this in general here.

For our final example, let's use a list:

If we take our original expression for a list:

$$L(a) = 1 + a * L(a)$$

We can rearrange to get:

$$L(a) = \frac{1}{1 - a}$$

(On the surface this may seem like nonsense, but if you take the taylor series of this result you get the definition we derived earlier.)

Now if we differentiate this, we get an interesting result:

$$\frac{\partial L(a)}{\partial a} = (L(a))^2$$

Thus one list has become a pair of lists. This in fact makes sense: the two lists produced correspond to the elements above and below the hole in the original list!

$\endgroup$
  • $\begingroup$ This was wonderfully insightful. Thanks. $\endgroup$ – D. Ben Knoble Apr 29 at 16:08
12
$\begingroup$

Numerical Methods. There exist cumbersome calculus problems that are unique to specific applications, and they need solutions faster than a human can practically solve without a program. Someone has to design an algorithm that will compute the solution. Isn't that the only thing that separates programmers from scientists?

$\endgroup$
  • 3
    $\begingroup$ Given the "list" nature of this question, every answer should attempt to give the full picture. Are you certain you want to claim that numerical methods are the only instance? $\endgroup$ – Raphael Apr 5 '16 at 12:16
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – Raphael Apr 7 '16 at 8:46
12
$\begingroup$

Automation - Similar to robotics, automation can require quantifying a lot of human behavior.

Calculations - Finding solutions to proofs often requires calculus.

Visualizations - Utilizing advanced algorithms requires calculus such as cos, sine, pi, and e. Especially when you're calculating vectors, collision fields, and meshing.

Logistics and Risk analysis - Determining whether a task is possible, the risk involved, and possible rate of success.

Security - Most security can be performed without calculus; however, many people who want explanations prefer it in mathematical expressions.

AI - The basics of AI can be utilized without calculus; however, calculating advanced behavior, swarm intelligence/hive minds, and complex values based decision making.

Medical calculations - Visualizing most health data requires calculus such as an EKG reading.

Science & Engineering - When working with nearly any other scientific discipline requires calculus: Aerospace, Astrology, Biology, Chemistry, or Engineering.

Many people in programming can go their entire career without using calculus; however, it can prove invaluable if you're willing to do the work. For me it has been most effective in automation, logistics, and visualization. By identifying specific patterns, you can simply ignore the pattern, imitate the pattern, or develop a superior method all together.

$\endgroup$
  • 7
    $\begingroup$ How are cos, sine, $\pi$ and $\mathrm{e}$ calculus? $\endgroup$ – David Richerby Apr 6 '16 at 4:34
  • 3
    $\begingroup$ Well, $\exp(x)$ can be defined to be the unique solution to $f(x)=f'(x)$ with $f(0)=1$, and similarly sin, cos can be defined as the unique pair of functions satisfying $f'(x)=g(x)$, $g'(x) = -f(x)$ with appropriate boundary conditions (I think $f(0)=0$ and $g(0)=1$ should work). $\endgroup$ – Chris Taylor Apr 7 '16 at 7:23
  • 2
    $\begingroup$ @DavidRicherby: Example: how do you implement those functions on, say, a microcontroller without an FPU? If you know some calculus, you immediately know a good answer: power series. $\endgroup$ – Nate Eldredge Apr 10 '16 at 5:55
6
$\begingroup$

The fact is that there's very little chance you'll ever use calculus. However, virtually every other scientific discipline DOES use calculus and you are working on a science degree. There are certain expectations of what a university science degree is supposed to mean and one of those things is that you know calculus. Even if you'll never use it.

It's okay if you do poorly in calculus, but make sure you put some effort into discrete mathematics. There are a lot of real-world programming problems where discrete math comes into play and ignorance of its principles can embarrass you in front of other coders.

$\endgroup$
  • 9
    $\begingroup$ Your first paragraph is completely wrong and bordering on conspiracy theory. There are plenty enough areas of computer science where calculus is useful (see the other answers for endless lists of them). Sure, it's possible to avoid all of those areas but it's very misleading to claim that dropping calculus will have zero impact beyond grades. $\endgroup$ – David Richerby Apr 5 '16 at 19:58
  • 4
    $\begingroup$ Depending on your degree program, you could complete a degree without ever using calculus, and I definitely think CS majors don't need as much of it as we get. But doing poorly in it will lock you out of some of the most interesting areas of computer science. There's plenty of time to be a web developer once you graduate; while in school, why not try to push yourself a little? $\endgroup$ – tsleyson Apr 6 '16 at 1:59
  • 3
    $\begingroup$ @tsleyson If you want to be a web developer, save the tution money and time needed to obtain a CS degree. $\endgroup$ – Raphael Apr 6 '16 at 8:37
  • 8
    $\begingroup$ @ScottB You seem to be confusing computer science with programming. $\endgroup$ – David Richerby Apr 6 '16 at 18:34
  • 3
    $\begingroup$ @ScottB Who's saying CS = math + programming? I myself have been advocating against this limited view for ages. But you have it backwards, too: mathematics are integral to CS, just like it is to physics. We need it, even if we don't want to practice it. (That said, this is not the place for this discussion. Please join us in Computer Science Chat if you want to continue.) $\endgroup$ – Raphael Apr 6 '16 at 19:49
4
$\begingroup$

Many people already provided applications in CS. But sometimes you'll find Calculus when you least expect:

Regular-expression derivatives reexamined

If you know automata this pdf might be worth reading.

$\endgroup$
  • $\begingroup$ I don't see any differential calculus there; I see the word "derivative", but I don't see anything resembling traditional differential calculus. $\endgroup$ – user49170 Apr 7 '16 at 18:16
  • 2
    $\begingroup$ It's called "formal derivative", and it is related to Calculus, in a way. You will also see this done with Generating Functions, some formulas related to discrete structures and other areas where you don't actually have a "smooth function". $\endgroup$ – Jay Apr 7 '16 at 23:58
  • $\begingroup$ @Jay : The important thing isn't the name. How does understanding calculus help with it? $\endgroup$ – Christian Apr 8 '16 at 11:31
  • 2
    $\begingroup$ It is explained in this Wikipedia page. The formal derivative is an operation on elements of analgebraic structure that contains polynomials, and it is formally "quite like" the usual rule for differentiating polynomials, however -- differently from what a student sees in Calculus -- the polynomials are not over reals; they may be polynomials over an arbitrary "ring" (another algebraic structure). And there are practical applications of the formal derivative -- I have seen at least one (Algebraic Cryptanalysis -- can't remember the details). $\endgroup$ – Jay Apr 8 '16 at 12:27
4
$\begingroup$

Some more specific examples:

  • Calculus is used to derive the delta rule, which is what allows some types of neural networks to 'learn'.
  • Calculus can be used to compute the Fourier transform of an oscillating function, very important in signal analysis.
  • Calculus is used all the time in computer graphics, which is a very active field as people continually discover new techniques. For a fundamental example check out Kajiya's rendering equation
  • Calculus is important in the field of computational geometry, investigate curve and surface modelling.
$\endgroup$
3
$\begingroup$

To these other excellent answers I add this point: rigor in testing.

In creating test cases for some applications I have had to make use of calculus to predict expected running times, memory sizes, and choose optimal parameters when tuning data structures. This includes understanding expected rounding error, etc.

While statistics is mentioned in other answers, I would like to specifically mention Monte-carlo algorithms, such as optimization algorithms and some frugal streaming algorithms that are based on Mathematical principles that include calculus.

Specific industries where I have worked where calculus was required include:

  • Finance (creating a trading platform)

  • Insurance (numerical integration of insurance policies in what-if scenarios to compute expected policy losses)

  • Logistics (optimizing the consolidation of transportation routes)

  • Signal processing

$\endgroup$
3
$\begingroup$

Calculus -- the integral portion -- is used directly in CS as a foundation for thinking about summation. If you work through any portion of Knuth's Concrete Mathematics section on summation, you will quickly recognize conventions common to calculus: understanding some of the continuous case gives you tools to consider the discrete.

Many of the uses of your CS study involve programming systems which monitor change, or in some cases, attempt to predict the future. The mathematics around those systems is rooted in differential equations and linear algebra, and differential equations are...calculus. There are teachers like Gibert Strang who advocate for moving more quickly into the differential equations part, but it is still a subset of calculus. When change depends on change in any system, it starts to be unstable (and stable) in ways which are both non-intuitive and very well understood. To understand why your sensible linear system is behaving in nonlinear ways, you either need the tools of calculus or you need to re-invent them for your problem space.

And finally, CS often requires reading and understanding the work of others, and calculus is the first exposure to a lot of shared vocabulary, convention, and history.

$\endgroup$
  • $\begingroup$ "Many of the uses of your CS study involve programming systems which monitor change, or in some cases, attempt to predict the future" -- I don't think this is representative of moust CS courses of studies. $\endgroup$ – Raphael Apr 10 '16 at 12:42

protected by Gilles 'SO- stop being evil' Apr 9 '16 at 21:28

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.