The role of $x$ in $\frac{\mathrm{d}}{\mathrm{d}x} y$ not only confuses my calculus students, it has also puzzled some well known mathematicians. Questions one might ask are:

  • Does the $x$ in the denominator bind the $x$ in $y$? (Clearly no, since $\frac{dx^2}{dx}=2x$.)

  • Can one substitute for $x$ in the denominator? (Looks like not, what should $\frac{dx^2}{d3}$ mean?)

  • Is the $x$ in the denominator itself bound?

I was wondering if interpreting that $x$ as a symbol, in the sense of Bob Harper's Practical Foundations for Programming Languages (Chapter 1.2 on abstract binding trees) might solve these riddles, and if this had already been worked out by someone?

Here's a quote from PFPL:

It will often be necessary to consider languages whose abstract syntax cannot be specified by a fixed set of operators, but rather requires that the available operators be sensitive to the context in which they occur. For our purposes it will suffice to consider a set of symbolic parameters, or symbols, that index families of operators so that as the set of symbols varies, so does the set of operators. [...] The only difference between symbols and variables is that the only operation on symbols is renaming; there is no notion of substitution for a symbol.

  • 2
    $\begingroup$ You can rationalize $\frac{dx^2}{dx}$ as $D(\lambda x.x^2)(x)$ in which case there are two variables, both called $x$, and one of them is bound by the differentiation operator. $\endgroup$ – Derek Elkins left SE Oct 9 '17 at 12:02
  • 2
    $\begingroup$ @MichaelBächtold: Isn't the logical conclusion of what you say that $\frac{d(f(x))}{dx}$ is broken notation that we should stop teaching? $\endgroup$ – Andrej Bauer Oct 9 '17 at 16:07
  • 1
    $\begingroup$ @AndrejBauer I'm not sure if the impossibility of making sense of $\frac{dy}{dx}$ is a logical conclusion of what I said. I hope it can still be salvaged. Here's a related piece of notation (that no one uses) but I suspect to be safe: Let $V$ be a vector space with basis $v_1,\ldots,v_n$. Since any vector $v\in V$ can be written uniquely as a linear combination of the base, we can extract the coefficient of $v$ in front of $v_i$. Denote this coefficient with $\frac{v}{v_1^0\ldots v_i^1 \ldots v_n^0}$. con't $\endgroup$ – Michael Bächtold Oct 9 '17 at 17:29
  • 1
    $\begingroup$ There's a reason I used the word "rationalize". Typical mathematical notation is rather ambiguous about what the scope of things is and about the distinction between a function and its value. I would write $D(f)(3)$ in more traditional notation as $\frac{d(f(x))}{dx}(3)$. It is indeed the case that $D(\lambda x.x^2)=D(\lambda y.y^2)$. This is just like how $f(x)=\frac{d(g(x))}{dx}(x)=\frac{d(g(y))}{dy}(x)$. Of course, people want to write just $f(x)=\frac{d(g(x))}{dx}$ and this is where the scoping gets all wonky. You can get some ways with viewing differentiation as a syntactic operation. $\endgroup$ – Derek Elkins left SE Oct 9 '17 at 23:37
  • 1
    $\begingroup$ If you take the more algebraic/syntactic approach, $x=0$ is already a contradiction, so deriving $1=0$ from it is no issue. This is completely obvious from the functional perspective: the above equation becomes something like $(\lambda x.x)=(\lambda x.0)$. While admittedly lost in the details, the thrust of my statements is that if you insist on using a notation that conflates a function and its values, and provides little to no means of delimiting scope and specifying variable dependencies, it becomes impossible to articulate what is actually going on. Confusion and "paradox" is the result. $\endgroup$ – Derek Elkins left SE Oct 10 '17 at 8:05

Some mathematicians find it natural to substitute for the variable in the denominator of a derivative, writing things like $\frac{d \log V}{d\log p}$. This suggests that the $x$ in the denominator $\frac{dy}{dx}$ is neither bound nor binding nor a symbol.

Rather $\frac{dy}{dx}$ seems to be an operation on "variable quantities" $y,x$ requiring some side conditions, similar to how the usual division $a/b$ requires $b\neq 0$, or how the operation $\frac{v}{v_1^0\ldots v_i^1\ldots v_n^0}$ in my comment requires $v_1,\ldots,v_n$ to be linearly independent and $v$ to lie in their span.

I suspect that the side condition for $\frac{dy}{dx}$ should be analogous: $dx$ needs to be linearly independent and $dy$ has to be a multiple of $dx$.

With these side conditions we are not allowed to substitute a constant for $x$ in $\frac{dy}{dx}$ since then $dx=0$ and so $dx$ is not linearly independent. That would explain why we are not allowed to write $\frac{d x^2}{d3}$ or to take the derivative of equation $x=0$ w.r.t. to $x$ to conclude $1=0$.

I'd be interested in hearing if someone sees an immediate problem with this interpretation.

(I see some subtlety in the fact that $d(x|_{x=3})$ cannot mean the same as $(dx)|_{x=3}$, since otherwise I'd expect $\frac{dy}{dx}|_{x=3}$ to be the same as $\frac{d(y|_{x=3})}{d(x|_{x=3})}$ which would not be allowed by the side condition. On the other hand, that is not so strange from a differential geometric point of view if we read $|_{x=a}$ as restriction: there is a difference between restricting a differential form and pulling it back. But I need to think more about this.)

  • $\begingroup$ You can't substitute a constant for $x$ in $d(x^2)/dx = 2x$, but you can in $d(x^2) = 2x\,dx$. Similarly with $d(x^2 + y^2) = 2x\,dx + 2y\,dy$ (but not $\partial(x^2 + y^2)/\partial{x} = 2x$) and even $v = a^1 v_1 + \cdots + a^n v_n$ (but not necessarily $v/v_1^0\ldots{v_i^1}\ldots{v_n^0} = a^i$). $\endgroup$ – Toby Bartels Aug 20 '18 at 7:41
  • $\begingroup$ Thanks for your comments @TobyBartels. I didn't understand the last part with $v/v_1^0\ldots v_i^1\ldots v_n^0$. I was thinking of the $v$'s as vectors, and it's not clear what it would mean to substitute a constant for one of them. But I could imagine substituting $v_1+2v_2$ for $v_1$ for example. $\endgroup$ – Michael Bächtold Aug 20 '18 at 12:37
  • $\begingroup$ In the vector case, the analogue of a constant would be the zero vector. As long as you changed $v$ to match (assuming that you want to keep the equations true), you could substitute $v_1 + 2v_2$ for $v_1$ in either version, and you could substitute $0$ for $v_1$ in the version written with multiplication, but it wouldn't make sense to substitute $0$ for $v_1$ in the version using (something that looks like) division, because that would violate the criterion of linear independence in that notation. (It also wouldn't make sense to substitute $2v_2$ for $v_1$, for that matter.) $\endgroup$ – Toby Bartels Aug 21 '18 at 20:42
  • $\begingroup$ If you start with $d(x^2)/dx=2x$ and substitute $y^3$ for $x$, then you get the true statement $d(y^6)/d(y^3)=2y^3$ (which can simplify further to $(6y^5\,dy)/(3y^2\,dy)=2y^3$). Also, if you substitute $y^3$ for $x$ in $d(x^2+y^2)=2x\,dx+2y\,dy$, then the result is $d(y^6+y^2)=2y^3\,d(y^3)+2y\,dy$, which simplifies to $(6y^5+2y)\,dy=2y^3\cdot3y^2\,dy+2y\,dy$, which is true. However, if you perform this substitution in $\partial(x^2+y^2)/\partial{x}=2x$, then you get $\partial(y^6+y^2)/\partial(y^3)=2y^3$, which is false at best. $\endgroup$ – Toby Bartels Aug 21 '18 at 20:54
  • 1
    $\begingroup$ The point is that if you write the equations with multiplication, then there are no side conditions; while if you write them with division, then there are the side conditions that you mentioned. These side conditions generalize the rule from ordinary arithmetic that one cannot divide by zero. (I don't want to suggest that your side conditions are wrong; they are exactly correct.) $\endgroup$ – Toby Bartels Aug 23 '18 at 12:26

Functions of a single variable

We can define a operator $\mathcal{D}$ on functions $f: \mathbb{R} \to \mathbb{R}$ so that $\mathcal{D}(f)$ is the first derivative of $f$. It is common to write this operator without parentheses, i.e., write it as $\mathcal{D} f$ instead of $\mathcal{D}(f)$ and write $\mathcal{D} f(x)$ instead of $(\mathcal{D}(f))(x)$ or $\mathcal{D}(f)(x)$.

What about functions of multiple variables? We can define a operator $\mathcal{D}_1$ on functions $f: \mathbb{R}^n \to \mathbb{R}$ so that $\mathcal{D}_1(f)$ is the partial derivative of $f$ with respect to its first argument, i.e., $(\mathcal{D}_1 f)(x) = {\partial f \over \partial x_1} f(x_1,\dots,x_n)$. Then $\mathcal{D}_1(f)$ is a function with signature $\mathbb{R}^n \to \mathbb{R}$.

These operators $\mathcal{D},\mathcal{D}_i$ have a clear interpretation that avoids the ambiguities you mentioned.

On to interpret more standard notation. Here's the thing about standard notation. When someone writes $x^2$ in a math textbook, there are two things they might mean. If $x$ is taken as a free variable, this might represent a function, namely the function $\lambda x . x^2$. Or, if $x$ is taken as a bound variable, it might represent a number: it is the value of $x$, squared. Since the same notation is used for both, the reader has to infer which was intended based on the surrounding context. That's OK for mathematical exposition, but problematic for programming languages, where we need expressions to have an unambiguous meaning.

And the same ambiguity infects notation surrounding functions and derivatives. The expression $f(x)$ sometimes is used to represent the function $f$, and sometimes to represent the value obtained by evaluating $f$ at the input $x$. The expression ${df \over dx}$ is sometimes intended to represent the function $\mathcal{D} f$, and sometimes to represent the value of that function evaluated at the input $x$, i.e., $(\mathcal{D} f)(x)$.

When you see someone write something like ${d \over dx}f(x)$ or ${df \over dx}(x)$ where $f$ is a function of one variable, that might be intended to represent $\mathcal{D} f$ or $(\mathcal{D} f)(x)$: you have to look at context to guess what was intended. Mentioning "$x$" in the denominator is a bit sloppy since $x$ is a bound variable of the expression defining $f$; the $\mathcal{D}$ notation makes it clearer that $\mathcal{D}$ is an operator that takes a function and returns another function. This operator doesn't really care what name you give the bound variable.

What about ${dy \over dx}$ or ${d \over dx} y$? Sometimes, the context makes clear that $y$ is implicitly a function of $x$, i.e., $y = f(x)$. Then, this notation might refer to the function $\mathcal{D} f$, or it might refer to the number $(\mathcal{D} f)(x)$ -- you have to guess from context. Or, if you prefer, it might represent $\mathcal{D} \lambda x . \cdots x \cdots$, where "$\cdots x \cdots$" is some expression that describes how $y$ is computed as a function of $x$, or it might represent $(\mathcal{D} \lambda t . \cdots t \cdots)(x)$. In other words, $y$ might represent either a function (of $x$) or a number. In the former case, we are basically writing $y$ as a shorthand for $\lambda x . \cdots x \cdots$; the context has made clear how $y$ varies as a function of $x$, so in communication with humans we don't bother writing it out a second time.

Functions of multiple variables

What about functions of multiple variables? When you see someone write something like ${\partial \over \partial x_1} f(x_1,\dots,x_n)$, this is implicitly the same as either $(\mathcal{D}_1 f)(x_1,\dots,x_n)$ or $\mathcal{D}_1 f$ (you have to guess from context which was intended). Everything should now follow from the discussion above.

If we take this understanding that ${d \over dx}$ is syntactic sugar for $\mathcal{D}$ and $\mathcal{D}_i$, then it becomes clearer how to answer your questions. In particular, we only need to answer your questions for the $\mathcal{D}$ operators. And most of your questions go away, because $\mathcal{D}$ no longer mentions a variable $x$, so we don't need to answer whether $x$ is bound or not, whether you can substitute for $x$ in the denominator, etc.

Last question: is $\mathcal{D}$ a symbol in the sense that Harper meant? I don't know. You'll have to check the definition in that paper.

  • $\begingroup$ There are several things in your answer that I don't understand. "since $x$ is a bound variable of $f(x)$"? Then in one paragraph you say "interpret $\frac{d}{dx}f(x)$ as $Df(x)$", in the next you say "Interpret $\frac{dy}{dx}$ as $\frac{d}{dx}f(x)$ ... in particular interpret this as $Df$". Isn't that a contradiction? Or maybe you equate $f$ with $f(x)$? $\endgroup$ – Michael Bächtold Nov 22 '17 at 8:13
  • $\begingroup$ Also when you say "these operators are symbols" are you using the term symbol in the same technical sense Harper does? $\endgroup$ – Michael Bächtold Nov 22 '17 at 8:14
  • $\begingroup$ @MichaelBächtold, good points. I've edited my answer extensively to try to clarify these points. Can you read through my answer a second time and see what you think now? $\endgroup$ – D.W. Nov 22 '17 at 17:44
  • $\begingroup$ Thanks. I can fully understand where this answer comes from. It seems to be in line with what Derek Elkins and Andrej Bauer say in the comments. My suspicion though, is that this modern perspective (which became popular among mathematicians and logicians only around 1900) ignores a useful interpretation of the concepts of "variable" and "function", that was predominant prior to 1900 (and seems still to be awaiting a proper direct formalisation). $\endgroup$ – Michael Bächtold Nov 24 '17 at 9:26
  • $\begingroup$ This comment section is to short to explain better what I mean, but maybe you can get a glimpse of the historical development of this change of perspective by reading Frege's 1904 "What is a function?" and by looking at the discussion here. $\endgroup$ – Michael Bächtold Nov 24 '17 at 9:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.