9
$\begingroup$

I'm trying to understand how Automatic Differentiation (AD) works.
For simple algebraic operation, I get the chain rule thing. But, when the code contains conditional statement like

1: test_sign = x*y < 0
2: if test_sign :
3:     biggest = max(x,y)
4: else
5:     smallest = min(x,y)

Does AD work?
If it works, can you explain why?

When the code executes line 1, how does AD interpret the inequality <?
Let's say, my AD is in foward mode, what is the differential for the if then else branch?

My understanding is that the above code use non differentiable function, in the sense that left and right derivatives are not the same. So, how does AD pick the good derivative?

Also, if I'm in forward mode, it is possible that line 5 is never visited. But when AD bumps x to x+h, line 5 should be evaluated. So the AD tape will incorrectly always differentiate line 3, instead of differentiation line 5.

| cite | improve this question | |
$\endgroup$
  • 1
    $\begingroup$ If your function behaves differently for different parts of your domain, shouldn't AD just use the same if-else? That's how I would implement it... $\endgroup$ – adrianN Sep 9 '13 at 10:09
  • $\begingroup$ @adrianN When I wrote the question I had in mind, a library for AD, like ADOL-C. Your comment makes sense, so if I have an if-else, it's better if I handle this part of the AD myself. $\endgroup$ – user10018 Sep 9 '13 at 13:37
  • 2
    $\begingroup$ Possible duplicate of Looping and branching with Algorithmic Differentiation $\endgroup$ – breandan Jun 29 '17 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy