10
$\begingroup$

I'm trying to understand how Automatic Differentiation (AD) works.
For simple algebraic operation, I get the chain rule thing. But, when the code contains conditional statement like

1: test_sign = x*y < 0
2: if test_sign :
3:     biggest = max(x,y)
4: else
5:     smallest = min(x,y)

Does AD work?
If it works, can you explain why?

When the code executes line 1, how does AD interpret the inequality <?
Let's say, my AD is in foward mode, what is the differential for the if then else branch?

My understanding is that the above code use non differentiable function, in the sense that left and right derivatives are not the same. So, how does AD pick the good derivative?

Also, if I'm in forward mode, it is possible that line 5 is never visited. But when AD bumps x to x+h, line 5 should be evaluated. So the AD tape will incorrectly always differentiate line 3, instead of differentiation line 5.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ If your function behaves differently for different parts of your domain, shouldn't AD just use the same if-else? That's how I would implement it... $\endgroup$
    – adrianN
    Sep 9, 2013 at 10:09
  • $\begingroup$ @adrianN When I wrote the question I had in mind, a library for AD, like ADOL-C. Your comment makes sense, so if I have an if-else, it's better if I handle this part of the AD myself. $\endgroup$
    – user10018
    Sep 9, 2013 at 13:37
  • 2
    $\begingroup$ Possible duplicate of Looping and branching with Algorithmic Differentiation $\endgroup$
    – breandan
    Jun 29, 2017 at 16:10

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy