I'm trying to understand how Automatic Differentiation (AD) works.
For simple algebraic operation, I get the chain rule thing.
But, when the code contains conditional statement like
1: test_sign = x*y < 0
2: if test_sign :
3: biggest = max(x,y)
4: else
5: smallest = min(x,y)
Does AD work?
If it works, can you explain why?
When the code executes line 1, how does AD interpret the inequality <
?
Let's say, my AD is in foward mode, what is the differential for the if then else
branch?
My understanding is that the above code use non differentiable function, in the sense that left and right derivatives are not the same. So, how does AD pick the good derivative?
Also, if I'm in forward mode, it is possible that line 5 is never visited. But when AD bumps x
to x+h
, line 5 should be evaluated. So the AD tape will incorrectly always differentiate line 3, instead of differentiation line 5.