Consider the following eqautions used in LSTM ( taken from Andrew ng's course on Sequential model)
In an LSTM model, LSTM Cell has three inputs at any time step t
- Input($X_t , a^{(t-1)}, C^{(t-1)})$,
Here $X_t$ is the input vector, $a^{(t-1)}$ is the previous hidden state and $ C^{(t-1)}$ is the previous cell state
Now the new cell state $c^t$ is given by the following formula :
$C^t = $ forget_gate * $C^{(t-1)} + $ update_gate* $\overline{C^t}$
Question:
If I swap the places of forget_gate and update_gate, I still get a valid $C^t$, So why are we multilyting the previous cell state with forget gate only and the current cell state with update gate only, what if Imultiply previous cell state with update gate ?
Edit : After swapping, the formula would look like this,
$C^t = $ update_gate * $C^{(t-1)} + $ forget_gate* $\overline{C^t}$