2
$\begingroup$

Many (tensorflow) implementations of batch normalization seem to use moving statistics at its inference phase (R2RT, tf.contrib.layers.batch_norm).

But in the algorithm from the original paper, they use population statistics in the batch normalization function. enter image description here

The thing is, pardon me, I don't understand why most of the implementations do not precisely follow the original algorithm.

The paper itself very briefly mentions about 'moving average' but does not give specific instructions for algorithm implementations.

Can somebody save me please?

$\endgroup$
  • $\begingroup$ Welcome to Computer Science! Don't use images as main content of your post. This makes your question impossible to search and inaccessible to the visually impaired; we don't like that. Please transcribe text and mathematics (note that you can use LaTeX) and don't forget to give proper attribution to your sources! $\endgroup$ – Raphael Apr 30 '17 at 16:52
  • $\begingroup$ Welcome to Computer Science! We expect references to fulfill the minimal scholarly requirements and be as robust over time as possible. Please take some time to improve your post in this regard. We have collected some advice here. $\endgroup$ – Raphael Apr 30 '17 at 16:52
  • $\begingroup$ The original paper is not the bible! If it can be improved upon, there is no reason to blindly adhere to it. $\endgroup$ – Yuval Filmus May 3 '17 at 6:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.