# Batch regularization & L2 regularization

Is performing batch regularization in addition to L2 regularization redundant?

Batch regularization: http://arxiv.org/abs/1502.03167

Notice how when performing batch regularization, you forgo using a bias term (and instead normalize your layer output, followed by scaling with a 'scale' term, and shifting with a 'beta' term. I was under the impression that L2 regularization punishes large weights & biases... but if I'm not using biases because of batch regularization, then would I just punish the weights with L2 regularization?

Thanks--

Batch normalization still has a bias term that is added after the normalization step (the $\beta$); it does not eliminate it. L2 regularization penalizes large weights and large biases. As far as I can tell, batch regularization doesn't try to do either (at least not explicitly).
When using L2 regularization with batch regularization, I imagine you could either L2-penalize just the weights; or L2-penalize the weights, the $\gamma$ terms (scaling), and the $\beta$ terms (shift). I don't know which will produce better results. My intuition says the latter might be preferable, but experiments and data beat intuition any day.