# Approximating Deep Neural Networks (DNNs) with Binarized Neural Networks (BNNs)

I am working currently as a research intern on Binarized Neural Networks where the weights and the activations of the network are binary. The architecture of this type of networks makes them memory efficient and computationally efficient, which makes them ideal for resource constrained environments, like embedded devices and mobile phones.

The interesting part about BNNs is that we can encode a binarized network as a CNF formula (Boolean Formula). Using this formula, we can verify some properties of the network like Robustness against adversarial examples (carefully crafted samples looking similar to usual inputs but designed to mislead a pre-trained model). We can also extract explanations that support neural network decisions, hence make the neural network explainable.

Currently, I am trying to make a DNN explainable by verifying its decisions using BNNs. The first direction of research is to reduce a DNN to a BNN. Of course the two networks should be equivalent. I am researching ways to make this reduction but I haven't found any works in the subject. Is it possible to carry out this transformation ? Is there any techniques that can "binarize" a DNN ?

Thanks :)

• If you increase the inner dimensionality of a DNN by replacing all of the weights with vectors of their binary representation, does that count as a BNN? – Oso Feb 27 at 15:31
• It will be considered as a BNN if you use binary activations (like the sign function). However, the question is: are the two networks equivalent ? will the output of a particular input be the same when using the two networks ? Probably not. – Achraf El Khamsi Feb 27 at 15:46
• Well you also have to adjust the operations to consume binary instead of decimal, but aside from that you're just changing the data representation, which is reversible. – Oso Feb 27 at 15:48