[1] discusses how to construct an ensemble of neural networks by giving each network a certain weight $\alpha_i$: \begin{equation} f_\mathrm{GEM}(\boldsymbol{x}) = \sum_{i=1}^N \alpha_i f_i(\boldsymbol{x}) \end{equation} where each $\alpha_i$ is obtained using a correlation matrix which is defined in the following way: \begin{equation} C_{ij} = E[m_i(\boldsymbol{x}) m_j(\boldsymbol{x})] \end{equation} where $m_i(\boldsymbol{x}) \equiv f(\boldsymbol{x}) - f_i(\boldsymbol{x})$, with $f(\boldsymbol{x})$ being the target function and $f_i(\boldsymbol{x})$ being the output.
I was wondering how should one compute $C_{ij}$ when the neural network has multiple outputs. Would one firstly calculate $\boldsymbol{m}_i(\boldsymbol{x}) \equiv \boldsymbol{f}(\boldsymbol{x}) - \boldsymbol{f}_i(\boldsymbol{x})$ for all examples and then compute $C_{ij} = E[\boldsymbol{m}^T_i(\boldsymbol{x}) \boldsymbol{m}_j(\boldsymbol{x})]$? I tried doing that (with MNIST data) but the accuracy was not better than when taking a simple average of all the networks in the ensemble. I would appreciate any help on this.
[1] M. P. Perrone and L. N. Cooper, "When networks disagree: Ensemble methods for hybrid neural networks," in Artificial Neural Networks for Speech and Vision. Chapman and Hall, 1993, pp. 126-142.