# Batch sizing for convolution neural networks — powers of 2 or powers of primes?

The conventional wisdom for convolution neural networks (CNNs) is to make the batch size a power of 2 because of hardware utilization/optimizations done in the convolution layers.

A similar logic applies to the Fast Fourier Transform (FFT). However, the FFT is actually best used when it is a size of a power of primes (e.g. $$2^M3^N5^P$$ where $$M,N,P$$ are positive integers.) I was wondering if this also applies to CNNs.

• This depends entirely on the implementation details. Have you had a look at the internals of whatever you care about or simply tried it? – Juho Jan 2 '19 at 17:37
• I am using cudnn which doesnt have source available that I am aware of. Also, I have tried to measure performance it seems powers of primes is just as good as power of 2. However, this could be by sure luck or some other inefficiency in my training regime. – Isaac Gerg Jan 2 '19 at 17:45
• There are plenty of implementations of both CNNs and FFTs that you can peruse; many of them are open source. – D.W. Feb 11 '19 at 0:03