AES NI seems to perform AES operations much faster than doing in software. However, if I have a machine with a large number of cores (say 32 cores), can I perform 32 AES encryptions using AES NI instruction i.e. one per core? Or can I just run one AES instruction?
It depends mostly on the memory bandwidth. Each core contains the dedicated AES hardware, but the CPUs need to obtain the data to encrypt or decrypt somehow. Even with a single core you have to be careful about how you dispatch instructions to get maximal performance, see Intel's white paper, page 48 bottom.
If you already have the machine, you can always try it out and let us know how it went.
Often the AES algorithm cannot be parallelised, because it is inherently serial.
Every round uses the result of the previous round as input.
Furthermore AES is usually run in CBC mode where again the previously encrypted block is used as input for the next block.
EBC mode is inherently insecure and should never be used.
Some modes of AES can be run in parallel, the Intel docs state:
The AES instructions provide a substantial performance speedup to bulk data encryption and decryption. When using parallelizable modes of operation, such as CBC decryption, CTR, and CTR-derived modes (GCM), XTS. The per formance speedup could exceed an order of magnitude over software-only, lookup tables based AES implementations. In scenarios where pipelined operation is impossible, for example in CBC encryption, the performance speedup would still be significant, around 2 to 3 times over (unprotected) software implementation.
You can however use those 32 cores to encrypt 32 different files at the same time...
Just don't try and speed up encryption by encrypting small snippets, because then you're recreating ECB mode with all its associated disadvantages.
The AES NI hardware relies heavily on lookup tables hardcoded in the chip.
(That's why it's so fast, it replaces lookup in memory with lookup on chip and it replaces bit manipulations in code with bit manipulations in silicon).
None of these operations require global state or set a global mode inside the chip, so you can run AES instructions on each of your cores at the same time.
Because single threaded hardware AES is so blazingly fast you will however run into bandwidth problems getting your data in and out of those 32 cores.
You'll quickly overload the memory bus.
From page 48 onwards the Intel documents explain how to optimize parallel operations with AES-NI.