I'm interested in how does cpu read 32 bit word from memory .If the processor has 32 bits of address space then it can address 4,294,967,295 locations (or 4 gb) Does this mean that each location has 32 bits.?
Address Space and Word Size are indeed two different things. If my processor has an $n$ bit address space, then it can address $2^n$ addresses. Note that this is merely the number of addresses and is not $2^n$ GB.
The word size of a memory refers to the size of the smallest addressable unit of the memory. In other words if my processor has an $n$ bit address space and a word-size of $b$ bytes, then my processor can address $2^n$ locations each of which are $b$ bytes long.