Most of the operations in computers are using floating-point arithmetic, and why a Floating Point Unit alone is not sufficient? Can we do away with ALU? Is FP operations are resource-intensive alone be the reason for this, over the advantages provided by FP operations?
Any 64 bit integer can be represented as an 80 bit IEEE 754 extended precision floating point numbers. Many processors (PowerPC and POWER, ARM) don't support extended precision floating point, so there is your first argument gone. They can't represent every integer number as a floating point number.
Most operations on integers can be done very cheaply with integers represented in an integer format, but are much more expensive if you try to implement them with numbers in a floating point format. That will reduce your clock speed and at the same time raise energy consumption, costing you battery life on any portable device.