Short answer:
Current computers are digital, and even binary, i.e. they represent numbers as strings of bits, and bits are "encoded" using some physical phenomenon such as electric voltages. Typically, the physical phenomenon can take two ranges of value that can be discriminated by comparison to a fixed threshold (low or high voltage). One could as well work with pressure in fluids or forces exercised by gears.
Every computation can be performed when you are able to build so-called nand gates, which are devices such that when you input two signals, the output is the logical complement of the logical and. In electronic computers, these gates are realized using transistors. But you can invent any other device that performs this logical operation.

By combining enough nand gates, you can represent any number, perform additions, multiplications and many other arithmetic operations. You can store data and route data where you want by means of addresses, which are also numbers. Last but not least, you can store numbers that encode a sequence of instructions and let you compose a program.