If you remove the battery on the motherboard then the computer wouldn't have any way to tell the time.

This is also the case with mobile phones. If you let a phone discharge and then not recharge it for more than a few weeks it will also "forget the time" because the small auxiliary battery is discharged completely and nothing is powering on the real-time clock.

You could try to power on an old mobile phone if you have one and check it yourself to see that it "forgot the time".