Let's say I visit an IP Checker online, is the OP shown there my IP or the IP of my router? I can't seem to find a conclusive authoritative answer online.
It depends on how your network is configured.
Since you talk about a router, most domestic users have a public IPv4 assigned to their router and then a private subnetwork for all the home devices. The router is used as a gateway to the internet and it handles translating the private addresses to the public address through a mechanism called NAT. In this case you'd see the public IP assigned to your router.
The situation could be different for IPv6 addresses. For example your provider can assign a /64 IPv6 prefix to your router, which then advertises it to your network. Each device builds its own (public) IPv6 address by combining the prefix with a 64-bit device-specific suffix (for simplicity, think of it as the MAC address of the network interface card, although this is not always the case).
Keep in mind that these are just two examples. Only you can know how your network looks like.