Moving my comment to an answer, while the word "port" is being used in the same non-technical sense for both "port-mapped I/O" and for "network ports", they have literally no technical connection.
In port-mapped I/O, the "ports" are accessed with special CPU instructions (
out in the x86 family) which communicate with hardware devices. For x86, at least, due to the limited "address space" of port-mapped IO (and other reasons), memory-mapped IO or direct memory access are used for bulk data movement. There's nothing magical about port-mapped I/O. Modern ARM processors, for example, don't have it at all and instead rely entirely on memory-mapped I/O.
Network "ports" are just pieces of data in TCP or UDP packets to allow multiplexing connections to a single IP address (or host because it technically could be over a network level protocol other than IP). As far as the network hardware and CPU is concerned, the port is just another piece of data in the packet like any other. The operating system will use the port to route the packet to the appropriate application on the host