Can the man in the middle not just take the keys swapped by the opponents, change the keys and then decrypt and encrypt the message again?
Yes, they can.
A key exchange protocol like (the "textbook" version of) DH is secure against eavesdropping (i.e., simply observing what is being transmitted on the channel), but completely breaks down against man-in-the-middle (MITM) attacks, as you have stated.
Certificates are an attempt remedy this, but another problem arises: How can you ensure both parties receive the correct certificate? Obviously you cannot just send the certificates over the insecure channel since this is again susceptible to a MITM attack.
The solution is the existence of an alternative, (completely) secure channel. This would be either the two parties meeting in person and exchanging their certificates physically or over some alternative, trusted channel (e.g., over telephone, if it can be trusted).
In computer networks, the alternative channel is usually a public-key infrastructure (PKI). This means your operating system or browser has a set of preconfigured root certificates from which other certificates are signed (and possibly even further certificates using these as intermediate certificates). Hence, when you visit some website, it presents a signed certificate, which is signed using (a chain of) certificates which you already trust. Then, by using this certificate, an authenticated key exchange is possible (e.g., to agree on an ephemeral key to use with ordinary symmetric encryption).