In order to compare 2 complexities just calculate a limit of their ratios as below (here we use L'Hôpital's rule to simplify calculating a limit for $\frac{\ln(k)}{k}$:
As you can see, $O(n^2\times\log(n))$ is lower than the other.
In order to compare 2 complexities just calculate a limit of their ratios as below (here we use L'Hôpital's rule to simplify calculating a limit for $\frac{\ln(k)}{k}$:
As you can see, $O(n^2\times\log(n))$ is lower than the other.