One common way to arrive at a tight big-$O$ upper bound of the complexity of an algorithm is to estimate one. Then prove that estimate works.
Let me take your computation by nested loops as an example.
In this post $\log(x)$ means $\log_2(x)$. The first loop runs $\lceil\log(n)\rceil$ iterations. For each iteration in the first loop, the second loop runs $\lfloor\log(n)\rfloor+1$ iterations. For each iteration in the second loop, the third loop runs $\lfloor(n - j+1)/2\rfloor$ iterations, where $j>0$ is the loop variable of the second loop. Ignoring the floor function, the addition by the constant 1, the subtraction by $j$, the division by the constant 2 and the constant number of operations executed by each iteration in the third loop, I will estimate the upper bound of the number of total operations as $O(\log(n)\log(n)\,n)$.
Is that $O(n\log^2n)$ a tight upper bound of the the number of total operations $T(n)$? Yes, it is indeed.
Firstly, let us show it is an upper bound. We can assume $n>2$. Let $c_3$ be the number of operations executed by each iteration of the third loop.
$$ T(n) \le (\lceil\log(n)\rceil) (\lfloor\log(n)\rfloor+1)(n/2)c_3\\
Now let us show it is tight. Let us do not count the first iteration of the the second loop. That is, let us only consider the operation done by the second loop starting from $j=\lfloor n/2\rfloor$. Then for each iteration in the second loop, the third loop runs $⌊(n−j+1)/2⌋ \ge n/2-1$ iterations. Then for $n\ge6$,
$$ T(n) \ge (\lceil\log(n)\rceil) (\lfloor\log(n)\rfloor)(n/2-1)c_3\\
So we have proved from the ground up.
The above proof, although rigorous and clear, is not the usual way to reason about the big-$O$ because it is so exact as to require too much thinking. That is almost the kind of computation needed to get the exact asymptotic approximation. Here is a simpler version assuming you are familiar with the big-$\Theta$ notations.
The first loop run $\Theta(\log(n)$ iterations. For each iteration in the first loop, the second loop runs $\Theta(\log(n)$ iterations. For all but the first iteration in the second loop, the third loop runs between $n/2$ to $n$ iterations, which is $\Theta(n)$ iterations. In the first iteration of the second loop, the third loop runs no iteration. So $$T(n)=\Theta(\log(n)(\Theta(\log(n)-1)\Theta(n)=\Theta(\log(n))\Theta(\log(n))\Theta(n)=\Theta(n\log^2(n))$$
which implies $O(n\log^2n)$ is the tight upper bound.
By the way, in all computations above we are not counting the operations to maintains the indice $i,j,k$. That is ignored by convention, because they will not change our approximation in big $O$-notations unless in extraordinary situations. Or because in many situations we are only interested in the number of the "real" operations.
The reasoning about the big-$O$ notations can be an art or a precise process. On the one hand, you can be very "sloppy" when you are trying to get an estimate and even when you are reasoning. That is because you can hide or absorb constant factors into the notations. That way you can arrive at a possible answer very quickly. With practice and experience, you can be certain immediately most of the times once you got an estimate. On the other hand, you can also verify every part of the "sloppy" process by rigorous mathematical deduction or by applying a set of valid shortcut rules. In the end, we will arrive at a precise conclusion using the big-$O$ notations.