One book I really like is Introduction to Algorithms: A Creative Approach, by Udi Manber. Unlike most other textbooks, he teaches how to come up with algorithms on your own. For each algorithm covered in the textbook, he provides an increasing progression of sections, the first one describing the most obvious approach and each successive attempt rectifying the mistakes of the previous one. It's an excellent text in my opinion.

I don't think it will be $O(n\log n)$. When your sweeping algorithm is right in the middle in the bad example that I mentioned, you will have n things stored in the queue. But an adversary can make the right edges of these n rectangles occur in any order. Since whenever you find a right edge, you need to decide which rectangle to remove from your queue, each of these events might take linear time to process. Linear number of events with linear processing time for each is $O(n^2)$.

So, btw, the combinatorial complexity of the union can be at most O(n) because each rectangle can contribute to only a constant number of turns on the boundary.

The standard definition of combinatorial complexity of the union is the number of vertices in the boundary, i.e., the number of places where the boundary takes a "turn". Your approach will perform badly for the following example: rectangle 1 starts at x coordinate 0 and ends at x coordinate n and has a height 1, rectangle 2 starts at x coordinate 1 and ends at x coordinate n-1 and has height 2, rectangle 3 starts at 2 and ends at n-2 with height 3 and so on.

Your solution is not really optimal. The number of intersections could be $\Omega(n^2)$ in the worst case. As Syzgy points out, a better solution can be constructed. Hint: think divide and conquer.

Here the variable is $n$. $r$ is a constant since it depends only on $a$ and $b$, which are constants themselves.

That's just how the big O notation works. 1/1-r is a constant and any constant can be written as O(1). For the other expression, big O picks out the highest power. I recommend reading up on big O on wikipedia if you are having troubles understanding this.