Let me rephrase your algorithm (starting at a different base case):
Initialize P to be the empty list.
for n from 2 to MAX:
if no integer in P divides n:
add n to P
return P
Let $p_1,p_2,\ldots $ be an enumeration of the primes. The probability that $p_1,\ldots,p_i$ do not divide a number $n$ is roughly
$$ \prod_{j=1}^i \left(1 - \frac{1}{p_j}\right) \approx e^{-\sum_{j=1}^i p_j} \approx e^{-\sum_{j=1}^i j\log j} \approx e^{-\frac{1}{2} i^2 \log i}.$$
Therefore the inner loop runs for roughly this many iterations:
$$ \sum_{i=1}^\infty e^{-\frac{1}{2} i^2\log i} = O(1). $$
In total, the complexity is $O(n)$ divisions. In contrast, the Eratosthenes sieve requires $O(n\log\log n)$ additions.
For a fair comparison, we also need to factor it the computational complexity of operations on large numbers. Assuming that division can be done in time $O(m\log m)$ (which is the conjectured running time), where in our case $m = \log n$, your algorithm has bit complexity $O(n\log n\log\log n)$, matching the bit complexity of the sieve. However, the best known division algorithms are somewhat slower, both asymptotically and in practice, and so I expect your algorithm to be somewhat slower than the sieve.
Atkin's sieve uses only $O(n/\log\log n)$ additions and so is faster than both your algorithm and the Eratosthenes sieve. It also uses only $\tilde{O}(\sqrt{n})$ memory, compared to your $\sum_{p_i \leq n} \log p_i \approx \sum_{m=1}^{n/\log n} \log(m\log m) = \Theta(n)$, also shared by the Eratosthenes sieve.
reference-request
to see where its been studied.... also its efficiency depends some on the implementation of the filter function... $\endgroup$ – vzn May 20 '14 at 0:59wheel of factorization
. Although, it may not be theoretically work optimal but can be work optimal given specific computer architectures. $\endgroup$ – Polynomial Proton May 20 '14 at 1:38