2
$\begingroup$

Determining whether or not an array has duplicate entries has two straightforward solutions:

  • Build a hashset of entries, then search for elements in this hashset. This takes $\mathcal O(n)$ time and $\mathcal O(n)$ extra space.
  • Sort the array, then search for consecutive elements that are the same. This takes $\mathcal O(n \log n)$ time and $\mathcal O(1)$ extra space.

Is there an algorithm that can solve this problem with the best of both approaches, using only $\mathcal O(n)$ time and $\mathcal O(1)$ extra space?

Question Finding duplicate in immutable array in linear time and constant space is similar, but the solution to that question only works when the values come from the set $\{1, ..., n\}$; my values are large integers. Also unlike that question, I allow the input array to be modified and used as a workspace.

One strategy might be to attempt to turn the array into a kind of hashtable. However, this seems like it won't work because of the lack of empty space (which makes both moving objects into place hard, as well as getting $\mathcal O(1)$ queries.).

However, I suspect that this cannot actually be done, but I'm not sure how to go about proving it.

Improve this question
$\endgroup$
6
  • $\begingroup$ The hashset method takes $\Theta(\log n)$ time. Hash table operations have $\log n$ cost, even if the constant is low in practice and they're often approximated as $O(1)$. $\endgroup$
    – Gilles 'SO- stop being evil'
    Jul 25, 2019 at 6:24
  • $\begingroup$ You can radix-sort array in place using MSD sort $\endgroup$
    – Bulat
    Jul 25, 2019 at 6:35
  • $\begingroup$ And if you need fast practical approach, look at cs.stackexchange.com/questions/93563/… $\endgroup$
    – Bulat
    Jul 25, 2019 at 6:38
  • $\begingroup$ @Bulat A radix sort is still a sort, and takes $\log n$ time in the worst case. A radix sort can take $O(1)$ time if the elements are small integers (with a bound that doesn't depend on the array size), but the question explicitly rules this out (“my values are large integers”). $\endgroup$
    – Gilles 'SO- stop being evil'
    Jul 25, 2019 at 6:43
  • $\begingroup$ This "log n" is actually a log of value range. It's up to you how to count it, but at least I find your measurement non-standard and deletion of my answer unreasonable. If you don't agree with Wikipedia, use comments rather than moderation tools to promote your opinion $\endgroup$
    – Bulat
    Jul 25, 2019 at 6:50

1 Answer 1

Reset to default
2
$\begingroup$

Suppose that your elements come from a domain of size $n^2$. Any algorithm using time $T$ and space $S$ corresponds to a branching program having depth $T$ and containing at most $T \cdot 2^S$ nodes. Theorem 6.13 of Time-space tradeoff lower bounds for randomized computation of decision problems shows that $$ T = \Omega\left(n \sqrt{\log \tfrac{n}{S + \log T}/\log\log \tfrac{n}{S + \log T}}\right). $$ In particular, if $S$ is $O(\log n)$ (which corresponds to your $O(1)$ extra space, assuming you cannot modify the input) then $T = \Omega(n\sqrt{\log n/\log\log n})$.

Improve this answer
$\endgroup$
10
  • $\begingroup$ For elements from domain of P(n) size, hashing will take O(log(n)) time which contradicts to what he was said. No, it's time to delete your answer ;) $\endgroup$
    – Bulat
    Jul 25, 2019 at 8:04
  • $\begingroup$ Hashing takes linear time. I don't follow. $\endgroup$
    – Yuval Filmus
    Jul 25, 2019 at 8:18
  • $\begingroup$ If domain size is n^2, then element contains 2*log(n) bits, so its hashing requires O(log(n)) time. Where I'm wrong? $\endgroup$
    – Bulat
    Jul 25, 2019 at 8:24
  • $\begingroup$ In the branching program model, querying an element is an atomic operation. It's an $n^2$-way branching program. $\endgroup$
    – Yuval Filmus
    Jul 25, 2019 at 8:25
  • $\begingroup$ I.e. the program size is depends on the input size? In order to generate such program, you need O(n^2) time. If we can use O(n^3) branching as a single operation, we can choose proper branch based on the entire array contents, so "solving" problem in O(1) time. $\endgroup$
    – Bulat
    Jul 25, 2019 at 8:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.