# How to count in linear time worst-case?

This question and this question got me thinking a little bit. For sorting an array of length $$n$$ with $$k$$ unique elements in $$O(n + k \log k)$$, we need to be able to store counts of values in the array. There are some suggestions, but I'm looking for a way to do this in worst case linear time. More specifically:

Given a list $$A$$ of $$n$$ elements with $$k$$ elements distinct, determine a list of tuples $$U = \{(x_i, c_i)\}^k$$ of all unique elements $$x_i \in A$$ such that $$c_i$$ is the count of element $$x_i$$ in $$A$$.

Here are some (failed) ideas I've had and have been suggested:

1. Balanced Binary Search Tree - With this it will take $$O(\log k)$$ to insert into the tree and increase values. After inserts we could do a tree traversal in $$O(k)$$. Thus, total time comes out to $$O(n \log k)$$ which is too slow.
2. Hash Map - With this we can get $$O(1)$$ expected inserts and thus $$O(n)$$ expected time. However, this is still not $$O(n)$$ worst case.
3. Empty Space Mapping - Find the minimum and maximum element in $$A$$. Allocate (but do not initialize) enough memory to cover this range. Use this memory basically as a hash map and include a random hash so that we don't try to access corrupted memory. This strategy presents issues. (1) It's probabilistic with very very very low probability of failing, but still not guaranteed. Using memory like this limits us to floating-point or integer constraints.
4. Associative Arrays - There are many other associative arrays that can be used, similar to hash maps and BSTs, but I am not finding any that match these constraints.

Maybe there is some obvious method I am missing, but I also think it could be potentially not be possible. What are your thoughts?

• It cannot be done in comparison model since the problem of element distinctness has a lower bound of $\Omega(n\log n)$ decision-tree complexity. – John L. Apr 24 '19 at 16:20
• @Apass.Jack, oh right that's correct. A trivial reduction I did not consider. If you write it up as a quick blurb answer, I'll accept. – ryan Apr 24 '19 at 16:37
• Why is the HashMap not assured amortized O(n) ? – StephenBoesch Apr 24 '19 at 21:04
• @javadba For example, suppose all elements are hashed to the same value. – John L. Apr 24 '19 at 21:19
• Ah ok so if it's an imperfect hashing. – StephenBoesch Apr 24 '19 at 22:04

This is a nice question.

In the comparison model or, what is more general, the algebraic decision-tree model, the problem of element distinctness has a lower bound of $$\Theta(n\log n)$$ time-complexity in the worst case as said in this Wikipedia article. So there is no algorithm to count distinct elements in linear time in the worst case, even without counting the duplicities.

However, it is not clear whether it can be done in another computational model. It seems unlikely in any reasonable deterministic computational model.

• Is this really an instance of the element distinctness problem? Just generating the tuples doesn't require the check for distinctness. Not disagreeing, just curious. – mascoj Apr 24 '19 at 21:00
• What I am saying is, if you can produce that tuple of distinct elements, then you can also solve the problem of element distinctness by checking if the size of the tuple is $n$. – John L. Apr 24 '19 at 21:06
• Good call. Thanks – mascoj Apr 24 '19 at 21:14

There exist randomized algorithms whose expected running time is $$O(n)$$; or where the probability that the running time takes longer than $$cn$$ is exponentially small in $$c$$.

In particular, randomly choose a 2-universal hash function, then use it to hash all of the elements of the array. This achieves the stated running times, if you choose the length of the output of the 2-universal hash appropriately.

As another example, you can build a randomized algorithm whose worst-case running time is $$O(n)$$ (it always runs in linear time, no matter what) and has a probability of error of at most $$1/2^{100}$$. (How? Run the above algorithm, and terminate it if it runs longer than $$cn$$ steps for some appropriately chosen $$c$$.) In practice, that's good enough, as the probability that your computer outputs the wrong answer due to a cosmic ray is already much higher than $$1/2^{100}$$.

Your approach 3 can be made safe using a solution to exercise 2.12 of Aho, Hopcroft, and Ullman (1974) The Design and Analysis of Computer Algorithms as described, for example, in Using uninitialized memory for fun and profit.

Basically, in addition to your array of N elements with the counts you have two arrays of N elements and one auxiliary count to create a sparse set indicating which of the counts are valid.

In C-like pseudocode:

uint* a = malloc(n);
uint* b = malloc(n);
uint* c = malloc(n);
uint len = 0;

get_count(uint x) {
uint idx = a[x];
return idx >= 0 && idx < len && b[idx] == x ? c[idx] : 0;
}

increment_count(uint x) {
uint idx = a[x];
if (idx < 0 || idx >= len || b[idx] != x) {
idx = len;
len++;
a[x] = idx;
b[idx] = x;
c[idx] = 0;
}
c[idx]++;
}


Practical implementation of the sparse set is discussed in this StackOverflow answer.

• PS c could be indexed on x or idx, but I used idx for better cache locality. – Peter Taylor Apr 25 '19 at 8:23
• I like the answer, but I've confused about what makes this safe. While, entirely improbable, couldn't you access a memory cell, which by some miracle has a "valid" entry in it even though it was never put there. If you just got unlucky with malloc? – ryan Apr 25 '19 at 16:50
• This solution only works if you have a large enough memory: if all array elements are in the range $1..u$, then you need memory of size at least $u$. In practice this is very limiting. The way we create a large virtual address space in practice is using page tables, which are a tree-based data structure; the hardware invisibly follows page tables for us. As a result, while we think of memory access as taking $O(1)$ time, if you're working in a large memory address space, each memory access actually takes logarithmic time (to traverse the page table tree structure). – D.W. Apr 25 '19 at 17:01
• @ryan, see research.swtch.com/sparse for what makes it safe. It's definitely a very clever trick. – D.W. Apr 25 '19 at 17:01
• @D.W., $3u+1$, but if $u$ is very large then you can do this on multiple levels, using an array of {a,b,c,len} structs for c instead of an array of counts. E.g. if you use radix 512 so that each of the arrays fits in a page (with 8-byte pointers), then you can go up to $u=512^3 = 134217728$ using at most $(3 \times 512 + 1)(1 + 2k)$ memory where $k$ is the number of distinct elements seen. – Peter Taylor Apr 25 '19 at 18:37