1
$\begingroup$

Is there a way to estimate the number of unique monthly visitors to a site based on a limited sample of one week of data? I have information about when a given user visited the site. This isn't as simple as just multiplying the number of unique visitors the first week by 4, due to the hotel problem. If 10 people visit your site the first week and the same people are the only visitors to your site the second, third, and fourth week, the total number of monthly unique visitors to your site is only 10.

I know you can use HLL to estimate the number of unique visitors to a site in O(1) space. I'm wondering if there's a similar approach to estimate how many unique visitors there will be after a month, preferably that also works in O(1) space.

$\endgroup$
0
$\begingroup$

After digging into this for a bit, I came across this paper which provides a solution. The paper gives an approach to estimating the number of new species that will be observed given an initial sampling period. It models observing a given species as a Poisson distribution. It gives the following estimator for the number of new species that will be discovered:

$$ \hat{\Psi}(t)=\sum_{k=1}^{k_{m a x}} N_{k} e^{-k}-\sum_{k=1}^{k_{m a x}} N_{k} e^{-k(1+t)} $$

where $N_{k}$ is the number of species that were observed $k$ times and $t$ is the length of the second sample relative to the initial sample.

$\endgroup$
-1
$\begingroup$

Not without making assumptions. You can't tell the difference between a site with a $N$ users who visit once a week (on a random day of the week), vs a site with $4N$ users who visit once a month (on a random day of the month).

$\endgroup$
9
  • $\begingroup$ Of course. Presumably there is some approach that factors in how often a new user shows up to come up with better estimate. I'm looking for any approach that could take a sample of information of the users (not necessarily just the number of unique visitors over the course of a week) and estimate the number of unique visitors for a month. $\endgroup$ – malisper Aug 1 '20 at 20:35
  • $\begingroup$ @malisper, I've answered the question as stated. Doing what you suggest would require extra assumptions or extra information not stated in the question. We can only answer the question that was stated; we can't read your mind about what other assumptions you might have in mind or be willing to make. Also, it is helpful to tell us in the question what you already know about the problem so we don't waste your time or our time telling you something you already know. $\endgroup$ – D.W. Aug 1 '20 at 21:23
  • $\begingroup$ Ok. I edited the question to try to make it clearer. Does it make sense? I think some of the confusion was around "I have one week of data". It seems you interpreted that as I know the unique number of visitors after one week, when I meant I have one week of data on how visited my site and when. $\endgroup$ – malisper Aug 1 '20 at 21:45
  • $\begingroup$ @malisper, there was no confusion. I understand what you meant from the start. My answer continues to apply. Having all the data for a single week doesn't resolve the issue I have pointed out. $\endgroup$ – D.W. Aug 1 '20 at 22:19
  • $\begingroup$ Aren't there statistical approaches you can apply? The German tank problem seems to be a similar problem. For the German tank problem you can't tell what N actually is, but there are still approaches to estimate it. I would think the same thing applies here. I don't know what N actually is, but there seems like there should be ways to approximate it. $\endgroup$ – malisper Aug 1 '20 at 22:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.