0
$\begingroup$

I have asked it already on math.stackexchange.com and stackoverflow.com but they said it was off-topic and could not help me. They pointed me to cs.stackexchange.com so hopefully somebody here can help me or atleast point me in the right direction.

We have been using a bayesian average on a website of ours for a couple of years now and it seems to have working great but when using the same formula with other (good) data on another website the output was a bit disturbing.

We use this PHP code:

<?php
$avg_num_reviews = 6264.3636363636;
$avg_rating = 4.527272727272727;
$this_num_review = 4;
$this_average_rating = 4;
$bayesian_average = (($avg_num_reviews * $avg_rating) + ($this_num_reviews * $this_average_rating)) / ($avg_num_reviews + $this_num_reviews);
echo $bayesian_average;
?>

This normaly is filled in dynamicly but to give you an idea we will put in the data manualy.

So this data:

$avg_num_reviews = 6264.3636363636;
$avg_rating = 4.527272727272727;

is correct.

If we then calculate the Bayesian average, then in my opinion something is going wrong. It would help us enormously if someone could say something meaningful about this.

$this_num_reviews = total number over reviews for this company;
$this_average_rating = the average rating of this company;

Now we get to the weird part.:

Company A

$avg_num_reviews = 6264.3636363636;
$avg_rating = 4.527272727272727;
$this_num_review = 4;
$this_average_rating = 4;
$bayesian_average = (($avg_num_reviews * $avg_rating) + ($this_num_reviews * $this_average_rating)) / ($avg_num_reviews + $this_num_reviews);

$bayesian_average = 4.5269362613254;

Company B

$avg_num_reviews = 6264.3636363636;
$avg_rating = 4.527272727272727;
$this_num_review = 111;
$this_average_rating = 4.2;
$bayesian_average = (($avg_num_reviews * $avg_rating) + ($this_num_reviews * $this_average_rating)) / ($avg_num_reviews + $this_num_reviews);

$bayesian_average = 4.5215746565744;

Company C

$avg_num_reviews = 6264.3636363636;
$avg_rating = 4.527272727272727;
$this_num_review = 15468;
$this_average_rating = 4.4;
$bayesian_average = (($avg_num_reviews * $avg_rating) + ($this_num_reviews * $this_average_rating)) / ($avg_num_reviews + $this_num_reviews);

$bayesian_average = 4.4366864211353;

If we would sort this based on the bayesian average it will be like this:

  1. Company A
  2. Company B
  3. Company C

But how can it happen that a company (Company C) with 15468 reviews and an average of 4.4 has a lower Bayesian average than a company (Company A) with 4 reviews and an average of 4?

The Bayesian average should ensure that these kinds of things are prevented.

The logical sequence should be as follows:

  1. Company C
  2. Company B
  3. Company A

When we re-calculate it by hand the outcome is the same.

Can someone explain to me why this happens and whether there might be something wrong in the formula?

Am I missing something or should I use a different formula to do this better?

If anyone can say something meaningful about this, I would be very grateful to that person because I am a bit off track at the moment.

$\endgroup$

1 Answer 1

1
$\begingroup$

There's nothing wrong with the formula. It's doing exactly what you told it to do. I suspect you might have a misunderstanding of how to apply the Bayesian average.

By setting $avg_num_reviews to a huge number like 6264, you've basically told the Bayesian average that you have extremely high confidence that the average rating should be 4.527. It takes a gigantic number of reviews to override this prior. So of course your formula assigns Company A an average of close to 4.527, because the number of reviews of Company A is so vastly smaller than 6264 that the prior dominates; but your formula assigns Company C an average close to the average of C's ratings, because 15468 is significantly larger than 6264 and this is enough to overcome the strong prior.

I would expect more typical values for $avg_num_reviews would be vastly smaller, like 10. It's basically a parameter you can set, which controls how strongly you want the prior to be. Perhaps your mistake is setting it to be equal to the average number of reviews per company -- that doesn't seem like a good choice to me.

See https://en.wikipedia.org/wiki/Bayesian_average.

$\endgroup$
2
  • $\begingroup$ Thank you so much for this explenation. That makes sense and when I change de $average_num_reviews to a lower number the outcome becomes more realistic. But this is not the way we would like to use it, put an hard number in this field. This should be dynamic but if I understand you correctly it is not working like this because the big change in the number of reviews and the average that is calculated based on them? Is there a way to solve this without setting the number to a static number like 10? $\endgroup$ Mar 20 at 10:40
  • $\begingroup$ @user2812779, sorry, I can't understand what you are asking, and this platform is not designed for back-and-forth questions. It is intended as a site where people can collaborate to build a knowledge base that will be useful to others, in the form of questions and answers. Perhaps if you identify the requirements of how you'd like your system to work and what properties you want it to have, you can ask on Statistics.SE for how to achieve those goals. $\endgroup$
    – D.W.
    Mar 20 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.