How not to sort by average rating

17 Oct 2017 in Algorithms

An interesting article about rating based sorting:

PROBLEM: You are a web programmer. You have users. Your users rate stuff on your site. You want to put the highest-rated stuff at the top and lowest-rated at the bottom. You need some sort of "score" to sort by.

WRONG SOLUTION #1: Score = (Positive ratings) − (Negative ratings)

WRONG SOLUTION #2: Score = Average rating = (Positive ratings) / (Total ratings)

CORRECT SOLUTION: Score = Lower bound of Wilson score confidence interval for a Bernoulli parameter

with a ruby implementation of:

require 'statistics2'

def ci_lower_bound(pos, n, confidence)
   if n == 0
       return 0
   end
   z = Statistics2.pnormaldist(1-(1-confidence)/2)
   phat = 1.0*pos/n
   (phat + z*z/(2*n) - z * Math.sqrt((phat*(1-phat)+z*z/(4*n))/n))/(1+z*z/n)
end

Whole article at evanmiller.org/how-not-to-sort-by-average-rating.html.