eep!

[athena]

did anyone else notice we have a three-way tie at the moment for the top of the Theatrical Shorts (http://keyframeonline.com/Top5/#theatrsh) category?


uhh, somebody go review something... :D

[MonkeyFunk]

Maybe it's time the qualifying number of reviews was bumped up to 4...

[athena]

Hmm... I just double-checked in the admin, and we have enough reviews to complete a Top 5 list with the minimum raised to four...


Duck Amuck is now back on top.

[GrafSpee]

So how is the score computed for the best & worst lists? Is it merely the average of the reviews (so as more people review a film, the harder it is to stay on top) or is it some weighted average? Or is it some kind of scheme where the highest and lowest reviews are dropped and the rest are averaged out?

[athena]

So how is the score computed for the best & worst lists? Is it merely the average of the reviews (so as more people review a film, the harder it is to stay on top) or is it some weighted average? Or is it some kind of scheme where the highest and lowest reviews are dropped and the rest are averaged out?

I wish I was a coding genius or we had more reviews, so that we could do things like elaborately weighted Top 5's, but at the moment the system works like this...

- a minimum number of reviews is required to be considered for the Top 5 list. Usually it's 5, but in some of the low reviewing areas we've dropped it back to as low as 3 reviews

- At that point the list is based on the average star-rating... except in the event of a tie, the profile with the more reviews takes the lead



I know it's not a perfect system because, as you alluded to, the #1 spot usually ends up going to a profile with five 4-star reviews... until someone posts a single 3.5 review and it gets shuttled down the list.

Getting some stability into those Top 5s is something we're still searching for, so I'm definitely open to suggestions.

[GrafSpee]

Yeah, the problem is that the population sizes are generally small (as far as the number of votes for most profiles).

As an experiment, I decided to take the Direct-to-Video category (relatively small group to work with) and took all profiles that had 4 votes or more and took an average of the number of votes for those profiles. It came out to be 6.82. Then, all profiles that had more than 6.82 votes, I dropped the worst and best scores and then took the average of the remaining votes. The profiles that had less than 6.82 just had their average votes taken. I ended up with:

1. Balto 3
2. Lion King 2
3. Beauty & Beast: The Enchanted Christmas
4. Balto 2
5. Bambi 2

It's the same five profiles that are already on the Top 5 best list (albeit, in a slightly different order). Bambi 2 was the only one with less than the mean number of votes.

The worst 5 was also close to the what is already there:

1. Secret of Nimh 2
2. Little Mermaid 2
3. Cinderella 2
4. Return of Jafar
5. Pocahontas 2

So, basically, it looks like the current scheme gives basically the same thing so you might as well stay with it.

[athena]

Thanks for going to the trouble of calculating all that.


I think what it comes down to is really just population. The higher we can raise that minimum number of reviews threshhold, the more stable those lists become.

[MonkeyFunk]

By the way... did you see my note in the editors' board about how the lists are showing more than 5 entries? I thought that had been sorted out...