Do Rotten Tomato Scores Matter?
Have you ever checked the Rotten Tomato score of a movie before deciding to watch it? And then which one do you check–the critic or audience score? Will either matter in your enjoyment of the movie? I’ve decided to take a stab at answering these questions by examining the 2022 movie ratings of three roommates (one of them being my wife’s sister, hence my access (with permission) to their scores). These are three professional women in their twenties living in Provo, Utah. I’ll refer to them as F, N, and A. This year they have watched twenty-five films so far, and you can see their self-rated levels of enjoyment (out of 10) of each film and the Rotten Tomato Critic score (rescaled to 10) below. You can click on the image to make it larger.
It is hard to visualize for each movie, so let’s average their ratings, and throw in the Rotten Tomatoes audience score too: And check out their distributions:
If you’re unfamiliar with boxplot charts, see this handy guide. In short form, though, it’s just an easy way to visualize data dispersion. The taller the boxes, the more dispersed the observations. From both of the above graphics, you can see that the averages (whether measured by mean or median) between the three roommates are very similar; they have highly comparable reactions to these movies on average, which is probably a good thing for roommate relations. It’s funner to watch movies with people who share your tastes. N is a bit of a wild card though; she is both more positive and negative about the films they watch. She has the greatest range of reactions, despite landing on the same median score as her roommates.
Interestingly, the roommates tend to be nearly as critical of movies as the critics on Rotten Tomatoes, it is again just a question of range.1 General audiences, however, tended to be much more enthusiastic about these movies when compared to the roommates.Makes sense–most people prefer going to watch movies they know they’ll enjoy.
Now let’s consider the relationship between the critics’ score and the enjoyment levels of each of the roommates.
F’s enjoyment is modestly correlated with the critics’ score.2 A one unit increase in the critics’ score for a movie is associated with a 0.23 increase in F’s enjoyment score. Using the original units, this would mean that when comparing two movies, where 80% of critics enjoy the one, and 90% of critics enjoy the other, F will enjoy the latter by about two and a half percentage points. Not a huge change unless you’re comparing two movies with Rotten Tomato scores of, let’s say, 20% and 90%. F should enjoy that movie about 16 percentage points more.
Similar but weaker for N.3 For her, a unit change for the critics is only associated with a 0.14 change for N’s enjoyment score.
And A’s scores show an even weaker correlation with the critics’ scores.4 Notice how flat the regression line is, which predicts the relationship. Each observation is far from that line, meaning there is essentially no meaningful relationship. And even if there were a relationship, it would be (slightly) negative. We can also think about the total variation in A’s enjoyment as predicted by critics’ scores; it accounts for about 1% of the variation. This suggests that A should never worry about what the critics think about a movie.
What about the Rotten Tomato audience scores though?
The effect size of audience score on F’s enjoyment is larger than the critics’ score, although the relationship slightly weaker.5 A one unit increase in audience score is associated with a 0.41 change in F’s enjoyment.
Once again, N’s enjoyment looks to be less related to outside scores.6 A one unit change in audience score is associated with only a 0.26 increase in her enjoyment scores.
A’s enjoyment scores have a similarly weak relationship with audience scores.7 A one unit change in audience scores is associated with a 0.27 change in enjoyment scores.
So far, neither Rotten Tomato critic scores or audience scores seem to matter for these roommates. So what can we find that does better predict how much they will enjoy a certain movie? One answer may be each other.
The above graph shows the relationship between movies F watches and N watches. They are highly correlated.8 Any movie that N likes, F will generally like too. If N scores a movie an extra point, F will score it an extra half a point. Thinking about predicted variation, whether N says she likes a movie explains about half the variation in F’s scores.9
N and A are even more similar.10 In fact this is the strongest statistical relationship of all. A’s enjoyment of a movie explains 63% of the variation of why N likes a movie. When A bumps a movie rating up one point, N will bump it up 0.89 points; i.e. their scores essentially move in lock step. You can see that clearly in the scatter plot; movies A likes are nearly always the movies N likes. And the observations (the individual movies) are clustered close to the regression line (where they are predicted to fall).
F and A’s scores are also highly correlated, though not at quite the same rate as A and N.11 When F nudges a movie up the scale 1 point, that is associated with a 0.57 increase in A’s score.
What have we learned?
Well, not much at a general level to be honest, although we have learned a lot about how these three roommates should approach movies. They should forget about looking at Rotten Tomatoes. That appears to have little to do with how much they will enjoy a movie (except perhaps for F, but only to the tune of a couple of percentage points). What they should do is pay attention to each other’s enjoyment of a film…which is hard because they obviously like to watch movies together. It may be good to know in the future though when they are no longer roommates. 10 years from now, A can call up N to ask whether Avatar 4 is worth seeing. If N likes it, A is bound to like it.12
Of course, Rotten Tomato Scores aren’t really the out of 10 scores that the roommates gave to movies. They’re the percentage of critics who had positive reviews about the movie. Hence, I’m not really comparing the same thing when I compare the roommates’ scores with Rotten Tomato scores. But I am going to pretend they are the same thing in this post to make things easier.↩︎
Pearson’s R = 0.38, P.value = 0.06, R2 = 0.15. I will report these three values for each of the relationships, which are derived from simple bivariate OLS regressions. I am including P.values for the sake of convention, but it is unclear to me how one would interpret them here. I am not, after all, using these models inferentially–these movies are the entire universe of movies the roommates have watched, not a sample of those. I suppose if I were inferring to movies the roommates might watch in the future, that would make this current set a sample? Then the P.values might be useful.↩︎
Pearson’s R = 0.16, P.value = 0.45, R2 = 0.03↩︎
Pearson’s R = -0.11, P.value = 0.6, R2 = 0.01↩︎
Pearson’s R = 0.29, P.value = 0.15, R2 = 0.01↩︎
Pearson’s R = 0.13, P.value = 0.53, R2 = 0.02↩︎
Pearson’s R = 0.19, P.value = 0.31, R2 = 0.03↩︎
Pearson’s R = 0.7, P.value = <0.01, R2 = 0.49↩︎
What explains the other half is unclear; I have so few variables that I’m measuring. Could be the genre of the movie, the actors in it, the mood F is in when watching the movie, or a million other factors. What we do know is that it is not Rotten Tomato or audience scores (although that is less true of F than for the other two).↩︎
Pearson’s R = 0.89, P.value = <0.01, R2 = 0.63↩︎
Pearson’s R = 0.53, P.value = <0.01, R2 = 0.28↩︎
That being said, I’d be interested to see if the effect sizes remain as strong once they are no longer roommates. The very simple OLS regressions I ran for this little post have an independence assumption built in; that is, each of the observations need to be independent of one another. I am a little skeptical of that assumption being met for this data because they are all rating movies right after they’ve watched them together. Their scores are thus not totally independent–they are no doubt influenced by how the other roommates felt about the movie at hand.↩︎