INDIVIDUAL PERFORMANCES OF THE REFS AND SOME BIAS FROM MY PART… OR NOT?
By Walter Broeckx
In this article I will try to find an answer to two questions. The one is linked with my bias and the other is how did each ref do and was his score good enough if he would be a ref in another country.
So the first table I show is the table in which I range the refs according to their results in the games we had.
| Pos | Ref | Total | Games | Score |
| 1 | Clattenburg | 305 | 4 | 76,25 |
| 2 | Atwell | 149 | 2 | 74,50 |
| 3 | Mason | 218 | 3 | 72,67 |
| 4 | Atkinson | 436 | 6 | 72,67 |
| 5 | Mariner | 203 | 3 | 67,67 |
| 6 | Friend | 134 | 2 | 67,00 |
| 7 | Oliver | 133 | 2 | 66,50 |
| 8 | Halsey | 194 | 3 | 64,67 |
| 9 | Jones | 252 | 4 | 63,00 |
| 10 | Dowd | 291 | 5 | 58,20 |
| 11 | Taylor | 55 | 1 | 55,00 |
| 12 | Probert | 104 | 2 | 52,00 |
| 13 | Dean | 207 | 4 | 51,75 |
| 14 | Foy | 255 | 5 | 51,00 |
| 15 | Walton | 99 | 2 | 49,50 |
| 16 | Webb | 86 | 2 | 43,00 |
So this is the score of each ref and I have put them in order of their average score in all the games they did Arsenal this season. Going from the best at the top to the worst at the bottom.
So the best ref in my eyes was Clattenburg. He can come to collect his prize at Untold Towers. Don’t know if Tony has something for him in his closet. But be prepared if he comes to claim his prize, Tony. (A bottle of Crabbies maybe? – Tony)
As you all know by now the average score a ref must get in Belgium is 70%. And if we take this in to these numbers you can see that only 4 refs qualify for next season. Clattenburg, Atwell, Mason and Atkinson.
Take notice of who finishes with the lowest score… Howard Webb. So it looks that to be successful in England and in the Fifa rankings you must make sure you don’t follow the Fifa rules and guidelines. I don’t know but this should ring a bell somehow I would say.
So you can look for your favourite ref in the table and see how much they got on average during this season. And I would really ask you to look for ref Foy. And look at how low his position is in this table.
Now let us take a look at the fact if these numbers are based on bias from my part. If I would be very biased in my ref reviews you should see a pattern. You should see that the more games we have won under a ref the higher his average score should be. So I tried to look at this and I made another table. A table based on how many games we have won under a certain ref. And in the next table I ranged the refs based on the more games we won under the ref the higher his place in the table.
| Pos gained points | Pos average score | Ref | Total | Games | Score | W | D | L |
| 1 | 14 | Foy | 255 | 5 | 51,00 | 4 | 0 | 1 |
| 2 | 1 | Clattenburg | 305 | 4 | 76,25 | 4 | 0 | 0 |
| 3 | 9 | Jones | 252 | 4 | 63,00 | 3 | 1 | 0 |
| 4 | 3 | Mason | 218 | 3 | 72,67 | 3 | 0 | 0 |
| 5 | 8 | Halsey | 194 | 3 | 64,67 | 2 | 0 | 1 |
| 6 | 5 | Mariner | 203 | 3 | 67,67 | 2 | 1 | 0 |
| 7 | 15 | Walton | 99 | 2 | 49,50 | 2 | 0 | 0 |
| 8 | 2 | Atwell | 149 | 2 | 74,50 | 1 | 1 | 0 |
| 9 | 4 | Atkinson | 436 | 6 | 72,67 | 1 | 4 | 1 |
| 10 | 6 | Friend | 134 | 2 | 67,00 | 1 | 1 | 0 |
| 11 | 12 | Probert | 104 | 2 | 52,00 | 1 | 1 | 0 |
| 12 | 13 | Dean | 207 | 4 | 51,75 | 1 | 0 | 3 |
| 13 | 16 | Webb | 86 | 2 | 43,00 | 1 | 0 | 1 |
| 14 | 7 | Oliver | 133 | 2 | 66,50 | 0 | 0 | 2 |
| 15 | 10 | Dowd | 291 | 5 | 58,20 | 0 | 4 | 1 |
| 16 | 11 | Taylor | 55 | 1 | 55,00 | 0 | 1 | 0 |
And now you see the refs that has brought us most points first and so on down to the refs that have brought us least points. The first column is the position based on the average points gained. And right next to it you can see the position he got when I did the reviews.
And then to my and hopefully your surprise we got the most points this season in the games under ref Foy. And yet he finished 14th in the average score table. So if I were to be biased and just give high points when we win then how could Foy end up in 14th place on the average points per game?
Another example is ref Jones. Only 9th in the average score table but he was the third best ref when it came to winning games. A ref that has made me furious this season was ref Mason. You know the ref in our home game against Everton. The one that Cesc asked something about who actually paid him. Well despite me being furious I did give him the points he should have got and he ended up rather high in the table.
This table clearly shows that I did give my points based on what I have seen from the ref in the games and not based on the final score. And I must admit that the nearer the season came to an end the more worried I got about this table. Because if the position of the refs would have been the same I would have to admit that I did show a big bias. But the way the two tables are mixed up I can say now that I really managed to do what I promised myself at the start of the season and that was to look at the ref and not at the score.
And yes there still will be some kind of bias in my reviewing of the ref. A biased known but I can prove now that my bias didn’t get in the way of assessing the refs in a way that can be called unbiased and not done with the intent of : “we won this game so I will give him a high score” or “we lost the game so I will blame the ref and give him a low score”.
Look out for the next article I would say. We come next to the issue of whether it all evened out at the end of the season.
Half the penalties in Arsenal games were wrongly given this season!
Untold Arsenal and Arsenal History on Twitter @UntoldArsenal
Untold Arsenal on Facebook here
History of Arsenal: when Fulham stole the Arsenal managers.
Making the Arsenal – the book of Arsenal death and rebirth
What a truly staggering amount of work you have put into this, this season. Remarkable.
Webb needs retiring. Or shooting.
I take my hat off to you Mr Broeckx. The time you must have put in to highlight the standard of officiating in the EPL must have been immense. The only misgiving i have is that this is based on four officials per match, three of which make decisions during play, and your hanging all decisions on the ref which in itself is unfair. I want you to succeed in highlighting this issue so I say “go after them all”, Select Group Referees & Select Group Assistants.
@ Walter
A great piece of work, publish and be dammed, well done. It is not a matter of opinion, it is all evidenced and in the archives.
Your reviews are based on application of the laws, I know I keep returning to that fact, but it is what referees are employed to do, APPLY THE LAWS!!!! The results are not surprising, I have long told my mates that Clattenburg is the best ref in the league, your reviews confirm this.
Webb may have got the World Cup Final, but he is the only ref I have ever seen booed when he picked up his medal in the WCF. Why? Because he failed to apply the laws, and as a consequence the game was ruined.
Let UA keep turning the screw, any Journos reading??
@adam – I don’t think its unfair to hang it all on the ref. As far as I’m concerned, he makes the final decision so must take responsibility and ultimately the blame for any mistakes. The higher a monkey climbs, the more you can see of his arse.
Appreciate the point, but I don’t actually agree with the conclusion here. It looks to me like there is indeed a correlation between our results and your rating. The top half scores 55 points and the bottom half 81, for starters. That’s a pretty big difference.
On the flipside this alone does not mean you are biased. Indeed Arsenal is one of the stronger teams in the PL so we ought to win more games, and it is often the ref intervening and screwing us over that takes points away, both for us and for the ref. So you might actually EXPECT to see some correlation between points scored for Arsenal and points scored for a “clean” refereeing performance, i.e. when the ref doesn’t get in the way too much the better team wins, and more often than not, Arsenal is the better team.
If there was really no correlation between Arsenal points and ref points that might point to you actually being harsh towards Arsenal!
So what I am saying is I appreciate the self-investigation and I for one truly believe you try to be as unbiased as you can, whilst acknowledging in great depth that it is not possible to be wholly unbiased; but this study proves nothing at all (unlike so many of your others). Sorry anyway, don’t mean to be a doom monger, appreciate the effort as always
Cheers Steve I can not get the picture of a monkeys arse out of my poor noodle now.
@Phoenix Gunner, I disagree, It does show that this requires further investigation. When a team of officials decision making shows a high percentage of wrong calls or even omissions then the system they employ has to be challenged. Mr Broeckx efforts are a work in progress and my hope is that he finds the help required to take this further.
Walter excellent work as for Pheonix some people will never be satisfied. I do think it was great to take a deeper look at the refereeing in the EPL it stinks.
I cant wait to see what happens next season particularly if you can find some other Ref’s to get involved and we can see an even clearer picture.
Even if we just had enough Refs to monitor the games involving the top 10 teams in this years EPL it would heap pressure on someone to improve the overall situation.
How can Oliver be 14th in the second list if we only had two losses under him? Surely he should be last with zero points gained.
Also, instead of ranking according to the total number of points gained under a certain ref, I think it is better to look at the average points per match gained under a certain ref. Because Arsenal has not played the same amount of matches under each ref, you cannot use absolute numbers for a comparison.
Great work Walter. To take up Kentetsu’s point-
based on Arsenal’s points per game, the order changes quite significantly: (position on average score)
1= Clattenberg (1), Mason (3), Walton (15)
4 Foy (15)
5 Marriner (14)
6 Jones (5)
7= Halsey(8), Atwell (2), Friend (6), Probert (12)
11 Webb(16)
12 Atkinson(4)
13 Taylor(11)
14 Dowd (10)
15 Dean (13)
16 Oliver(7)
I would not be sorry to hear that Dean retires/gets sacked/dropped from the list. He’s been a pain for far too many seasons. Being a scallie, he probably hasn’t forgiven us for the “Up for grabs now” result! Plus of course, he did the Carling Cup!
Great work Walter. I think with the data you have, we can make quite various analysis. What would be interesting to know is what were the score of the refs corresponding to our league table. One of our assumption is that the refs will try to make us as far as possible from top. Thus perhaps we can plot the refs point in the graph vs the time or our position in the table. Perhaps we can see something interesting here.
Walter- you’ll need to find a ref who’s neither an arsenal fan n neither your friend and then you’ll have to verify your individual ref reviews with him. Only then would it be called unbiased. Untill then, its biased. Also, as far as i remember, both you and Tony have expressed here on this site itself that you guys are biased bcoz obviously, you’re arsenal fans.
Walter: To help the reader (me, say) both remember and think better about the implications as he/she reads the table, I would like to see another column which just lists the opponents name. Then we could see and think about – Ok, this ref officiates when we play this team, or when there’s a top 4-6 team, etc. I mean there might now, or in the future, be that type of correlation that surfaces and, if so, it could reveal an assignment pattern – which may or may not correlate with ref ranking. I would love to see that at a glance and it would provide a way to think about whether something might be bent. That is, I wouldn’t retreat from that possibility in the presentation (or lack of presentation) of simply who we played. It may or may not fork lightning, but it is definitely part of the record. Of course I could look this up, as anyone could, but it’s better to just provide it (which would take comparatively little time) and give UA readers something juicy at least to consider for going forward. Cheers for the ongoing contribution in this brilliant case study that need not make a fetish of an unobtainable perfect objectivity. You have announced you are an Arsenal fan, done yeoman work and full disclosure of that, and so, fans can think for themselves as they look at this. That is honest: people can pretend there’s a reachable standard of 100% value-neutrality, as the ungenerous of spirit Dark Prince (master of the roundabout); or, as in half of the social sciences, people can welcome a value-influenced analysis that openly and up-front states that “I am a fan of team x, I have done 1, 2, 3 to minimize bias, and so, here, dear reader are the results of what I have seen. This is utterly legitimate and how good qualitative analysis which does not reject (in fact utilizes) quantitative tools ALWAYS proceeds in the social sciences. And lastly, having multiple referees help as we move forward, as Red Gooner suggests, could in itself (one hopes!) have an impact on some referees and bend the arc of calls/non-calls toward that of fair play and a more level pitch. This is positive and salutary and, obviously, a major and ongoing great leap forward. Small minded logicians do nothing to tarnish the real-world contribution that you continue to make with this work. Avante!
Great article Walter.
I note that over the season you took great care to be totally subjective and accurate in each review. The reviews and overall analysis in combination certainally show a bias in the ref decisions. The pattern illustrated by your analysis seems to confirm what we all suspect – that Arsenal have been the victims of an orchestrated campaign by certain refs rather than simply having suffered from an unfortunate series of bad decisions. What we can’t yet identify with certainty is the person or persons who have been pulling the strings.
I don’t think it is correct for Dark Prince to suggest that your reviews are biased just because you are an Arsenal supporter – to prove a review to be biased it would first have to be shown to be in error and despite a large readership I have not noted too many disagreements with your comments. However, the more support you have for your reviews, especially from professional and unaligned sources, the stronger your position.
Keep up the good work!
@Walter: So, assigment-wise, for example, Webb may only have been assigned to referee two games (which were split 1-1 or 2 draws), but knowing which games he was assigned to and its outcome, will help you/readers think about the context – the timing of the assignment to the match. It reinstates the possibility of skewed ref assignments, rather than keep it in the shadows. No harm (at least) and possible anomalies (at best) may or may not surface when we all get a chance to consider the games that each referee officiated. There might be more than chance at work (ahem) when Arsenal (or others) play a certain side (fill in the blanks) at a certain moment in the season. Now, methinks, anyone who wants a fair outcome could withstand having that also included in a second version of this, your last table. The column could be labeled “Opponent,” or better (to my lights), “Opponent/Assignment.” Please consider.
@btj gooner: do you mean “totally subjective and accurate” or “NOT totally subjective and accurate” or what? please clarify what you’ve meant…
@bob. I wrote “totally subjective and accurate”.
My take on Walter’s reviews is that they are very accurate and further he has used his experience, knowledge and integrity to produce high quality and fair reports uninfluenced by by any other considerations, i.e. in my view unbiased.
Meanwhile back at the Ranch:
Sorry for this being off-center a bit, but surely in-spirit: As we were sleeping, this from today’s Guardian as the FIFA (w)hordes descend on Wembley: “The claims against Blatter mean that 10 of Fifa’s 24-man executive committee have now either been found guilty or accused of corruption in the last 12 months.” Now it’s not “purely objective” to suggest that something might well be rotten in the state of Denmark, now, is it? Surely it’s self-selecting of me to call this to our mutual attention as worthy of ongoing notice – I mean just in case “coincidence” and “human nature” and “errors in judgment” and anything but something bent or anomalous or beyond statistical probability is asserted re. Arsenal matches. That is, there seems (ahem) to be a culture of corruption that is rapidly surfacing on the public radar and fans would do well to pay close attention.
@btjgooner: Ok, great. One can be avowedly subjective as to one’s loyalties and do accurate work, right? and being open about one’s leanings does not in itself – unlike what several assert in these spaces – invalidate the validity of the outcome, yes? Is this a fair reading of where you stand in your posting?
@bob. Correct!
@Walter
Nice presentation/article – I would echo the comments above though, that the referee ordering should be based on Arsenal points per game – the ‘games won’ ordering is obviously affected by number of games refereed.
Not that I think it makes too much difference to the conclusions, just think it would be more accurate
@bjtgooner, Walter: Are we the only ones paying attention today, or did everyone go to the FIFA meeting to vote on the new president? Hmmm. The silence is deafening.
Bob,
I’m preparing myself for the trip to the Emirates tomorrow. 🙂
But in Dutch there is an idiom that is saying : the rotten the head of the fish is, the more rotten the rest of the body.
Well I don’t know if the message comes over but the more possible corruption at the top of an organisation, the higher the possibility of the people at the lower echelons (the countries FA) are also a bit corrupt
@Walter: yes! we have “a fish rots from the head down”. Alas, it’s too true to be good. (Would you put a bet on whether Herr Blatter gets re-elected? Then again his opponent is also under serious investigation. Some people would call that election a win-win prospect.) Cheers! (And enjoy the Emirates.)
Great work Walter, just need this taken to the next step really. Highlight of the year was being featured in the London Newspaper. Fantastic promotion for the great work you have done.
One thing i would like to see for the next season, is the addition of one of your Belgium ref friends. And get him to score the ref with you all season, and then compare the results. It would be interesting to see what two refs from Belgium who require a 75% statistic think. Also it would remove the bias element some what. The ref at the end of the season can give a little summary as to why he/she came up with the results and also what he/she thinks of the standard of English referring
@Aaron – which newspaper? is there a link? I’ve had my distractions, so would be grateful for that reference, please.
To RedGunner/Adam
You’re both completely misunderstanding. I think the ref reviews are wonderful and have stated my support for expansion of the project many times, as you would know if you read them all rather than jumping to conclusions.
As someone who has studied stats to a master’s level, I am telling you that this particular study, i.e. the effect of Walter’s own bias, is not approached right. That’s all. I’m also trying to explain why the drawn conclusion is wrong and also why the intent is wrong (i.e. the assumption that Walter’s bias would mean refs which martial Arsenal victories would get more points from him than Arsenal losses – I am arguing that provided Arsenal are a ‘strong’ team, this effect is a natural phenomenon and does not NECESSARILY have anything to do with his bias).
Feel free to explain to me why I’m wrong rather than telling me ‘some people are never happy’. If the study is inaccurate then it’s inaccurate, I can’t do much about that other than explain that to the person investing their time and energy in it, in order that they may learn.
i gotta say, the real villain for me was Dean and Dowd…they just managed to develop a weird dislike for Arsenal, specially song…i remember all the games we played under Dowd and he was abysmal…hate these two
why am I not surprise to see Dowd ranking low in both tables,it seems Dowd is on a mission to make Arsenal a losing side.