Telegraph follows Untold in ranking referees: how does their analysis match up with ours?

By Tony Attwood

The Telegraph has published an article, “Every Premier League referee ranked and rated by Keith Hackett”, so what I have done is taken the refs rankings by the Untold team and compared with the Telegraph rankings.

We’ve ranked refs in two ways – their major wrong decisions and their correct major decisions.  The Telegraph gives refs a score and a very brief commentary although no real explanation.

Here’s a brief summary of their views before the comparison with ours – but before we go on I do think we should realise just what is happening here.  For the first time, as far as I know, a national UK paper has done a review of referees.  And not just that, the review found four of them lacking enough to warrant their removal.

As you will see I think their ranking system is gibberish, but this should not deter us from seeing progress in my view, for I really can’t believe that we could have got to this had Untold not be here.  I think we need to give the fullest thanks to the team not just for all the work they have done, but for the impact of the ceaseless and tireless work of Walter across the years in bringing the issue of referees to the attention of everyone.   Without Walter’s early work we would not have got to the point of a newspaper analysing referees, even though their analysis is bizarre.  This is, I believe, stage one of a change.

Of course it could all fall away if none of the other papers follow up on this, and if the PGMO warns the Telegraph not to touch the subject again, and the Telegraph gives in.   But I suspect the door has been opened and others will follow.

Remember when we started out the PGMO were telling us the 98% of referee decisions were correct and the media were quoting this without any word of doubt.  Now we have something different.

Here’s the Telegraph’s table of marks for the refs…

  • Clattenburg – 9 out of 10
  • Atkinson – 7 out of 10
  • Oliver 7 out of 10
  • Dean: 7 out of 10
  • Marriner: 6 out of 10
  • Taylor: 6 out of 10
  • Madley: 6 out of 10
  • Swarbrick: 5 out of 10
  • Friend: 5 out of 10
  • Pawson: 4 out of 10
  • Jones: no score – needs to be removed from PGMO
  • Moss: no score – needs to be removed from PGMO
  • Mason: no score – needs to be removed from PGMO
  • East: no score – needs to be removed from PGMO

Now a comparison with our tables.   First…

Major Wrong Decisions Made by Referees

I have removed referees not covered in the Telegraph report

Referee Matches 2nd Yellow Red Penalties Goals Total Avg Telegraph
Roger East 5 2 3 8 2 15 3.0 Remove
Mike Dean 10 5 8 8 4 25 2.5 7
Robert Madley 10 4 11 9 1 25 2.5 6
Craig Pawson 8 3 8 8 0 19 2.4 4
Lee Mason 7 2 3 6 3 14 2.0 Remove
Mark Clattenburg 8 7 3 5 1 16 2.0 9
Michael Oliver 11 5 8 7 2 22 2.0 7
Andre Marriner 10 7 5 6 1 19 1.9 6
Jon Moss 10 6 6 5 2 19 1.9 Remove
Anthony Taylor 11 4 5 8 3 20 1.8 6
Martin Atkinson 10 5 9 3 1 18 1.8 7
Kevin Friend 7 3 6 3 0 12 1.7 5
Mike Jones 8 4 5 3 1 13 1.6 Remove
Neil Swarbrick 7 1 3 3 1 8 1.1 5

In our chart the worst referees are at the top (they make the most mistakes), and the best (in terms of the least number of disasters and cock-ups, at the bottom.   The Telegraph and ourselves agree that East should be removed, but then give 7 out of 10 for Dean.

Now let us move on.

Correct Major Decisions Made by Referees

Again for simplicity I have only included the referees mentioned in the Telegraph report.

Referee Matches 2nd Yellow Red Cards Penalties Total Avg Telegraph
Andre Marriner 10 0 1 15 16 1.6 6
Anthony Taylor 11 0 0 9 9 0.8 6
Craig Pawson 8 0 0 10 10 1.2 4
Jon Moss 10 0 2 6+1* 9 0.9 Remove
Kevin Friend 7 0 0 8 8 1.1 5
Lee Mason 7 1 0 5+1* 7 1.0 Remove
Mark Clattenburg 8 1 0 19+1* 21 2.6 9
Martin Atkinson 10 0 0 7 7 0.7 7
Michael Oliver 11 0 0 12 12 1.1 7
Mike Dean 10 1 0 16 17 1.7 7
Mike Jones 8 0 0 10 10 1.2 Remove
Neil Swarbrick 7 0 0 11 11 1.6 5
Robert Madley 10 1 0 14 15 1.5 6
Roger East 5 0 2 7 9 1.8 Remove


There is a link here – Clattenburg gets the most decisions right and the Telegraph give him their highest score – but then they want to remove East who has the second highest average number of right scores.

Now pulling all this together.   Our figures for the referees in terms of major wrong decisions, major right decisions, and the Telegraph score.

The average major wrong decisions are ranked with the ref with the LEAST major wrong decisions on average per game being ranked 1 and the ref with the MOST major wrong decisions being ranked 14.

The average major correct decisions are ranked with the ref with the MOST major correct decisions on average per game being ranged 1 and the ref with the LEAST major correct decisions being ranked 14.

So the perfect ref should be ranked 1 for average major wrong decisions and 1 for average major correct decisions.  That is the fewest wrong and the most right decisions.

Referee Matches Avg major wrong decisions Rank Avg major correct decisions Rank Combined Telegraph
(Lower the better) (Lower the better) (Lower the better) (Higher the better)
East 5 3.0 14 1.8 2 16 Remove
Dean 10 2.5 13 1,7 3 19 7
Madley 10 2.5 12 1.5 6 18 6
Pawson 8 2.4 11 1.2 7 18 4
Mason 7 2.0 10 1.0 11 21 Remove
Clattenburg 8 2.0 9 2.6 1 10
Oliver 11 2.0 8 1.1 9 17
Marriner 10 1.9 7 1.6 4 11 6
Moss 10 1.9 6 0.9 12 18 Remove
Taylor 11 1.8 5 0.8 13 18 6
Atkinson 10 1.8 4 0.7 14 18 7
Friend 7 1.7 3 1.1 9 12 5
Jones 8 1.6 2 1.2 7 9 Remove
Swarbrick 7 1.1 1 1.6 4 5 5

We don’t have a ref with a score of 1 in each column (very few incorrects, but lots of corrects).  But we do have two refs scoring under 10 – Swarbrick and Jones.   But for the Telegraph Jones should be removed on the grounds that he is “too soft and inconsistent to be a referee at this level”.  For Swarbrick the comment is that he is underrated and doesn’t get enough games.   Which seems odd for a man only given 5/10.

So our top referees are

Mason, Dean, Madley, Pawson, Moss and Taylor

For the Telegraph Mason and Moss should be removed, Dean is fairly highly rated, Madley average, Pawson is poor, and Taylor average.  There is no linkage.

One obvious reason for this is that the Telegraph gives, and probably has, no statistics.  Everything is reduced to a three or four line comment, whereas we have given detailed statistics and evidence.

This makes me conclude that the whole piece in the Telegraph is done simply to jump on the fact that our constant referee analysis has found interest among readers.  They want to get in on the act, and so, rather as I said to my fellow writers “would you like to give any awards for 2016?” (and thus far they have by and large ignored my plea) the Telegraph said, “let’s do a quickie on refs – and lets have some that ought to be removed.”

Now I think they put this last bit in, because a huge amount of the publicity we got for our original work criticising the PGMO came from their insane “98% correct” gibberish.   So the Telegraph wanted to stay away from numbers.

Also we should note that their star man Clattenburg does get a lot of decisions right – so I think they have only focused on this, rather than on the horror show of the decisions that are wrong.

The PGMO won’t like the Telegraph report because it suggests a number of refs should be removed, but I suspect they will be relieved that nothing much that the Telegraph says or concludes has any relevance to the truth.

Untold Arsenal

Wenger ponders whether Yaya Sanogo will ever really be good enough for Arsenal. 

What’s up with Ozil? Actually nothing much. In fact he’s still at the top of his game.

Arsene Wenger Still Believes Arsenal Has What It Takes Despite Being Financially Weaker

The making of a referee: could former professional footballers become refs (and other one-shot solutions)

If Alexis and Ozil depart, Arsenal have plans to replace both of them with one player!

Nominations for the best of, worst of, what we need more of and what we need less of in 2016

Football in England is at the crossroads and its future hangs in the balance

The psychology of winning.

England’s refereeing in the dark ages. In Europe they are already talking about the problems AFTER the video box



7 Replies to “Telegraph follows Untold in ranking referees: how does their analysis match up with ours?”

  1. Somebody came onto BBC 5 Live on that phone in they do on match days (606 is it?) and really laid into the PL referees. The guy was very articulate and he made the point that they were very poor apart from, in his opinion, Michael Oliver. One of the studio people mentioned Dean & he said he was by far the worst. I got the impression he was a Gooner by the way he spoke about the state of refereeing. Unfortunately he was cut short by a live update on a match that was going on and they didn’t return to him and the discussion ended there. The fact that it was allowed at all is progress. Maybe something has started at last.

  2. I think the Telegraph newspaper organization should be acknowledged for their coming out against the unyielding establish order in professional football referring body the Pgmol to highlight their self imposed shortcomings through their match officials in PL matches referring job.

    For once, the Telegraph have come out against the abysmal referring indecency seen in the PL matches for some number of years now by the Pgmol appointed match officials referring in the Premier League matches.

    This Telegraph action suddenly taken which surprises many is looking to buttress the Arsenal Untold efforts in their outcry to have a stop put to the deliberate unfairness being employed weeks-in weeks-out in PL match officiatings by the Pgmol’s match officials which the Arsenal Untold have been highlighting with no attention paid to it by the official football authorities of the Sports ministry and the FA to arrest the anomaly.

    Nevertheless, whatever be the level of the grand plot the Pgmol have put out against Arsenal in an attempt to stop them win the PL title this season will come to nothing as Arsenal will be more than equal to their grand plot and render it into utter failure.

    Arsenal will win the PL title and even the FA Cup and the CL as well this season to have a treble winning season to the amazement of their title rivals and doubters as they unfailingly accomplished their aims and objectives.

  3. Clattenburg has two of the three lowest scores in Arsenal games so far this season and has 5 wrong Important Decisions (all against us). Atkinson is little better with his first game getting 60.7 overall and 6 wrong Important Decisions all against us. Pawson has been the worst we have seen so far this year with an overall score of 38.2%. The best referees so far have been Roger East 89%, Mike Dean – yes that Mike Dean – with 82.9% and Michael Oliver with 82.3 and 81.4%. These referees in their 4 games (up to week 14) have 3 wrong decisions only and one of those was in our favour. Michael Oliver and Craig Pawson are the only two referees so far to have given a broadly balanced game in terms of bias both being within my 60/40 rule of thumb.

  4. Although I do not always agree with his comments, I think it is a positive development that Keith Hackett has a regular column in the Telegraph. He frequently makes critical comments about the PGMOL.

    I also agree that this is a tribute to the work that Walter and the team have done for several years and it is testimony to the truth that if something is correct and you persist in saying it, there is a chance that it will get through. If you don’t say it, or give up easily, there is no chance.

    So thank you Tony and Walter for Untold and its work.

  5. There does seem to be more scrutiny of refs by refs (rather than by explayer muppets). Some credit due I suppose to the Telegraph although Hackett is of course getting publicity for his website which has been doing ref game reviews since 2014 at

    Their reviews are ok but don’t go into the same level of detail as the UA team and do not have video clips. They tend also to be a little congratulatory and favour the English way of game management.

    I often see whether they have a different take on the the big decisions than the UA team. They sometimes do agree but there are that many howlers it’s difficult not to.

    We can see how even a video ref would not please everyone but would be a big step forwards.

  6. Just went through referees performance in ARSENAL games so far, Roger east seems to have fared well, very well.

    Ignoring the bias (which is obvious) general performance in ARSENAL games would look like the table below.

    Taking into consideration the correct decisions (weighted), wrong decisions(weighted) and % of correct decisions(weighted). The ranking would be.

    1 Roger East
    2 Kevin Friend
    3 Michael Oliver
    4 Mike Dean
    5 Mike Jones
    6 Jon Moss
    7 Martin Atkinson
    8 Robert Madley
    9 Andre Marriner
    10 Craig Pawson
    11 Mark Clattenburg

    Taking number of decisions taken during the games into consideration:

    1 Roger East
    2 Kevin Friend
    3 Martin Atkinson
    4 Mike Jones
    5 Michael Oliver
    6 Mike Dean
    7 Jon Moss
    8 Robert Madley
    9 Mark Clattenburg
    10 Andre Marriner
    11 Craig Pawson

    # Statistics experts please correct me.

Comments are closed.