The Untold Analysis of Refereeing in the first 16 weeks of the 2016/17 season – all matches

Earlier this week Untold presented its review of referee performances in the first 160 weekend games in the Premier League this season.   That is 42% of all the games played this season – an analysis level far higher than one would see in virtually any other statistical and analytical enquiry.

That introductory report links to the week by week reports.   Now the team that produced those reviews look at what they discovered

The analysis of refereeing in the first 160 games of the 2016/17 season

By The Referee Review Team

Sixteen weeks; 160 matches.

These 160 games were officiated by 17 referees who had been employed to cover between 3 and 14 games each.  In total we found that there were a total of 322 wrong Important Decisions in those 160 games and 221 that were correct.

Averaging these out per game typically a referee would have to make 3.4 Important Decisions per game and got 1.4 of them correct and 2 wrong; a success rate of 40%.  Putting it simply referees are more likely to get Important Decisions wrong than right (and by a considerable margin).

Here is a summary table showing the breakdown by referee and type of decision for all 543 decisions (the Right decisions are shown in Green, those we adjudged to be wrong are shown red) :-

Ref	Games	All Decisions		2nd Yellow Cards		Red Cards		Pens		Goals
		Right	Wrong	R	W	R	W	R	W	W
Andre Marriner	12	20	24	0	7	1	7	19	9	1
Anthony Taylor	14	18	26	0	5	3	8	15	10	3
Craig Pawson	11	12	26	1	3	0	11	11	9	3
Graham Scott	3	7	8	0	0	1	4	6	3	1
Jon Moss	12	11	23	0	7	2	7	9	6	3
Kevin Friend	8	8	13	0	3	0	6	8	3	1
Lee Mason	8	8	15	1	2	0	4	7	6	3
Mark Clattenburg	11	24	23	2	7	0	9	22	6	1
Martin Atkinson	11	8	22	0	6	0	10	8	5	1
Michael Oliver	14	13	32	0	7	0	11	13	12	2
Mike Dean	12	19	26	1	5	0	9	18	8	4
Mike Jones	9	16	18	0	6	0	6	16	4	2
Neil Swarbrick	9	14	8	0	1	0	3	14	3	1
Paul Tierney	4	7	4	0	1	0	2	7	1	0
Robert Madley	13	20	32	1	4	0	15	19	12	1
Roger East	5	9	15	0	2	2	3	7	8	2
Stuart Attwell	4	7	7	0	2	0	3	7	2	0
TOTAL	160	221	322	6	68	9	118	206	107	29

If we look at the decisions that the referees did manage to get right in those 16 weeks the numbers tell a very clear story :-

Second Yellow Cards – 6 (one each by Messrs Pawson, Mason, Dean and East and two by Mr Clattenberg)

Red Cards – 9 (one each by Messrs. Marriner, and Scott, 2 by Moss and East and 3 by Mr Taylor)

Penalties – 206 (well distributed across all referees)

The overall accuracy of penalty decisions was 66% correct which is really not at all bad and probably can’t be improved without full real time video refereeing becomes available to the Referees.

However, with disciplinary offences (red and yellow cards), the referees seem to rarely apply the rules to the on-field antics of the players.

We believe that the Referees made 68 wrong calls on second yellow cards (mostly by not issuing them), and 118 wrong red card decisions.  Compare these with the six and nine that we judged they got correct.  Less than 1 in 10 of these decisions was been correct according to the rules of the game.

These are scandalously low figures and can only mean that the referees have been given clear instructions to interpret the laws in an oblique and incorrect manner.  Either that or every single referee in the UK is ignorant of the rules which would be surprising in the extreme.

In case you are thinking that our reports are simply wrong here are a sample of the red and second yellow card decisions that we believe that we believe the referees got wrong in Week 16 – and of course we have video evidence throughout.

No Red Card Against Shane Long (Min 37) – Long swung an elbow in the chest of Martins Indi after he was being man marked. Violent conduct. [Ref – Anthony Taylor] Stoke vs Saints Long Red , Stoke vs Saints Long Red002

No Red Card Against Matt Lowton (Min 29) – Lowton sees Carroll and swung his elbow in the face of Carroll. Use of excessive force. [Ref – Robert Madley] West Ham vs Burnley Lowton Red

No Red Card Against Ben Mee (Min 61) – Deliberate use of the elbow towards the head of an opponent. He came from a distance so knew the other player was there so didn’t have the right to use his elbow coming in like that: red card. [Ref – Robert Madley] West Ham vs Burnley Mee Red

No Red Card Against Curtis Davies (Min 58) – Davies attacked with his studs on the knee and leg of Sissoko, serious foul play. [Ref – Andre Marriner] Spurs vs Hull Davies Red

No Red Card Against Zlatan Ibrahimović (Min 74) – Ibrahimović deliberately swung an elbow back in to the face of Cabaye. Violent conduct. [Ref – Craig Pawson] Palace vs Man United Zlatan Red , Palace vs Man United Zlatan Red002

No Red Card Against James McCarthy (Min 42) – McCarthy lunged in from on Coquelin with a very late sliding tackle, long after Coquelin played the pass. It is a deliberate kick on the achilles from behind with the ball (when he kicks out) already meters away: a red card as kicking a player hard on the achilles is using excessive force without any chance of playing the ball. The foul was made with the other foot than the one he challenged for the ball. The referee failed to give the red card for the second time and instead gave a lesser punishment of a yellow card. [Ref – Mark Clattenburg] Everton vs Arsenal McCarthy Red , Everton vs Arsenal McCarthy Red002

No Second Yellow Card Against Marten De Roon (Min 65) – De Roon committed two reckless challenges but was not booked for either of those. [Ref – Jon Moss] Boro vs Liverpool De Roon Yellow , Boro vs Liverpool De Roon Second Yellow

Here are the two that the referees were correct with for the same week

Correct Red Card Against Marko Arnautovic (Min 23) – Arnautovic raised his foot very high and kicked his studs on the knee of Bouffal.  [Ref – Anthony Taylor] Stoke vs Saints Arnautovic Red

Correct Second Yellow Card Against Phil Jagielka (Min 93) – Jagielka committed one last man foul ( he should have been given a straight red card for this but was instead given the lesser punishment of a yellow card) and one deliberate attack breaking foul for which he was correctly given a second yellow card. [Ref – Mark Clattenburg] Everton vs Arsenal Jagielka (Yellow given as a Lesser Punishment) , Everton vs Arsenal Jagielka Yellow002(Yellow given as a Lesser Punishment) , Everton vs Arsenal Jagielka Second Yellow , Everton vs Arsenal Jagielka Second Yellow002

You can check all of the other weeks yourself if you wish – we have presented all the video evidence in our reports (see below).

Another worrying feature was that we believe that the referees were wrong in 29 goal decisions in these 16 weeks – that’s nearly two per week.  These arise from three causes

1. Incorrectly awarded penalty decisions
2. Not called offside decisions
3. Goals scored directly from wrongly awarded free kicks.

The introduction of video refereeing should ensure that these errors are virtually eliminated.

But it all evens out in the end I hear you say

Unfortunately the simple answer to this simple statement is that it doesn’t.  Some teams seem to benefit each week from incorrect decisions, others suffer, and it tends to be the same teams each week.  Sunderland (6), Burnley (7), Leicester (8) and Man United (9) have far fewer wrong decisions against them than man City, Liverpool, Crystal Palace (all 21), West Ham and Bournemouth (25) and Arsenal who have 31.

Looking at the teams who benefit from wrong decisions, Arsenal are bottom of the list with 4, Liverpool and Bournemouth (9) and West Ham (12).  United had benefited most with 31, followed by Spurs (25), Watford (23) and Leicester (21).

All of these decisions are potential game changers involving sending offs, penalties or goals.  Arsenal have a net result of 27 such decisions against them in the 16 weeks, Man United have 22 in their favour and Tottenham have 10 in their favour.

In each weekly report we looked at the possible impact of the wrong decisions on the outcome of the matches.  In the case of an incorrect goal we simply deducted that goal from the final score.  In the case of a non awarded penalty we assumed that the resulting kick would have been a goal (80% probability or thereabouts).

For a sending off we looked at the time remaining and normally judged that with 30 minutes or so to play the team with the numerical advantage would have scored an additional goal.  With two players sent off the time to count an additional goal was reduced.  These rules are purely subjective it is true but we were consistent in our approach.

In the 16 weeks Bournemouth were the team hardest hit in terms of points.  We judged that they should have a total of 10 more than they actually had (two in each of weeks 3, 7 and 9, one in week 10 and three in week 15); Arsenal should have had nine more, Middlesbrough should have six more, Palace five and City and Swansea four more.

Of the teams that seemed to benefit from decisions in their favour,  United should have 11 points fewer than their total (2 each in weeks 3 & 15, 1 in each of week 7, 8, 10 and 12 and 3 in week 16), Watford had benefited to the tune of seven points and Tottenham by four points.  The other 11 teams were within three points of their actual points total.  For Sunderland and Hull there were no points adjustments at all.

Of the teams currently in the fight to avoid relegation, Middlesbrough, Palace, Watford and Bournemouth seem to have been affected by wrong refereeing decisions.  With the financial consequences of relegation being so severe it opens the question as to how long it is going to be before one such affected team decides to go to court for recompense.

Are some referees better than others?

On a week by week basis yes – we have named both the best and worst every week.

Over the 16 weeks as a whole the answer, statistically, is no.  The best have an average of just less than one wrong decision per game, the worst has an average of three.  The overall average is 2 and the standard deviation ( the expected variation from the average) is 1 so, to all intents and purposes all of the referees fit within the range of performance that we should expect.

Are our FIFA Accredited Referees better than those without this accolade?

Of our referees the following eight have FIFA Accreditation.  Here is a table giving their wrong decisions in the first 16 weeks :-

	Games	Wrong Decisions						Correct  decisions
		2nd yellow	red	pen	goal	total	avg/game
								Total	Overall % correct
Martin Atkinson	11	6	10	5	1	22	2.0	8	27
Mark Clattenburg	11	7	9	6	1	23	2.09	24	51
Robert Madley	13	4	15	12	1	32	2.46	20	38
Andre Marriner	12	7	7	9	1	24	2.0	20	45
Michael Oliver	14	7	11	12	2	32	2.28	13	40
Craig Pawson	11	3	11	9	3	26	2.36	12	32
Lee Probert	0
Anthony Taylor	14	5	8	10	3	26	1.85	18	41

Looking firstly at the wrong decisions in isolation – here we can clearly see that all of the seven FIFA accredited referees are no different to the group average.

If we take into account their correct decisions as well only Mark Clattenburg has more right than wrong (24 to 23) with an overall score of 51%, Martin Atkinson’s numbers are abysmal his correct percentage is a measly 27%.

I think we can legitimately ask why these referees have been awarded their FIFA badges – it clearly doesn’t seem to be due to their on pitch ability.

We will present a more detailed look into the numbers highlighting some of the quirks and oddities that we have found in the next article in this series.

 Links to all the reports with the video evidence.