By Walter Broeckx
When is a referee error not a referee error? Now we know – it is when PGMO (the shadowy body that controls English Premier League refereeing) doesn’t see it.
As Mr. Riley head of the PGMO has come out in the open with his numbers that I have talked about in another article I think it is time to look further in the numbers and compare the findings of Untold with the findings of PGMO.
As I said in my previous article Riley skipped the question of how many of those decisions or non decisions were correct. Of course the non-decisions are not really talked about in the numbers Mike Riley released. And this is something that can have an even bigger influence than a given decision.
A not-given penalty can have a very big influence on a game. But does it get any reflection in the way the PGMO and the FA and the PL look at the refs?
What better way than to look at an official publication from the PL where not only the instructions are given to match delegates and referees but also has a report on the numbers of last season. And thanks to a beneficial wind this report got blown on to my desk.
And what a great coincidence that Untold Arsenal did a big referee review of the same season.
So for the first time we can compare our numbers with the numbers of the PGMO. Now one thing is that we don’t know how they came to their numbers. We know we reviewed each game we did completely and we did more than 40% of the games in the season 2011-2012. The PGMO did all 380 games.
So let us see what they say about the performances of the referees in the last few seasons.
In the last column you can find the number we gave based on our reviews of 155 games in the PL.
Top class performances according to the PGMO
| 07/08 | 08/09 | 09/10 | 10/11 | 11/12 | Untold |
| 10% | 12% | 11% | 12% | 10% | 4% |
Good performance in the PL according to the PGMO
| 07/08 | 08/09 | 09/10 | 10/11 | 11/12 | Untold |
| 80% | 80% | 79% | 80% | 81% | 83 % |
“Key errors” in a performance in a Premier League game (known as “bad performances” in Untold terminology)
| 07/08 | 08/09 | 09/10 | 10/11 | 11/12 | Untold |
| 10% | 8% | 10% | 8% | 9% | 13% |
Now I don’t know what the criteria are for the PGMO but for us, the top class performances are performances with a score of +90%.
The good performances are based on a score of between 60% and 89,9%. And the key errors performances are based on a score of below 60%. I do think that with the description “key errors in performances” the PGMO is trying to avoid the word: bad performances. Untold however likes to call it how it is.
Now I must say that I have hesitated to call the performances between 60% and 70% as good performances. Because in my book I don’t think only making 60% correct decisions as a ref is not really that good.
If we take a stricter application of our numbers we get this score:
| PGMO | Untold | |
| Excellent | 10% | 4% |
| Good | 81% | 50% |
| Key errors / Bad | 9% | 46% |
Now of course as we don’t know at what point the PGMO thinks a good performance becomes a bad performance (or an excellent of course) we don’t really know how we can compare the numbers. A little bit of informing the public wouldn’t be bad I think. But then again we are talking about the PGMO.
Now let us move on to the errors in the games. And in the report we find these numbers.
According to the PGMO and the PL there have been 36 key errors in all the games in the PL in the season 2011-2012. Based on their reviews of 380 games. The numbers from Untold are only based on 155 games.
Also notice that we didn’t make the division the PGMO did so the Untold numbers are total numbers. And in a way it doesn’t matter if a goal decision is wrong on how it was wrong. Wrong is wrong and right is right. That is what matters.
Anyhow the key errors can be divided like this:
| PGMO | UNTOLD | |
| Missed red card offence | 4 | |
| Incorrect send off | 4 | 77 |
| Incorrect penalty | 11 | |
| Penalty missed | 19 | 91 |
| Goal incorrectly disallowed for foul | 1 | |
| Goal given but foul on the keeper | 1 | 40 |
| Ball over the line but goal not given | 2 | |
| Incorrect DOGSO | 1 | |
| Total | 36 | 208 |
So all the people from the PGMO could find was 36 wrong key incidents. And we at Untold could find in 40% of the games a total of 208 wrong key incidents. Let this sink in. They found 36 mistakes in the whole season. We found 208 mistakes in 40% of the games.
Let me put this another way. If we assume that the 40% of games Untold reviewed that season were typical of all the games (and yes I know that is an assumption but in terms of statistical analysis that is a far bigger sample than that which is generally used), then Untold would have found 620 errors against 36 from PGMO.
For every error PGMO found, we found 17.
If our figure is correct PGMO is only reporting 5.8% of all the errors.
Shall I put that another way? For every 100 errors Untold found during that season, PGMO found under six.
This is totally out of proportion if you ask me. I really would like the PGMO to give us some more information on that short list with key errors mistakes. A short list of 36 mistakes perhaps, with just the game and the names of the teams involved and the time of the incident and why it was wrong. Then we could check and compare this with our numbers and our data. Would the PGMO dare to do this?
You could argue that at Untold we have been too strict. But we reviewed the games as if we were a 4th official with the power to correct mistakes. I have a feeling the PGMO has done their reviewing in the way of: “could he have seen it or not.” And that totally ignores the fact that within seconds someone on the other end of the camera could see it.
Or maybe the PGMO just wanted to fabricate a report to make them look good? Just imagine that? Could this be possible? Could it be that the PGMO is not an honest organisation?
Or could they just be getting a little fed up with the constant focus that Untold and Referees Decisions give to Premier League Referees
Just a question I ask.
Do you trust an organisation that doesn’t give anything away during the season and is as closed as can be? An organisation that doesn’t do what they should do (like informing the public about referees – as it says in their limited company declaration). An organisation that has rules but which doesn’t follow the rules?
So do I trust them? Can I trust them? Do you trust them? Do you trust your government? Completely? Don’t you think they try to keep things covered up?
So is there a difference between the PGMO and what any government would do? Change the numbers a bit so they will fit them? As long as the PGMO doesn’t open up and let us see their analysis from each game I cannot trust their numbers.
As for Untold and Referee Decisions all our reviews are published and are open to see for each and everyone.
Coming to this web site very soon – a world first:
Referee Rumours
The books…
- Woolwich Arsenal: The club that changed football – Arsenal’s early years
- Making the Arsenal – how the modern Arsenal was born in 1910
- The Crowd at Woolwich Arsenal FC: crowd behaviour at the early matches
- Royal Arsenal: from the Common to the Manor. Coming next.
The sites from the same team…
- Referee Decisions – just what are the refs up to this season?
- The Arsenal History Blog from the AISA Arsenal History Society
Wow , Walter ,great work ,truly impressive ! Except that last part that you wrote “a question “,but went on to ask 10 !
Well my answer is NO ! to all ; but a big YESSSS!!! to, “Don’t you think that they try to keep things covered up ?”
Ah well Brickfield once I start questioning I can’t stop 😉
Excellent, Walter. Thank you.
36 mistakes in the whole season? Are you sure you haven’t mis-read PGMO’s figures? Haven’t there been Arsenal matches where 36 wrong decisions have been made?
Please keep questioning – and let us know the results. Thanks again.
It’s not just inconsistent, it’s damning.
Keep at it Walter, we’re all behind you on this.
So in 155 games, 70 sending offs were incorrectly called or not called according to your reviews? That’s a staggeringly high figure, is it correct Walter? If you take it as an average, that would work out to just over 170 in the season.
That’s still less than a sending off every other game, and remember if the refs started to sanction players correctly, the players would adapt quickly to stay on the pitch.
Keep in mind that the not correct sending offs can also be the result of not giving second yellow cards.
Something one sees a lot with some teams. If you know what I mean? 😉
We know what you mean Walter!
It’s particularly jarring to view this in comparison to the progression recently made in another huge sports competition, the NBA, which has just brought in retrospective punishments for ‘flopping’ or diving to earn fouls calls. This development is backed by huge pressure from fans, and it strikes me these kinds of tweaks to officiating systems are a small steps toward a better officiated contest whether verdicts are given during matches or retrospectively.
How do English refs perform when controlling European or international matches? How does their performance compare to European refs doing matches that are not of their country?
Great work Walter/Untold Towers!
How do “Match accessors” and “Match Delegates” come to there conclusions/results?
Do they sit in the stands, at games and make their reviews of referees from one static position? That cant be right. Maybe that’s how they come up with stupid numbers. Or, do they review the game post-match from video before making their conclusions?
The PGMO(B/L) needs opening up, be made accountable to the fans!
Role of the PGMOL Assessor
An assessor working for PGMOL has been an active match official in the professional game who has trained as an assessor and has been accredited to work in The FA Premier League, The Football League and The Football Conference as well as in Reserve Team Competitions.
He reports to a Divisional Manager who in turn reports to The Management Team including the General Manager and has two key functions:
• Guardian of standards in terms of the Application of Laws of Association Football
• Acknowledgement of good practice and advising on strategies for improving performance and developing the skills of referees, assistant referees and 4th officials
Each match official’s performance has to be measured against a national standard and percentage marks awarded to inform a merit table.
In order to carry out this role, an assessor has to be present pre match to make contact with the refereeing team and then observe all 4 officials in action from a position over the halfway line thus having a clear and unobstructed view of the four officials. Relevant information is recorded as the game is played to facilitate the recall of events to inform both the verbal and the written feedback.
What does the assessor look for during the game?
• Accuracy of Law application
• Player management skills
• The overall quality of teamwork between the four members of the referee’s team
• Technical skills displayed by referees and assistant referees
Post match, the assessor de-briefs the team in their dressing room, alluding to the above areas of the performances of the four officials offering praise for those aspects of their performances that were demonstrated well and giving practical advice to address any shortcomings that have been noted. Accuracy of decision making regarding key incidents, such as penalty kicks awarded or not awarded, are later checked against a video / DVD before completing the written report which is usually compiled within 24 hours of the game and fed back to the individuals concerned, their coaches and the appropriate League or Competition. Speedy feedback is important to enable the recipients to consider the advice contained in the report prior to the next game and it is a fundamental requirement that there will be no surprises in the written report as everything that is written should have been discussed in the de-brief at the ground immediately after the game.
All PGMOL assessors are accredited having attended two full day training Seminars each season, and their written reports are appraised twice a season to ensure that they continue to meet the standards expected by PGMOL of its assessors with their position on the National List reviewed twice per season.
Assessors play a vital part in the training and development of the match officials operating in the professional game and PGMOL insists that they, too, are accountable to the Management Team for maintaining their own high standards so that the active match officials are supported in their endeavours to produce high quality on field performances week in, week out.
Derek Bray, PGMOL Assessment Co-ordinator
———————-
Sorry for the long copy & paste, but,
Hope it helps.
So the assessors do exactly what the Untold assessors do, but we have a massive disparity in results, all are trained to official refereeing standards.
This may highlight the disparity between both sets of documentation, also the closed nature in which the assessors work with the assessed, they actually admit to a certain amount of collusion, but they call it training?
In other industries (mine included) our data is verified by another unrelated company or entity, however the PGMOL are allowed to verify their own findings, interesting.
Fantastic work Walter. The PGMOL standards are so poor that the organization should be replaced immediately by a fully accountable body.
Two questions: –
1. Do we know how the key errors affected each team, or if there is too much work in that, the top 5 or 6 teams?
2. Have the full PGMOL audited accounts ever been published?
bjtgooner,
In a way the PGMO is saying that only 36 key errors have been made. Which is rather ridiculous of course. But it coincides with the old saying: ‘it all evens out at the end of the season”. After all ‘only 36 mistakes’.
So in my opinion it is just another way of pretending all is fine and well in the referee world.
Now a next step should be that the PGMO presents that list of those 36 key errors. Shouldn’t take long I think to publish that list.
But for some reason I will not be holding my breath till they publish that list 😉 They maybe would love me holding my breath 😉 🙂
OK so next season Walter, could it be possible to break down the incorrect decisions when summarising the ref reviews into ‘Incorrectly Called’ and ‘Incorrectly Not Called’. It’s really good having knowledge of the incorrect decisions but this way it should identify how the bias is distributed and we may see some more patterns emerging.
@Adam
I feel the pgmol assessors/delegates don’t quite do exactly what Untold does, not even close. My questioning was more about how they get to their findings, on the day. If they only review the ref, from one static position from the stands and not look at post-match video, how can they make an accurate assesment?
Maybe I was a tad eager. But the information is there for you.
It clearly states how they come to their findings.
@Adam
(June 16, 2013 at 1:06 pm)
Thanks, I’ve read it thoroughly not skimmed over it like I did before(sorry). Some much text on a small screen.
So, the pgmol policing themselves. Just like the politicians and bankers, we know how that turned out.
Would referees confess ? So much we could learn !
But in the meantime –
A Priest was being honoured at his retirement dinner after 25 years in the parish. A leading local politician and member of the congregation was chosen to make the presentation and to give a little speech at the dinner.
However, since the politician was delayed, the Priest decided to say his own few words while they waited:
“I got my first impression of the parish from the first confession I heard here. I thought I had been assigned to a terrible place. The very first person who entered my confessional told me he had stolen a television set and, when questioned by the police, was able to lie his way out of it. He had stolen money from his parents, embezzled from his employer, had an affair with his boss’s wife, taken illegal drugs and gave VD to his sister. I was appalled. But as the days went on, I learned that my people were not all like that and I had, indeed, come to a fine parish full of good and loving people.”
Just as the Priest finished his talk, the politician arrived full of apologies at being late. He immediately began to make the presentation and gave his talk:
“I’ll never forget the first day our parish Priest arrived, said the politician. In fact, I had the honour of being the first person to go to him for confession.”
The response was deafening!
Moral: Never, Never, Ever Be Late!
Cheers ! And goodnight !
@Brickfields
My sides are hurting!
@ bjtgooner – good , it means that my work is done and this definitely is goodnight from me ! Cheers !
I now lay me to sleep and ask God my ……ZZZZZZZZZZ
Those 36 incorrect decisions could have come from a couple of matches only, given what we were saw last season. The pathetic figures presented by the PGMO are embarrassing, compared with Walter’s figures. The PGMO is not fit for purpose, and the only other thing I can think of that comes close to this body’s levels of incompetency is a parastatal, under a dictatorship, in a 3rd world economy.
What we need from the refs is more consistency either more consistency in correct decisions or consistency in wrong decisions.
Excellent work as usual Walter by the way- but it occurs to me that the people who employ PGMO are the problem.l They see the games, see the bias and the unfairness– presumably have a dialogue with Riley and then do what? If I was responsible for that rag-bag of performances and the load of tosh and soft soap they come out with– as an FA employee– I would be ashamed to enter a football stadium never mind lunch there on a match day. Who are these people- they have names and they are paid out of fans hard earned? And they must people above them to whom they must answer. Where is the spineless Press when you need them? All this needs routing out. All fans want is fairness. An incompetent but fair ref is better than a dodgy ref any day. Why do the dodgy refs and dodgy assessors not get the boot?