“Making the Arsenal”: In case you didn’t get given it for Christmas it is available on Amazon, Arsenal on line, the Woolwich Arsenal site, and in the Arsenal store.
Follow us on Twitter @Untold Arsenal
By Walter Broeckx
60 games done. 60 games reviewed by our team of reviewers. And I would like to thank them for their hard work. And sometimes you get to see strange things. In the numbers. This time I would like to take a look at the offside decisions in all those games.
A few weeks ago a top guy from the FA said (flanked by former top ref Colina) that 98% of the offside decisions were correct in the EPL. Now I don’t know if the FA check each decision that is made and use the white line that is used on TV to see if the decisions are correct.
I do know how we check the offside decisions. We check them when we can check them. This means that I think 50% of the decisions go unchecked and are considered correct. Because what we can’t check is assumed correct. So if the TV doesn’t show a replay we think it is correct. In fact our system is very very soft for the refs in fact. Because most of the calls are just assumed correct.
But when they show replays we can check and then we can see how the decisions were. But remember this is only the case in half of the offside decisions. But when we make up the final numbers we count all the decisions together. Also those that we couldn’t check. And this leads to a final score in our reviewed games as shown in the table below.
| Offside decisions | Games | Correct decisions | Total decisions | % |
| Total EPL | 292 | 320 | 91,25% |
The first thing you see is the fact that we come to a number of 91,25% correct decisions after 60 games and the FA tells us the fairy tale of 98% correct. Who is fooling who? And then again think of the fact that of that 91,25% we have given 50% to the refs as correct because we couldn’t check them based on the video we had at our disposal. Just imagine what the number would have been if we could check all the decisions completely.
Of the 60 games we reviewed we had a number of 39 games with a 100% score of the offside decisions. In the other 21 games we had mistakes. That is in 35% of the games mistakes have been made by the assistants. Now I do act as an assistant myself on a very regular basis this season so I know how difficult it is. But I really wonder how many goals have been given or have been wrongly disallowed from those wrong decisions.
And as an assistant I follow the guideline that in case of doubt we should give the benefit of the doubt to the attacker. I wonder if this rule is also used in the EPL in every game? Maybe they could change it and give the striker a margin of doubt even before to raise the flag and then use video evidence to check if the striker was onside or not to give or cancel the goal? That would be revolutionary but would take a lot of pressure from the assistants and also they could keep the flag down in case of doubt and wait for what will happen. The ultimate “wait and see-rule”….
In my work as an assistant the refs (the good ones) always tell me that they follow me on the offside calls we as assistants make. But I don’t know how this is done in the EPL. I sure hope the refs follow the assistants. In our ref reviews this is very important for the refs because we count the wrong decisions from the assistants on the bill of the ref himself. So bad linesman give low points to the ref.
When I checked the numbers of the offside decisions I also noticed that in all those 60 games we had 2 games where 50% of the offside decisions were wrong. That was the lowest score in the offside decisions in those games.
And if we take a look at those games we see a few similarities. The first one is …. the ref: Phil Dowd. Despite Dowd having high numbers so far this season he also has the record of the lowest offside decisions in his games. Twice a score of 50%. And when you have a score of 50% in some decisions as ref this is almost the same as tossing a coin to make a decision. There is one chance in two that it is heads or speaking in football terms one chance in two you made the right decision.
So how does it comes that it was twice Dowd who had such a poor offside record in those game? It couldn’t have been the assistants because we have checked them it was 4 different assistants in those 2 games. So there is my question: how does it comes that Dowd is “unlucky” in two occasions to have bad assistants? I will not mention the possibility that they got some instructions from the ref himself before the game how to flag and when not to flag? Oh, I just did.
But if we just assume that well it was a coincidence. We might look at those games. And then we see that the first game was Manchester United – Chelsea. MU won with 3-1 and 2 goals where clear offside goals. Okay lightning strikes sometimes. The total offside decisions to make was 4 and 2 were wrong.
And the other game was Manchester City – Arsenal. 6 decisions to make and 3 wrong. And one was the goal from Arsenal that was cancelled.
So we can see that both Manchester teams got 3 points in those games from wrong offside decisions against…2 London teams. And both under a Northern ref (if my geography is a bit what it used to be when I visited school a long time ago).
Does lightning strikes twice in the same city against visitors from the same part of the country? Well a London team better hide for the lightning when they visit Manchester and Phil Dowd is in charge.
“Making the Arsenal” – is available on Amazon, Arsenal on line, the Woolwich Arsenal site, and in the Arsenal store.
Follow us on Twitter @Untold Arsenal
————————————————————–
Referees: the geographical bias
1: The Ref’s Association has closed its web site
2: London people are unfit to referee according to the referees’ association
NEW: 3: What did Mr Wenger mean when he said “We know what to expect in London”?
Also…
Corruption in the Premier League: Poor-Bert – the next head of PGMO
Incompetent or bent? Certainly one or the other. Fulham v Arsenal – the ref review
———————————-
The 10 iconic moments that defined Arsenal’s history
Part 1: Opening the club to all comers
Part 2: The Great Conspiracy – when they tried to shut Arsenal down
Part 3: Death and rebirth in 1910
Good morning Walter,
Having had a quick skim through (I’ll read properly later at lunch) I am just wondering if the 98% accuracy figure quoted by the FA only considers offsides which were given and doesn’t include offsides that should have been given but were missed? I can’t imagine the FA having a panel like yourselves to look into such matters.
In these two games, Dowd has been extremely unlucky to have such incompetent linesmen with him for these crucial fixtures.He has been one of the top referees in the league and the likes of the arsenal-newcastle game (a classic example of british game) are included in his resume. He is one of the strong contenders to follow Riley.
Just because he got some wrong decisions in these two games, ruling him out as one of the best referees in the league a mistake only a novice will commit.
That’s what would have been said by the media (if they bothered to publish this ).
Nice article Walter 🙂 hope we see more articles of the referee’s who are in pole position for the riley’s job. I am sure it would be someone capable of working completely anonymous in the higher (corrupt) echelons.
This is a very good article. Referees have a 50% chance of getting decisions right by sheer chance alone.
We need a completely neutral, independent body to monitor what is or is not given by refs and to review the decisions in each EPL game not a body like the FA that clearly have an interest in portraying referees as being always right. Given the level of incompetence of EPL referees who can look at a man being tripped in the penalty area in front of their eyes and wave play on as Lee Probert did at the weekend- it beggars belief that they are getting offsides right 92% of the time. You have to ask when they are seriously throwing around a stat like that- are they for real?
Thanks for the good work!
Even 98% is a poor figure for an offside decision as only one wrong decision can change the result of match. The offside decisions must be 100% correct and that’s possible only through technology.
@Stuart
I believe the analysis (Walter’s white line) is something that Prozone Sports could do for PGMOL. Certainly they are providing some kind of service to PGMOL, as there have been announcements of arrangements between PGMOL and Prozone. But supposedly these arrangements with Prozone started before Poll retired. And yet, someone in another thread mentioned that all the feedback the referees get is a number, no “training”.
@Walter
I still don’t think technology is the savior you do. If you bring in video replay, and leave the system the same as it is now, it may help the assistants. If you bring in video, and then change how the offside is administered, the attackers will change what they are doing to “push the envelope” more, and it will result in little or no help to the assistants.
It is likely that only the decisions that are called, are analysed. Certainly that is the “lazy” way to analyse something. Otherwise, someone has to decide what has to be analysed.
If we look at 292 correct out of 320, we expect the standard deviation on 292 to be something proportional to 17. Which means that 309/320 is statistically the same as 292/320 (96.5%), at the 67% confidence level.
If some analyst looked at every situation where an attacking ball passed the line of the second last defender to see if any attacking players were 1) in an offside position and 2) in a position to take advantage of that offside decision, we would probably see something like twice as many decisions? What is the ratio of called to uncalled? If we double the number of correct and total decisions, and look at what adding sqrt(correct) to correct brings, instead of getting 96.5%, we get 95%.
A big part of the problem is that the league hasn’t properly defined what they mean by correct. And even if they did define it properly, they need to provide the data so that this 98% number can be verified.
@Walter
I better go through my numbers again, I had only been awake 20 minutes or so, and no coffee. I’ll get a better answer as to how likely or unlikely that 292/300 is consistent with 98%.
Back again. If I treat how many decisions in total as a Poisson variable, and then draw a binomial variable from that total number of decisions that has a probability of success of 98% (what EPL claims), and generate 100,000 experiments, I see an actual correct fraction as low as 94% in 0.008% of the trials.
If I drop the claimed rate to 97%, I see 0.002% of trials getting as low as 92% (just outside your 91.25%).
In order to observe a fraction correct as low as 91.25% in 5000 trials out of 100,000 trials (95% confidence level), the “true” ability to judge offsides correctly (or however one words it) is about 94% (or less), not the claimed 98%.
Another way to word it; if we observe 91.25% correct, at the 95% confidence level we expect the actual percentage correct to be between about 88 and 94%.
@Gord
I agree its very important to check to make sure the statistical significance is appropriate, but I would take a different tact and look at the teams being evaluated and see if certain teams have higher total #’s of offside calls(due to style of play) and check that against the percentage of each team reviewed. That initial bias could distinctly impact/skew the calls. For example, fow many spurs games have been evaluated – Adebayor is notorious for being offside. Arsenal also tend to play through the last line leading to a relatively high number of offside situations compared to say Stoke who cross into the box.
@walter
Maybe you’ve posted it and I missed it, but it would be helpful to see the percentage of games per team evaluated. So out of 60, probably 25-30 have been Arsenal games. As I mention above, this can skew the statistics due to a natural bias in the way Arsenal play. It would be interesting to see that rates for each of the top teams. Off the top of my head I would bet Arsenal and Spurs have the highest number of offsides, followed by city, then chelsea, just by the way they play.
@Todd
Perhaps you could generate some data? 🙂 Being an engineer (with a minor in statistical mechanics), I’m good with numbers. There is a problem in doing this per team, in that the total number of decisions falls, which is going to increase the standard deviations of any analysis I do. The thing to do, would be to start from the teams for which the most data is available, and work down until it gets too noisy.
@Walter
I went to the gym, and then came back and did a bit more. Instead of just 1 binomial deviate for each Poisson total number of calls (and 100,000 of those), I did 100 binomial deviates for each Poisson. Nominally 10,000,000 trials per run.
The EPL has no grounds to say that they are 98% accurate based on what referee reviews UntoldArsenal has done. Not even 1 in 10,000,000 will hit 91.25% accuracy if their “true” rate is 98%. With this expanded run of simulations, it is pretty obvious that the true rate for getting offside decisions correct, is somewhere between 88 and 94%.
we had some 16 Arsenal games in the total of 60
pretty ‘sophisticated’ discussion happening here. My comment won’t contribute anything to the discussion. I just want to say how amazed I am with the work on this site about the refs. I learned a lot about rules of the game and the ‘truth’ of Premier League and I share what I get from here with my friends in Indonesia. I want them to be Arsenal supporters who do not only care about scoreline.
super work, everyone!
Making statements based on statistics is not a good thing. It is sort of like standing on quicksand that has a crust on it. Is it going to break through?
You cannot prove anything with statistics. If you are lucky you can disprove things. But advertisers the world over, continue to say that some clinical study proves some product has biological efficacy. The clinical study is there to gather statistics, a clinical study can’t prove a darned thing. Except that any company that says that some clinical study proves their product is effective, means that you should never buy their product ever again as clinical studies are not meant to prove anything (outside of advertiser stupidity, which knows no bounds anyway).
Claiming 98% success on anything is good and bad. People like to believe that the 98% claimed is real. Being able to prove 98% can be difficult. And any proof is going to hinge on how the success is defined. Not just in general, the success may only be possible to do some very specific details in the definition.
A referee in a game (or an assistant referee) only has a small interval of time, and only a single view of the incident, to make a decision as to offside or not. Reviewers from UntoldArsenal (or other places) have an indefinite amount of time and as many reviews as they want to make a decision. Assuming they can get views to look at.
I think that technology is useful in training referees. Which is something that apparently Graham Poll has said recently, and wondered why EPL referees weren’t using. I’ve asked on UA about why Prozone Sports isn’t providing this, seeing as they have been providing services to PGMOL for many years and are seemingly capable of doing so based on what they advertise to provide. And nobody on UA seems to want to comment on this. Am I beating a dead horse? What services are Prozone Sports providing PGMOL, if they aren’t training referees using analytical methods and video? BJs?
Walter wants to prove something about referees. I went to the trouble of doing something like 100 million experiments to see whether an EPL claimed offside effectiveness of 98% is reasonable. I think I’ve shown that 98% is not consistent with what UA reviewers have been seeing, and that any real offside detection rate in the EPL is somewhere between 88 and 94%. And no comment. Either online or private (as he has access to my email). Which I don’t understand.
If further analysis of things is useful, having some kind of dbase access is useful. My suggestion, is CSV (comma separated value) is a useful format for everybody. UA has reviews (mostly for Arsenal) going back a year or three. Among other things, a person could look at decisions based on nationality, region of England, race or a few other things. But scraping HTML to make database entries to do something is not a time effective way to start things.