Quantifying the Bias of Arsenal’s Referees vs. The Rest of the League’s Clubs
By Zach Slaton and DogFace
This post can also be found at http://numbersgameblog.blogspot.com/
Author’s Note: Special thanks to DogFace for his co-authorship on this post. His voluminous data set, unending patience, invaluable insight and contribution, and constant editorial feedback throughout the creation of this article was invaluable. He’s a wonderful blogging partner with whom any Gunner or statistician would be lucky to work.
In my first post in this series on Numbers Game blog I used DogFace’s match data to explain how Phil Dowd is the least desirable referee for Arsenal as he not only shows the most biased officiating in terms of fouls, yellow cards, and red cards, but he also shows the largest effect on Arsenal’s likelihood of winning a match. That analysis focused on the effects of all of Arsenal’s referees, but did not quantify how those referees officiated other teams’ matches. This post contains such an analysis, and the results are very interesting.
To aid in such an analysis, a binary logistic regression (BLR) model was created for each team’s likelihood of winning a match based upon a number of factors. Each BLR model includes terms that capture the effects of venue (home/away) and differentials of shots, shots-on-goal, corners, fouls, and fantasy points for yellow and red cards. Not every term was significant for each team – terms that had a p-value of 0.10 or less were eliminated from the team’s BLR and their coefficient for that term is set to zero.
I know that may upset some stats geeks who would prefer p-values of <= 0.05, but the reduced sample size requires a lower p-value threshold or very few teams would have any significant terms. The resultant BLR’s allow a construction of each team’s odds of winning each match, and a study was constructed as to how each official impacts the odds of winning each match based upon their officiating versus the expected average officiating from the club’s total number of matches.
Background on the Data Used in the Analysis
Before diving in to any analysis, a few statements on the types of data used are in order.
The intuitive way to model bias in a referee would be from the fouls per booking and bookings per match figures – although DogFace has often found these to be counter intuitive in that, if a bias exists, it can be expressed within the game in different ways – these often depend on the styles of play of the teams involved.
The noise in these figures is further increased by the fact that the official statistics only reflect what the referee deemed to be a foul rather than the reality of a foul (according to the laws) and/or the consistency of the referee’s interpretation thereof. “Sins of omission”, where a referee should have called a foul but for some reason or another did not, are tough to quantify in an analysis that uses such statistics.
To combat this effect, Untold Arsenal utilises a professional referee (Walter Broeckx) to analyse matches and it is clear that what is recorded in the official statistics is often way off the mark. DogFace’s own data sets confirm this effect. The rise in popularity of statistical analysis in football has led him to notice a trend for parity in the fouls per booking figures that suggests that the referee is aware of his statistics in game – it could be said that this ‘trial by media’ has created the bias we see in these figures and that a referee himself has a conflict of interest in every call he makes or indeed does not make.
The ideal situation would be an independent body to record/analyse referee performance and provide an open source database for us to comb through though – this data would include the most important data of all i.e. the standard ‘human’ error. Alas, we do not have such research based on an independent body, and instead you will have to make do with the work of humble bloggers like Walter Broeckx, DogFace and me.
Quantifying How Arsenal’s Referees Officiate the Rest of the League
Just like the last post, each team’s average foul and fantasy point total was calculated. These variables represent the match attributes directly controlled by the referee. Each team’s actual odds of winning each match were compared to the odds that would be realized if the referee had given the team’s average foul or fantasy point differentials (different values were used based upon the team’s averages for home and away matches). This allows a calculation of an odds differential for each match.
Matches were then grouped by referee to allow an analysis of each referee’s overall bias vs. the average odds. In this case, the same referees as the first post were examined – Atkinson, Bennett, Dean, Dowd, Foy, Halsey, Webb, and Wiley – as they had officiated the greatest number of Arsenal matches and in general a large number of matches overall. Most importantly, they have officiated a range of home and away matches so the impact of home/away bias is minimized.
Unfortunately, the limitations of the number of matches in a season and the availability of each of the referees prevents an analysis where each referee officiates an equal number of home and away matches, but the statistics used in the analysis take the effect of venue into account to minimize its effect on match outcome. The data used for this analysis was also isolated to the 2006/07 through 2009/10 seasons to ensure an adequate number of samples were available from each referee.
It should be noted that for the first part of the analysis, referee data from all four seasons was grouped together. This allows us to look at gross bias throughout the seasons, and keeps a pretty high sample size for the analysis. Later in the post key referees’ data is broken out across seasons to show the effects of referee by season.
Such a study by season helps us see any bias that may be dependent on club’s objectives (Premier League title, European competition qualification, avoiding relegation) and current quality of play (above, at, or below form). Inevitably, such seasonal analysis suffers from limited sample size and is saved for those referees whose overall bias is the most noteworthy.
One further adjustment to the data set had to be made. When creating BLR’s for each team, several of them showed no impact to their odds of winning a match due to foul or fantasy point differentials. That is to say that none of the two coefficients for those terms in a team’s BLR were statistically significant – the coefficient’s p-value was greater than 0.10.
This means that we can’t evaluate the impact of officiating on those teams’ likelihoods of winning a match. Thus, teams that did not have a significant BLR term from the two associated with officiating bias – fouls or fantasy points – were eliminated from the analysis as they’d provide a zero impact and generate misleading results.
A general linear model (GLM) using referee and team officiated (Arsenal vs. not Arsenal) was created from the reduced data set to observe the resultant interactions. The results of the analysis are represented in the graph below which shows each referee’s average odds differential for Arsenal and the rest of the teams. The two plots in the graph show the same data in two different manners. The key plot is the one in the lower left.
Before diving in to the plot, readers with a keen eye will note that the red line, which represents Arsenal’s average odds differential by referee, is a bit different from a similar line plotted in my previous post. This is due to the two different ways the data is analyzed. In the previous case, I was looking at the referee’s performance as a function of just Arsenal’s matches by year. That led to the grouping and averaging to be a bit different than this analysis, which simply looked at Arsenal versus the rest of the league regardless of season within the four analyzed. This latest analysis essentially ignores the effects of time.
Looking at the lower left hand corner of the plot, it’s clear that Dowd shows the largest gap between how he officiates the average match and how he calls an Arsenal match – about a 1.5% penalty for Arsenal. However, what’s different about this analysis is that it also clearly shows that Howard Webb is pretty biased too – also nearly a 1.5% penalty for Arsenal. Rounding out the top three are Wiley, Halsey, and Bennett each with a nearly 1% penalty for Arsenal. Atkinson and Foy seem to show the smallest gaps.
Another way to look at in-match bias is to look for odd patterns in fouls-per-booking. A plot of four factors – Referee, Arsenal/Not Arsenal, Season, and Home/Away – and their impact on fouls-per-booking is shown below. Values on the left or right side of the plot represent the fouls per booking for the plots across that row. Values above and below the graph represent different levels of each factor expressed as text in one of the boxes in each column. The legends to the right of the graph explain what each color/shape combination represents in each row. Thus, if one wants to understand how each referee’s average fouls per booking for Arsenal compares to their fouls per booking for the rest of the league, one can look to the square in the 2nd row/1st column or the 1st row/2nd column. Each of those two plots shows the same data, but is each plotted in a different manner.
Looking at the plot of data in the 2nd row/1st column, one sees there is only one referee that has a higher fouls per booking for Arsenal than the rest of the league – Steve Bennett. Yet again, Howard Webb and Phil Dowd lead the pack with their differential of fouls per booking for Arsenal versus the greater numbers of fouls-per-booking they allow for the rest of the league. Surprisingly, the favorable Chris Foy also has a large gap between how he calls Arsenal matches vs. the rest of the league.
What’s especially disturbing is the fact that Dowd ends up with such a bias against Arsenal in fouls-per-booking given his disproportionate number of home matches he’s officiated for Arsenal (7 home matches to 2 away matches). Take a look at the plot in the lower left, which shows home (red) and away (black) fouls-per-booking by referee.
Overall, Phil Dowd shows one of the largest gaps in favor of a higher fouls-per-booking average home versus away – nearly two fouls per booking. Yet such an advantage is not showing up in Arsenal’s numbers when Dowd is officiating their home matches. In fact, when looking for statistically significant factors in the GLM of fouls-per-booking vs. referee, club, season, and venue, the interaction of referee and venue is the only statistically significant term! Some referees are being consistent between home and away matches, while others are showing large gaps.
Comparison with Pre-Match Expectations via the Asian Handicap Swing
Does this comport with pre-match expectations? One loose measure of such expectations is the Asian Handicap assigned to a match. Luckily, DogFace has also been recording this statistic for each match. DogFace has also introduced the concept of the Asian Handicap Swing (AH swing) to readers of Untold Arsenal. The AH Swing is computed via the following equation:
AH Swing = Actual Goal Differential – Asian Handicap
The AH Swing represents the performance or deviation against the handicap in actual goal difference. The betting line data (or Asian Handicap) is an average calculated from around 30-50 bookmakers across Europe and Asia. We assume the average bookie handicap does not use corruption/bias as a significant factor in the markets – not only for the reasons stated below but also because referee driven match fixing would only affect the markets significantly in terms of an ‘upset’ against the odds.
If we were to take the stance that the betting line reflects referee corruption/bias then any deviation from that line in terms of an over/under performance would be understated i.e. we could say that any ‘noise’ in the original handicap from the bookies would actually understate the bias we are attempting to model. Luckily previous research does not indicate this case, but rather indicates that on the whole bookmakers may be creating a market that is essentially based on efficiency and competition. At the least, they are essentially hedging a gestalt metaphysical abstraction based on media disinformation and the credulous belief that “it all evens out at the end of the day”. However it is worth considering that, as perception of bias in referee performance becomes more ‘main stream’ and paradigms shift, we will see this effect in reflected in the markets of the future.
This post will instead focus on the reality of the results as it is a far more revealing stance to take. The Asian Handicap line is one that reflects, more or less, the illusion of an uncorrupted market. The swing from that line allows for an examination that a corrupted, or biased, market exists.
Just like odds differential, the AH swing has been calculated for each and every match in the database. A plot of four factors – Referee, Arsenal/Not Arsenal, Season, and Home/Away is shown below. Values on the left or right side of the plot represent the Asian Swing for the plots across that row; values above and below the graph represent different levels of each factor expressed as text in one of the boxes in each column. The legends to the right of the graph explain what each colour/shape combination represents in each row. Thus, if one wants to understand how each referee’s average AH swing for Arsenal compares to their AH swing for the rest of the league, one can look to the square in the 2nd row/1st column or the 1st row/2nd column. Each of those two plots shows the same data, but plotted in a different manner.
Looking at the plot in the 2nd row/1st column, it is shown that Mike Dean, Howard Webb, and Phil Dowd have the lowest average values for Arsenal’s AH Swing. This means they are consistently officiating Arsenal’s matches tighter than the other referee’s. Unlike the analysis of fouls-per-booking, the interaction between referee and venue is not statistically significant. Referees consistently provide a significant AH swing advantage to home teams compared to away teams.
It has now become clear that on multiple fronts that Dean, Dowd, and Webb appear to be the most biased against Arsenal. To get an even better understanding we must look at how each of the three officials impact each team and compare those matches to how bookies might expect them to turn out.
Quantifying How Arsenal’s Referees Officiate the Rest of the League
One way to visualize whether or not Arsenal is the most penalized when it comes to the officiating of Dean, Dowd, and Webb is to look at how each team’s average odds differential compares to their average Asian handicap swing when these three officials are present. The three graphs below represent just such a comparison for each referee. The x-axis represents the average Asian handicap swing. The y-axis represents the average odds differential.
The key to the graph is the lower left hand quadrant. Teams that end up there, especially those that end up further away from both of the lines along an imaginary diagonal line extending from the origin of the graph, are likely experiencing a higher amount of bias in officiating than the other teams the referee has officiated. The three graphs below show such plots for each team against each of the three referees. A summary of conclusions is found after the third graph.
Here are some conclusions that can be drawn from the three graphs:
● There is clearly a band around the ± 2% region of the odds differential where most teams cluster.
● It’s also clear that the top teams – Arsenal, Chelsea, Liverpool, and Manchester United – tend to do better on the AH swing than other teams. Indeed, teams that would generally have a more positive Asian Handicap tend to be to the positive on the swing.
● While Arsenal do seem to be experiencing some bias at the hands of the three referees, they don’t seem to be the worst off. Liverpool clearly pays a bigger penalty under these three referees.
● Of Arsenal’s main competition for league trophies the last half decade, Manchester United and Chelsea get a much better shake from the referees. Each has a vastly superior AH swing, while each of the two teams finishes much better in terms of odd differential than Arsenal for two out of the three referees. These advantages translate to an average of a 0.73 goal benefit in AH swing and a 1.6% benefit in odds of winning a match.
● Chelsea’s average odds differential is 1.6% better than Arsenal’s, while their average AH swing advantage is 0.78 goals.
● Manchester United’s average odds differential is 1.5% better than Arsenal’s, while their average AH swing advantage is 0.68 goals.
● Mike Dean is the only referee of the three to put both Chelsea and Manchester United to the positive. He’s also the referee demonstrating the highest bias against Arsenal in AH swing, both nominally and when compared to Chelsea and Manchester United (a 1.52 swing deficit to both).
● Perhaps Ryan Babel’s tweet with Howard Webb wearing a Manchester United jersey wasn’t too far off. It Manchester United is his most favored team by a mile both in terms of swing and odd differential.
● Of any referee who has officiated the Big Six, Mike Dean gives the most favorable treatment to Chelsea. Their average odds of winning a match are improved by nearly 1% when he officiates one of their matches, and they experience their best AH swing under him.
● Of Arsenal’s North London rivals, Spurs are treated about even by Dean and worse by Dowd and Webb.
There are some interesting interactions that go on between referees and managers as well. One could be referred to the Dean-Redknapp effect. When looking at matches where Dean is the official in matches involving Portsmouth and Tottenham, both teams do better in terms of AH swing with Redknapp at the helm than when he is not, bordering on statistically significant effects (odds differential was a wash).
Interestingly enough, the interaction of manager and team is not significant, so Redknapp’s record under Dean is not unduly boosted or penalized by his record from either of the clubs. It stands on its own. Such a detailed analysis would have to be the focus of a subsequent post, and deserves a wider treatment of referees, to ensure that it’s not simply an effect of Redknapp’s superior coaching. Nonetheless, it provides intrigue when studying the effects of officiating.
A Comparison of The Big Six By Season
It has been shown that over the 2006/07 through 2009/10 seasons that Dean, Dowd, and Webb have given Arsenal a bit of the short end of the stick when it comes to fouls-per-booking, AH swing, and odds differential. One last bit of examination remains – what happens if we open up the examination to the entire six years of data in DogFace’s database, isolate for the Big Six for these three referees, and examine how a few of the trends may be changing over time?
The graph below provides a plot of such data. The solid lines represent the teams’ average AH swings under the three referees, the value of which can be found on the right hand side of the graph. The dashed lines represent the team’s average odds differentials under each of the referees, the value of which can be found on the left hand side of the graph.
A few general trends can be observed:
● All teams except Manchester City have been on a general downward trend over time in terms of AH swing.
● Manchester City also is also the only team to demonstrate a steadily increasing odds differential over time.
● Arsenal started out with the third lowest swing in 2005/2006, and their decline has been consistent to the point of falling below 0 this season with the three referees. They are the only team of the Big Six to experience a negative swing.
● The dashed set of lines shows Chelsea and Manchester United hanging around neutral (i.e. 0%) odds differential over time.
● Again, Arsenal is on a steady downward trend to the point that their average odds differential under the three referees is on track to be greater than -2% this season. Only Tottenham has a worse odds differential.
So who’s driving this downward trend for Arsenal? The plot below shows the same data as the plot above, but it eliminates the other three teams and instead focuses on Arsenal’s referees:
On the AH swing front, there’s been a steady erosion the last year. However, before that Mike Dean’s officiating showed a distinct downward path compared to the reed of the referees. His AH swing went negative in the 2009/10 season, and Webb has joined him now in 2010/11. It appears it’s a case of Dean pulling the rest of the average down with him, and the other two referees joining him this season.
As for odds differential, it’s Dean again that leads the pack. He started out as a neutral referee in 2005/2006, but then has steadily eroded that neutrality to a -2% differential by last season. Dean’s low average and the recent degradation in Phil Dowd’s officiating are what are leading to Arsenal’s precipitous drop in 2010/11.
Conclusions
Whether it’s actually poor form generating a higher number of fouls and cards or actual referee bias, it is clear that Arsenal pay a bigger referee penalty than all of the teams they’re competing against for the Premier League championship save for Liverpool. This might be combated by having the three highlighted referees – Dean, Dowd, and Webb – officiate fewer Arsenal matches. They officiated an average of ten matches, or 26% of Arsenal’s season, each of the last four years. Statistics would suggest that having a fewer number of referees officiate a greater number of matches for each squad would lessen the chance of a poorly officiated match impacting a team’s season point total. Such an assumption is based upon the idea that the official’s errors or bias are randomly distributed. The data above suggests otherwise.
Zach Slaton is the author of “A Beautiful Numbers Game” blog where he writes about soccer statistics. He is a supporter of Arsenal and Seattle Sounders FC, and lives in Seattle, Washington. You can follow Zach @the_number_game
Untold Arsenal on Twitter is in the top 1% of all Twitter sites for followers @UntoldArsenal
Untold Arsenal on Facebook here
History of Arsenal including the series on the failures of Herbert Chapman
Making the Arsenal – the book of Arsenal death and rebirth
I am speechless
What a piece of work!!!
That was a lot of reading.
Wow! Amazing analysis. Thank you.
Just a thought…are there any lip readers amongst the Untold faithful? Might be very interesting to compare these data with what actually gets said by referees.
Oh my gosh… this is a statistician fan’s dream come true… thanks, Untold, and writers!
This sort of work is amazing. The next step is to use such data to generate a betting model and sign up enough willing participants so as to cause a bookmaker a serious cash flow problem. If we (as Gunners) keep hitting bookies where it hurts, they will have PGMOL investigated faster than you can say “Pakistani Bowling Betting Scam”.
On a related note, I wonder what the odds of a Real Madrid player being sent off last night were? Aside from the obvious 1:1.
This post took a while as you can imagine – my thanks go to Zach Slaton – check out his blog when you get a chance!
This is going to take some processing by my poor brain. 🙂
On a bias-related note, for those interested in the general background: Mourinho has made some strong comments (including a reference to the recent RVP sending off vs Barcelona):
http://www.guardian.co.uk/football/2011/apr/28/jose-mourinho-real-madrid-barcelona
No comment can justify the beauty and majesty of this piece, so I won’t try.
I think I followed at least some of it. A stupid question though if you dont mind.
Although there seems to be a statistical bias, how does it manifest itself in our results?
The PGMOL is a structure that is designed for failure.
As the Arsenal manager says every once in a while:
‘We are not stupid’.
I think it’s a little presumptuous to talk about bias in referees and then use (wrongly implemented) statistical analysis to confirm your own biases.
From a statisticians viewpoint, to increase the threshold of the p-value when dealing with smaller samples, is the opposite of correct.
This way you increase the likelihood of committing a Type 1 error, or said simply: finding a false positive / seeing patterns where there are none.
It’s nice to see some statistical analysis in football, but please, if you want to have anything relevant to say, use it correctly and rigorously.
If your mindset is “I want to see how biased the referees are against Arsenal” and then use wrongly implemented statistical analysis to confirm your beliefs, of course you’re going to find results that match your beliefs. That certainly doesn’t mean that the things you find are the truth.
With that said, I too would like to see a independent body analyzing referees performance, with a giant database to comb through. It could make for some excellent reading.
Lastly, I would like to leave with some words of advice from the great Christopher Hitchens: “That which can be asserted without evidence, can be dismissed without evidence”
Statistical analysis must always be rigorous and strict.
Wow.. What an effort. From you guys obviously, but from me to read it and try and make sense of it too 🙂 I wish I could understand numbers and statistics better to offer some criticism and suggest any deficiencies. I have to say that mainly I just relied on whatever conclusions you drew to understand the meaning of the numbers.. My worry is that it will be discredited as just Arsenal fans moaning, like Walter’s excellent work (which is easier to understand) is often denounced as. But keep up the good work guys. The effort is much appreciated.
It’s difficult to argue with these figures – there were so many avenues of investigation that we had to cut it down as it was getting out of hand.
But these will be explored in further posts… and hopefully not just by Zach and I but other hardcore stat-monkies too.
Oh – and did I mention that this Blog was a different gravy?
😉
Anyone calls you a whinger, tells you to stop moaning and claim that it all evens out – then just link them to this.
@Dogface
Haha. that should do their head in. If they can’t understand a simple enough thing like it does NOT even out in the end (what would be the odds on that actually?), then I’d like to see them make sense of this 🙂
Good post, can u have this published after adding correlation and regression analysis of the refs Bias and Arsenal loosing matches.
Well, the data analysis is quite magnificent, provided that the ‘independent’ analysis by the revered Arsenal fan, Walter Broeckx, is suitably ‘independent’.
I’m not saying it’s not, I’m just saying that football is littered with sets of eyes who see the same event in a different light.
If I were refereeing this as a scientific paper I would say:
‘Interesting analysis. Now get three referees, known to support different sides, to do the same analysis independently next season on the same matches as Broeckx. Then get the data opened up by a third set of independent parties who don’t know who the analyses come from, merely that they come from Arbitrator A, Arbitrator B and Arbitrator C. This will tell you how much contention there is about the ‘independent’ analysis. If the independent analysis, carried out under supervision without the ability to contact the other two arbitrators, comes through with only small disagreement, then you have proven your case.
Until then, however, there must remain a certain uncertainty as to the accuracy of an analysis carried out by only one person, who is certainly not unbiased in their view of English football, being an Arsenal fan who writes every single published article from one of the most partisan, pro-Arsenal viewpoints on the blogosphere.’
I hope this is seen as a rigorous evaluation rather than a pooh-poohing.
Because the fact is that Arsenal supporters see what they want to see, just as Man Utd, Liverpool, Spurs etc etc fans do too.
It might be helpful if the independent arbitrators supported teams like Leyton Orient, Walsall or Hartlepool, as such clubs are unlikely in the near term to reach the lofty heights of the EPL and hence the analysis will not be coloured with ‘I mustn’t offend a particular EPL club, manager or set of fans……’
Even that’s not easy, as I’m sure Leyton Orient are immensely grateful to the FA Cup revenues this season…..
Pete;
As much as I hate Morinho and his brand of football, he may have a point. I’m refering in particular to RVP’s red stupid card.
@Rhys – none of this is based on Walter’s data… this is based on the official data as recorded by the referee’s log book and the betting line odds posted by the major bookmakers. This is explained quite clearly in the post I think.
Good work but it needs publishing in full view of the sporting public.
Mourinho has done wonders for the game by questioning the Barca effect on referees. He didn’t mention the Van Persie sending off for the most pathetic reason in Champions League history – time wasting by kicking the ball. With several balls available the time lost is minimal. There have been far worse versions of time wasting that are not penalised. There are questions to be answered and Wenger was right to make his statement the UEFA charged him for.
It is a shame that the costs for defending the truth can be prohibitive.
I hear an awful lot of different theories on Arsenal players and Arsene Wenger… and then I got banned for going off topic and posing total bollocks
Ken –
I can say that yes, it is always best to apply a lower p-value. In the case of the yellow and red card statistics, I can tell you that the p-value was well below .05 for the Big Six and nearly every other team. A few of the match statistics – shots, shots on goal, corners, etc. – were where I found a few teams to have p-values for their BLR terms that were a little greater than 0.05. In most cases, they were 0.07 or below. Excluding non-referee match statistics that had p-values slightly higher than 0.05 would have only served to increase the impact of the referees and shown a disproportionate impact on cards that I think would be a bigger error in the model.
It should be pointed out that at the league level, all statistics were significant to p-values of 0.02 or below. I eschewed the use of league wide BLR coefficients as they neglect the fact that some teams are affected more or less than the league average when it comes to yellow and red cards. They same would be said of shots, shots-on-goal, corners, etc. Thus, individual team BLRs were created with a small allowance for a rise in the p-values for each BLR statistics.
DogFace and I will return this summer where we can include the the full 2010-2011 season data in the foundational BLR analysis and resultant coefficients, and I will apply some commentary as to what happens to the p-values associated with them.
@Sammy The Snake
Yep. He has gone up in my estimation. 🙂
Can I just say as editor/publisher that it is an honour for Untold to be selected for joint publication of this work. This is just so far beyond the stuff that appears on most blogs that it really takes us into another dimension.
This is really where I want to take Untold – it is a true reflection of the chosen name. We are Untold because most of what we publish does not appear in other forums.
James,
do you actually have anything serious to say?
Do you ever read an article on Untold? If yes please remain on topic. If no what are you doing here?
Zach and Dogface,
This really is something that is beyond everything I have ever read before on a football blog.
I can only agree with Tony and just want to say that such an article makes me just proud of being a part of Untold Arsenal.
I don’t have a hat like Tony but I would take it off and make a deep bow if I had one!
Always refreshing to read an adult blog. This analysis would be lost on the vast majority of Arsenal fans, such is a pity. The effort must be to get the Premier League officials to understand the impact poor refereeing could have on their bottom line. If they can understand that fans switch off when they view ridiculous refereeing and that such an action could impact their television revenues, they may sit up and take notice. I don’t expect the geriatrics at the F.A. to ask much of their referees’ association but i believe relegating a referee at the end of the season will be the only way to stamp out bad performances.
Quite a stunning piece of analysis on many levels. This goes a long way to confirming what many suspect. It seems so often we get these 3 refs as well.
Wenger is a stat man – I am sure he has his suspicions on refs like the rest of us – maybe someone should send this to him, and put his players on alert so we do not give these refs any more of an excuse than they seem to need.
I was surprised to see Liverpool doing as badly under them as us – could this be a leftover from that Johnny Foreigner Rafas days – when in one season especially, he was a serious threat to Utd. Rafa did go public on refs favouring Utd after all.
@Walter, Tony and “Mike”
Walter, far worse than being off topic is the fact that this is a cut and paste from onlinegooner!
Oops. Sorry I meant “James”..not Mike…too hot under the collar to check on the name!
It is interesting, but the simplest analysis is the one which the FA refuses to undertake, a point by point analysis of games, incident by incident.
If a referee gave more cards to Team X it might just mean that team x are dirtier than other teams.
Reading this very interesting piece and then watching the Sky coverage of the Mourinho Affair today got me thinking about our own experiences with Barcelona this year. Sky got Dermot Gallagher on to earn a few bob for himself by assuming the role of God’s football arbiter to pass judgement on whether Mourinhos’s claims of refereeing bias against his team and, kore pertinantly, for Barcelona. I watched Gallagher last week after we conceded a penalty against Liverpool in the 12 th minute of an 8 th minute extra time. He agreed with the extra time claiming that when the ball is out of play for, say a throw in or a goal kick the ref should stop the clock. He failed to realize that if refs did this throughout the game each match would last about 2 and a half hours. So, I don’t have a lot of time for Gallagher’s whitewashes myself. When Sky played previous sendings off against RM there was a hand to the face which Gallagher immediately said was an automatic sending off. Fair enough. Then I thought that most people always judge the refs on the decisions they do make and not on those they don’t. If a hand to the face is an automatic sending off then why didn’t Busaca send off 3 Barcelona players in our match at the Nou Camp? Walter pointed this out in the aftermath of that horrible evening. Everyone seems to be much more concerned with RVPs dismissal which was bad. But failing to send those 3 players off was even more damaging. I know that Mourinho is a bit of a Pratt but I do feel a bit of sympathy with him.
Can I just add – before someone else points it out – that yes, Liverpool’s data is out of kilter with the other big 6 against the chosen referee’s because Mike Dean comes from the Wirral and cannot officiate Liverpool games because, hilariously, there might be a danger of bias?!
I would also add to this that, having looked at my data, the conclusions would not be much different for Liverpool if we were to sub another Ref in for Dean on their numbers.
This post is merely meant as a platform for further study and an introduction to where we are hoping to go with the numbers… better data is coming as are further studies what will be more targeted at the various shenanigans in the EPL that present themselves as fascinating blips in our data.
DogFace and Bloodhound Zach will return in the summer.
About bias refs and Utd: just seen the last half our of the reserves game. What a biased trio once again. A clear backpass from Utd which an Arsenal player intercepted and then was brought down by the keeper for a clear penalty. And then they gave an offside???!!! on a backpass??????
Jeezes what the fuck is going on with United and the refs….
Are we having Webb on Sunday or will it be Dean?
Looks like foy this weekend. Lee Mason as fourth official. Think Chelsea have marriner. I am sure Webb would have been there if we were still a threat!
Today AW said that of course there are no ‘agendas’ in football– he wondered how he could do his job in such circumstances? What else can he say? But Arsene must know better than anyone that Arsenal are affected by ref bias. Only the thick and the disinterested can fail to see it (that covers most of the media and press then!). Assessors must know. Riley must know. Nothing, it seems is done about it. So if that isn’t an agenda– what is? But the strange thing is– so far– no top ref or ex top ref has broken ranks on it. To me that can only mean one thing– the pressure is coming from above and the threats are heavy. There will probably be Warwick House Rules in operation. This means- once you are let into the coterie, you must never repeat outside, or countermand what has been decided in, that organisation. This applies even if you know what is going on is wrong. Threat of punishments from blackmail and dirty tricks to bankcruptcy and worse have been used. But that might not stop an anonymous insight to what is going on. Plenty of people out there must know. Come on, who is at the bottom of this? Let us know– in confidence of course!!!
Trouble is once a ref does something wrong and I am not talking about an honest mistake here, they have you forever unless you confess all and ruin a lucrative life. That is why the fa and scudamore are so scared of fergie he knows all their dirty little secrets and where the bones are buried
Jeez Walter, are you kidding us???
Offside from a backpass???
Yep C47, the new offside rules when MU is involved…..
Or no, it is a rule that is used when Arsenal is involved. In the 2007-2008 season Adebayor intercepted a back pass and scored at the Emirates in the first 5 minutes. The goal was disallowd for offside also. We then only could draw the game after that horror decision from the ref.
I think it was after that game that Wenger said: ‘We are not blind or stupid’ (or something in that style…)
It was decisions nr. XX that really was going against us game after game.
Great work.
Now I can stop watching or reading about football.
Seriously.
I’ve waited 5 years for this article.
yeah I’m sick of this nonsense it gets so painful I feel sick and not want to watch football! I started watching football after playing pro evo and used arsenal so supported them first game I saw was veira 2005 fa cup and its been lovely since but its sooooooooo fixed!!!!! ahhhhhhhhhhhhh!!
What a huge work kudos!!
Thanks for the excellent analysis. 2 points.
1. Suggest you put this together into a fairly formal letter to head of the Ref assoc Mike Riley (without jokey language and reference to names like Dogface – should be a serious letter). In the letter inform him that there is clear bias against Arsenal and Liverpool and cite the evidence with these referees, highlight that this is available to betting syndicates (ref Asian handicapping, etc) and that unless this is addressed they will be facing a serious betting scandal which will have serious consequences for the EPL, its referees and Mike Riley’s position.
2. Area for further study. “When bookings are made”. An area for obvious bias is to issue similar numbers of bookings to each team (easy way to try and show impartiality), but book the Arsenal players early in the match and the opposition late in the game when it is not of such significance. I see this week after week and is one way to influence the outcome of the game.
Gee, that was something else. What a body of work! It is gratifying to read stats that confirm what my family has been carping about for ages. Well done & thanks.
@jbh – the concern I would have is that Mike Riley has some fairly tasty stats himself and he was put in the position in the PGMOL as Scudamore’s parting shot to maintaining control.
i.e. one can reasonably assume that:
a) He is already well aware of the situation and/or
b) He wouldn’t really give a fuck – as he does not answer to the fans.
Do you recall the game against Manchester United which ended our unbeaten run? It looked pretty fixed to me – but I’m no expert… oh wait – yes I am.
I get that a lot about my name but alas it is the name I was given due to my “dog faced” outlook on the game (that and this is a murky industry and, as I poke it repeatedly with a stick; I do it from a position of valued anonymity) – the jocular nature of the article is in some ways to make it more accessible and also to not depress the reader too much – the only way change will happen in this game is from a combination of public pressure, paradigm shifts in perception and responsible journalism to question the reputation of the brand – this will affect the profit margin of the rapists and looters of our sport… their avarice and arrogance have deafened their ears to everything else.
The fans need to wake up and, in some ways, professional football [in the modern sense of the word] needs to die.
Dogface I disagree. The only way you can put pressure on the likes of Riley is by exposing it widely and by highlighting to him that people are onto their game.
Great if the media can pick up on this, but they have a vested interest as well in showing that the game is “not corrupt”, and you know the appalling track record of most of the media in their anti-Arsenal rantings.
Yes Riley doesn’t answer to the fans but if he sees his position at risk it could have an impact.
The point made by jbh is a good one, book Arsenal players early to minimise their influece on a game.
How did Davies get away with his early fouls last week!
How many times have we seen Song walking a tightrope because of an early booking?
I accept that Song has much improvement to make in his tackling, but I do get a sense that he is booked as early as possible by Refs.
It is one of the things we will look at when we have the new data – usually the DM or CB (or whatever position the opposition have a tasty attcker against) is booked early to take him out of the challenge in a game where the ref is up against us.
It’s already under consideration and jbh is indeed correct. We are a little way off that kind of analysis yet though – but soon!
More comments and suggestions please.
🙂
have we all seen who the ref and the 4th official will be tomorrow? chris foy(yeah the same one who ref`d the fa cup match) and lee mason respectively…mhh i smell something
Anyone here thinking about the effect of yesterday’s refereeing on Spurs’ future?
With a 1-0 win, they are seriously still in with a chance for £20m+ of revenue in the ECL next year.
With two incorrect decisions, one of them scandalous, they are close to being out of it.
It’s not all bias against Arsenal you know.
And if you were looking at refereeing in general you’d realise that…….
Yes Rhys – hence this article pointing that out.
Chelsea and Manchester United are the most favoured with Liverpool and Arsenal being the most peanlised. Did you read this or just skip through it?
Interesting analysis. Great to see a soccer stats blog.
I had a few questions which you could hopefully shed light on:
1- Exactly how many matches did you have for your analysis? (since you mentioned you eliminated a few teams)
2- Classifiers (esp regression based ones) can be affected by the imbalance in class sizes. In your case, I would expect that there are many more non-Arsenal matches than Arsenal ones. How do you correct for that?
3- The question of choosing what terms to keep in the regression model is essentially one of model-selection. How do you ensure your models are not overfitting?
4- The p-value choice of 0.1, as you point out, is indeed a little upsetting (only kidding). Especially since you are testing multiple regression coefficients for significance and would likely need to correct for multiple hypothesis tests. If you have less data, isn’t a more reasonable solution to use another method that can work with the data than relax p-value thresholds (which at 0.05 are already very loose)?
Great stuff, just confirms what I have always suspected. This season, we have been cheated of the league title by Refs.