The problem with Untold Arsenal; moving things on

85 Replies to “The problem with Untold Arsenal; moving things on”

1. hi Laundry…

   Its a nice idea, just that its a bit too elaborative.. Why dont we just start with the top 5-6 teams and have 2 reviews of all the matches played by them.. Since it will be our first year of doing so we should be able to manage these 6 teams first and then we should go for expansion..

   This should also be good for untold as fans of other teams can also read the ref review of their matches.. But to do this we need to have a statndard template for review..

   But this is a great idea and i am definitely willing to contribute for it… Also just imagine a liverpool fan coming to untold for complaining against united.. It would be also be great if we can put more photos and videos to make it more interesting..

2. I’m in complete agreement with Abhishek… perhaps rolling out to Arsenal, Chelsea, Man Utd, Man City, Tottenham, Liverpool next season would be a sufficiently broad first step… so I’m guessing you’d “only” need about £2k? If the project then attracts favourable attention from their fans then you could raise more donations and expand accordingly.

   I’m also willing to contribute a bit of cash – who knows how far these grass roots movements could go

3. Though perhaps it should be launched on a non-Arsenal domain to give it more credibility

4. A new site called Untold Football or Untold Refereeing…

   A first step would be to recruit a good amount of refs; depending on how many you can get, you can decide which matches to review. I think you can get a reasonable number of qualified refs – supporting teams other than Arsenal – without any pecuniary rewards. Surely people at Arsenal are not the only ones fed up with the current level of EPL refs.

5. I’m sorry but

   This comment has been cut. It has been said many times, and it appears in various articles on this site, that our rule is that one comments on the article in question, not on another matter. For clarity it has been said before, and I repeat it here, this means does not mean one can get around this by saying “you are talking about the wrong subject here, the problem is….”

   There are many other sites that deal with different issues, and one of the points of Untold is we take in issues not dealt with elsewhere.

   This has been said so many times on Untold that I begin to wonder about people who write in, and ignore this rule. I can only assume that it is a deliberate ploy to disrupt the site.

   Tony – editor.

6. @vretou
   what has that to do with the article?

7. @vretou – rest assured that your utterly irrelevant comment will soon be deleted… if you wish to put it on another site I would copy/paste it now before it’s too late.

   I think that this data would be something that could be funded by certain interested parties… we would probably have to do a deal with the devil (so to speak) and go cap in hand to the gambling community… mind you – they would probably want to insist on exclusivity or at least a window of exclusivity to protect their investment.

8. Just to clarify on the issue of disruptive articles – it can take a little while to get them down, as I am at work, and sometimes get distracted by that. But they do come down eventually.

   Tony

9. Anyway, back to the main point, I think the ideas in the article are wonderful – but it is an issue of time. Untold works because there is a group of people who all contribute articles on a regular basis. But even with that, and with my only writing occasional pieces, I do have to give over a fair amount of time to the site each day – and when I can’t Walter steps in and does it himself.

   So to get this project going we are going to need someone to administer and edit it. It can be on Untold, or it can be on another site – and I can arrange for the new site to be hosted etc, and promoted through Untold. But there’s still a huge amount of admin to be done.

   I am not trying to knock the idea – I love it, and want it to happen. But I want to make sure we don’t set it up, and then find we can’t sustain it.

10. I’m looking at creating a website that will automate much of the administration and data gathering. W

    alter – could you send me your latest Excel file so I can capture your specification please?

11. Trying to review every single EPL match in full would not only be costly, but you would have difficulty legally obtaining the full 90+ min footage for every game played. Sky provide edited extended highlights only. Why not start by focussing on games televised in their entirety only – regardless of the teams involved? Most of these inevitably involve the top 6, but this would surely be a credible yet viable step forward… with a view to purchasing full access to all the game footage in future.

12. Strongly suggest you start next season only with the top 6 teams.

    1. Quality of review is your key matter. if you try too much with a large number of unproven refs reviewing them all, I think the quality will suffer. Then you immediately lose credibility.
    2. Top 6 teams are the key as that is where the championship is decided and has the most money (Chelsea, MC, etc)
    3. I’d be less concerned with accusations of bias because your reviews are very open. Every incident is referenced, anyone can look at the foul called in the 10th minute because you reference them to the minute and the players involved. Far more open than the PGMOL.

    Must be on a separate website (to gain credibility amongst fans of other teams and the media) and should not have comments, otherwise you will spend most of your time fighting the mindless trolls. Stamping out violent tackles should be an objective together with educating the readers over what the rules actually say (eg against tackling from behind).

13. In terms of getting the games legally – we’d need the club’s equivalent of ATVO which (unless I’m mistaken) will have full games available as downloads like Arsenal TV.

    As for the administration side of things – I’m currently on holiday from Uni until mid September so could step up and take the workload during that period. After that it’s unlikely as I’d be a final year students with numerous other extracurricular activities too.

    In terms of funding the 2.5p average sounds great, especially when you consider the most regular contributors to the site would likely not be fussed at putting a bit more in. I’d happily put £20 in. Also I wouldn’t be fussed about getting paid and I’m sure a couple of others wouldn’t either, so that might knock the total required down a little bit.

    I’m also a qualified ref at the Lancashire FA, so could get in touch with them regarding the refs. In terms of quality control I feel each game would need to be reviewed three times by different referees, which suggests that a smaller review would be worthwhile to begin with, and then expand outwards as funding allows.

    Perhaps particularly controversial moments when a reviewer is unsure could be earmarked for the msot experieinced ref to advise on, further increasing quality. Finally, each reviewer should have his games re-reviewed on a random basis by the senior referee in the team to ensure no un-registered bias and for general quality control.

    Bloody great idea on the whole!

14. This is just amazing! This has to happen to prove what we have been saying all season & really make the FA etc sit up and pay attention. I’m not a ref so can’t help out there but I’m more than happy to throw some cash at this. I also agree with covering televised games and having a seperate website, can I suggest http://www.catchingthecheatingbastardsatit.com

15. 350,000 hits per month, do you have the data on hw many of those are unique visitors??

    Based on users visiting twice daily my guess would be around 5000-6000 maybe, you’re still only looking at £2 a head on average. If 1 in 10 were to donate then you’ll only need £15 per donator. I’d be game.

16. Stuart – if those stats quoted are the ones given by Tony (which he also tells me as a regular contributor) then he calculates it as unique users.

17. This effort would allow various breakouts which have been issues for years for MANY fans, and not only Arsenal. I think a list of “Watches” could also be maintained and be cumulative — namely, “VIDIC Watch” “Webb Watch” “Dowd Watch” “Walton Watch” “Rooney Watch” etc. etc. etc. I would urge UA/others to recommend a list (say a top 5 or 10 “Watches”) which would be publicized. Whenever one or more of those on the “Watch” lists made a pivotal play – that is a pivotal call or non-call that seriously impacted a match – then it would go on the list. The criteria of a pivotal call or non-call should not be that hard to define. Keeping such a list comes out of the process and other people and patterns may emerge for the watch list. It is not only Arsene who have been done hard by these blokes in the past. And even if that were so, the data/reports/eyes on them would have been culled not only by UA, but the other participant-referee-observers. There is a huge upside to this: once it becomes known that they are being monitored – who will howl, as will SAF or his press minions, which will show that this matters and has traction! – then it may have a chilling effect on their blatancy and better in itself re-balance some or important matches. They say a watch kettle doesn’t boil; or, less folsky, Heisinger demonstrated how an observer alters the perceived behavior of particles. No matter what the outcome, the fact that an organized set of impartial eyes are on them, and issue a public report (unlike the PGMOL) will have consequences in and of itself. Maybe – just as an example – it would pressure, for one example, PGMOL to make its post-match referee reports public knowledge. I doubt it, but they’d likely try to do something to make themselves “appear respectable.” That is, if/as our efforts get more public attention and they start to lose their grip in the “court of public opinion,” then their product is less credible and they have to be concerned about escalating public demands for fair results – video replay, radio on the ref, etc. – and perhaps institute one or more to try and take the steam out of the (hopefully) snowballing protest. It doesn’t require having 51% of the football public on-side; only just enough to raise their eyebrows as it raises our pulses. The excitement around efforts like this will get a LOT of attention (and brickbats); and, as the Iroquois Indians say, we will need “skin as tough as the bark of a pine tree” to stay the course. So, methinks, as long as UA continues doing what it does so well – maintain this case study, even as it feeds into the larger effort – then we should have some Significant good results (even as unintended consequences). Doing nothing, as all recognize, will only be more of the same – that is, NO significant good results (to put it mildly). Anyway, I propose all the above and to please consider putting forth ideas for Watch Lists that would be culled from the same data, and provide rewards and discussion and possibly good behavior by those being listed – those who have unduly benefited, many believe, from calls and non-calls in the past. It can only help and won’t divert energies as it would flow from within the same effort (and, to my lights, should have been underway long ago). Cheers, mates!

18. A thought for the sake of sustainability and legitimacy: I don’t think any one supporter of this project should contribute more than 20 quid or so, each; or the project could potentially be de-railed by that person’s withdrawing support. The cost, whether it’s 7600 in total, or whatever it is announced to be, should be widely enough spread out, so that it has relatively and reasonably broad support – and could not be dismissed as merely one or a few people’s vanity project (such as the FA, Chelski, or, well, you know…) Anyway, let’s agree on a minimum and a maximum and do count me in.

19. logical fallacy of defective induction(argmentum ad verecundiam) makes whole project useless…nobody with a brain will take it seriously. Dogface so excellent with numbers yet base the whole project on defective system? no good. It will be the brilliant idea if you only focus on objective calls from officials i.e) ball crosses line, time wasting, offside etc. otherwise no good because of flaws. If it is done in the properly way i will be happy to donate my share in the name of research… my email address is with the editor….nasty arrogant people can make fun of me now.

20. @Zebediah: please spell out in detail “the proper way,” “objective calls” and what you mean by “flaws” so that you actually add something of value that can be acted on or incorporated into the proposed analysis. And, by the way, since you are so focused on counting objectively, do non-calls where official rules apply count as referee errors? What’s your position on non-calls? Please be specific.

21. @zebediah: one more thing. Are off-sides, under the current “rules” countable in a purely “objective” way, other than referee interpretation. Is there room, in your view, for a “flawed” referee interpretation of an offside call – that is, where another set of referees would be reviewing that call? What is your position on referee’s offside calls? Please be specific.

22. the flaw is of very basic logic..i think no need to explain…non-calls/calls are subjective so no 2 referees will call them the same way.Unless of course the ball crosses the line/offside/timewasting is not called. What is yellow card for 1 =red card for another…bias in this way is impossible to prove. You could have 1000 refs on ur books that disagree against a descision of a match official and it still would not prove anything..I am sorry but if untold wants to play with the big boys then they must not make such basic logical errors. @Bob I AM BEING CONSTRUCTIVE just because I dont drop my pants and bend over for the authors like you… does not mean that I am a negative nancy…If they want to waste money on a stupid childish system then fine but I pointed out why it would never be taken seriously.

23. You say please be specific a lot. You must like these words. Yes for the “interfering/non interference” with play rule as it is subjective it must be ignored.

24. @bob

    The idea of ‘Vidic watch’, ‘Rooney watch’ etc goes completely against the whole point of the article – i.e. objective non-Arsenal centric analysis.
    Unless you’d be happy for ‘Fabregas watch’ or ‘van Persie watch’ just to balance things up?

25. @zebediah: You say, “bend over” “drop my pants” “nancy,” you say. Well, dear boy, would you say those are “constructive” ways of adding to the conversation? Would you look in the mirror next time you are being constructive? The fact is you would rule “non-calls” out of bounds, and you pretend that “offside” is a clear call when all on-the-pitch evidence shows there is a LOT of problems with the new way the rule has been applied and called this season. So, dear boy, what is left to attend to? Whether the ball crossed the line (as your comments do)? Well for that, we’d need video replay and an umpire in charge of it to overrule a bad call. To be specific (yes I love and value being specific), would you call for video replay, for one instance? Are there any other specific reforms you would back? Please be constructive.

26. @BobbyP: yes, I’d be happy with any of those Watches. That’s my point. Let’s have a comparative watch list and see what the calls and non-calls for any select group of players would yield. Excellent idea, mate. And, by the way, Webb and Vidic and Rooney are WIDELY seen as favorite sons. That is not a mere Arsenal-centric perspective, though of course Arsenal/UA have consistently felt hard done by them. Some of the Arsenal views are in fact widely-held and perceived and it does not help to censor them as if UA/Arsenal fans are so insulated from reality and somehow paranoid or conspiratorial for holding those views. So, to your objection, I’d say, yes: let’s add Cesc and whoever to a Watch List of 5-10, for balance as you define it. I’m totally in favor of that and convinced it would be productive and even more effective at the end of the day for including some of our own in that list. That is fair and balanced and the point is the truth that leads to a fair pitch for all.

27. @ Zebidiah
    The basis of refs reviews will be according to Walters method. I would hope that Walter would want to play big part in the project. The remit to reviewers would be applying the laws of the game, the laws are in black and white, most are not a matter of interpretation.

    Regardless over a period of time and with neutral reviewers we would expect to see no bias, surely poor decisions and failure to apply would go one way then the other. The larger the sample, the more likely no bias will occur.

28. @BobbyP, Laundry Ender, all: I would love to test, quantitatively, whether the non-calls on fouls against Cesc would be greater or lesser than the non-calls on fouls by Vidic. And, qualitatively speaking, their impact on the course of those matches. That’s, now, an intuition based on however many seasons of observation. The proposed methodology here could help verify or help dismiss that perception as a general rule. It still would not dismiss the impact of such calls in any specific match, as all matches do not have the same weight (which is something I think that this new initiative would need, somehow, to factor in). How, for example, would a bad call in a mid-table match that didn’t impact the EPL championship result count exactly the same as a bad call in a season-defining match with massive implication for the outcome of the league? That is, wouldn’t there need to be some kind of weighting? This is not to criticize the effort, but to raise an honest question as I am not expert in statistical weighting, whereas it seems that this needs to be taken into account somehow. And, btw, that is one reason I love and applaud the case study that UA continues to do, as there is a marriage of quantitative and qualitative analysis to arrive at a fact-based interpretation. Something to consider at least…

29. I think this is such a great idea (although I’d be surprised if the Professor or anyone else in the Marble Halls hadn’t already thought of it).

    I kind of agree with Zebediah, insofar as the methodology has to be focused on those aspects of the laws of the game that are OBJECTIVE in nature – offside or not, penalty or not, crossed the line or not.

    However, the methodology should also take into account those aspects of the game that are SUBJECTIVE in nature – those aspects of the rules where the referee is entitled to his (or her) judgement – yellow card or not, yellow card or red card, time wasting or not, injury time, or not. etc etc.

    Probably the answer is a solution that incorporates both categories, with either separate scoring for each, or a combined scoring that employs some degree of weighting.

    Before the project ‘goes live’, I would suggest that whoever is going to take ownership of it put together a manifesto of sorts, that (a) details the methodology, and (b) explains the scoring system, and then publish it. Then:

    a) Invite criticism and improvements from all corners
    b) Incorporate changes into the methodology and the scorecard based on the above

    ..so that both the methodology – when adopted – and the results – when published – are as robust and ‘bullet-proof’ as could possibly be.

    Physics of time and geography prohibit me from participating in the project, but it’s the best idea to come along in a long, long time, so will gladly chip in.

    With respect to one earlier comment, I don’t think there needs to be a cap on individual $$ contributions. Just so long as ‘investors’ understand that they cannot influence due process, regardless of the amount of their contribution.

    Whatever gets this thing up and running ASAP!

30. The sponsorship process might benefit from a register. You can bank my £20 already, but I’d like the opportunity to show my name in the list of original sponsors, even if only in an obscure corner of the website. (and I agree that an independent website is a must to attract a wide sponsorship.)

    My (limited) experience of these things makes me think that uptake levels anywhere near 1% of hits would be regarded in the industry as a miracle.

31. @toby: great stuff! I want to chime in that both interpretive (qualitative) and quantitative aspects are needed. There is not purely objective data that speaks for itself, without interpretation. I agree totally with the need for weighting in different ways (see my last post above) and that the weighting needs to be made plain and transparent and understandable to readers. Your advocating input and critique of the proposed methodology is really welcome. I proposed a cap on contributions so there would be no high-roller bias or wrecking by pulling money out. But, as you say, it’s better that it gets going and, as long as the whole effort doesn’t cost too much, someone’s pulling out could be dealt with by the rest. I just didn’t want to be naive in a belief that only angels would back this effort. Anyway, my cheers for your excellent points.

32. @ bob, @ laundry ender:

    This project will success or fail by how it is designed. If people are bought into its design, then not only will those involved in it on a day to day basis enjoy participating, but even if a handful of $$ contributors pull out, others should be willing to take their place.

    Do not underestimate the design process of this project. If we invest the proper amount of time, and do the right amount of due diligence, then it will succeed.

    If we short-change the design process, and come up with something that is flawed, or ridiculed, then it’s all just a waste of time.

    I would also suggest that it is optimistic to believe that the project will be ‘perfect’ right out of the gate. Expect some tweaks to the methodology and the scoring system.

    I’d recommend spending the 2011 summer dedicated to the design / infrastructure process, and the 2011/12 season as the ‘beta’ or test version, tweaking it in the summer of 2012 for the 2012/season.

    This is not something to rush into. It is something to DO RIGHT.

33. Zebidiah
    You are missing the point, or don’t understand. Walter started these reviews using the methodology applied for Ref reviews (formally in the Belgium league and presumably in other leagues like the EPL). So these are always done, he is just making the findings public. You seem to not want this…. I wonder why?
    The fact that his findings are very detailed down to the minute, etc means there is complete openness. If you find that the foul in the 12th minute should not have been a foul then fine, you can personally adjust the numbers in your head and see what a revised result is. In fact you could go through the whole game and criticise any number of decisions, but I doubt you will find that many controversial decisions. If you had read any of his reviews (have you?) you will have noticed that he often says “If foul x on y minutes is a yellow card, then definitely foul z at zz minutes should also have been a foul”. This is the way the formal ref reviewers work. If it is good for them, why not us?

34. … should also have been a yellow card”

35. @toby, laundry ender: agreed, the design needs to be careful and a good front end is essential for the rest. That said, I think that while this effort proceeds, there is no time to waste in bringing pressures for specific practical reform based on what are eyes show us on a match to match basis. This, both before and during next season. Walter is soon to make his report public and that could well be the basis for further analysis and practical suggestions and raising public awareness of the PROBLEM, that there is a problem – some combination of referee incompetence and possibly (if not likely) worse (choose your noun). If everyone waits for two seasons – not that I’m saying you are proposing this – then no pressure can be focused and mounted as SAF (Don Fergus, to me) prepares at the 19th banquet for the 20th feast of feasts. If he earns it, he deserves it. But there is the evidence too of what is seen, match to match, season by season, and what is unseen (by refs on the pitch) that needs to be brought to bear. As this effort gets refined and launched, there is also the urgency of now.

36. Walter
    In your season review the key things I am looking for are:

    1. Overall % of wrong decisions (this indicates general competency and or bias)
    2. % of wrong decisions that favour Arsenal, % of wrong decisions that favour the opposition (do the decisions even out (I doubt it))
    3. Number of goal scoring and game changing wrong decisions (eg non penalty (Ramsey), non goal (Chamakh) in the AV home game.

    thank you

37. For future plans. Please include in your analysis.
    Timewasting
    Opta provide numbers of passes for each side and use this as the basis for “% of possession”. I think this is correct and the fairest approach. Most stat providers also use this.
    Others use time of possession. The actual time in mins/secs that the ball is with a team (this includes the time they take to take a throw in, goal kick, etc.

    The difference between these 2 figures, in percentage terms then applied to time gives a fair indication of time wasting (eg ever been surprised how the possession time says 50/50 but number of passes possession is 60/40)
    Another approach is to actually count the seconds a team takes to make a goal kick, or throw in and sum them all (but this is very time consuming).

38. The trouble with counting the seconds is that Sky producers get bored and change to a different shot, so you never know for sure when the cheating keeper actually plays the ball. I think the possession/passes ratio is interesting but it is more a test of cheating teams than bad refs. If they go unpunished (as is usual), then then a refs’ ratio of 0/5 is the same ratio as 0/50

39. Hi Zebidiah,

    I agree with some of what you say and I am of the opinion that matches are placed in the context of consistancy i.e the ref allows a physical game so we then draw the line as to what is/isn’t a foul and go by that. It is a tough call. Or do we take the letter of the law approach and get referees to apply only that – I would say that way we would get more consistant numbers and guage the style/consistancy of the referee [on the pitch] rather than the referee doing the marking.

    These comments are useful – so thankyou for contributing… it’s something to think about and I will discuss with Walter.

    This is a work in progress so all constructive critisim is taken on board.

    One way to solve this would be to take the average of more than 1 referee i.e. 3 seperate refs reviewing the game – which might be possible if we cut down to the big 6?

40. Also: in addition to Walter’s report, we are expecting/hoping for in the near future, at least two video compilations that back up some of the significant/key (as jbh puts it) goal-scoring and game-changing wrong decisions. One if Walter can fashion it, as he’d hoped for; another from an Indian Gooner of a few days ago (- I didn’t note his name). So, there will be lots on our plates to digest, and more. It’s refreshing to meet the pain of this season in such proactive ways that UA/readers here continue to consider.

41. @ Bob –

    I agree with you – the findings / results from this work should be ‘actionable’ in as timely a manner as possible. But I think it is optimistic the FA or anyone else for that matter would take the findings and then immediately turnaround and say, “Oh, so perhaps we should get someone other than Webb to referee the next week’s United vs. Chelsea match at OT.”

    What will actually happen is:

    a) Findings / scores are published

    b) Methodology will be challenged, and then

    c) Authors’ credibility will be challenged, and then maybe

    d) FA (or other) will decide to do their own investigation, which may then lead to

    e) FA (or other) doing their own study, which may then lead to

    f) recommendations for changes to be adopted 2-3 years hence

    We see how slow these organizations move to adopting changes – even goal-line technology is something that people are talking about for the 2013/14 season. Why not next season, for moses’ sake?

    So, to expect results to be published on Friday and changes to be adopted on Monday…it’s just not going to happen.

    This is a 3-4 year investment and to expect results any earlier is fooling ourselves I’m afraid.

42. Also – the data capture is only the first step – and we will need to make sure we capture all relevant data.

    The analysis of this data (as to what can or can’t be written off as subjective) is way off down the road – but given that we will be using qualified referee’s as expert witnesses; the foul/card data may be very useful (more foul than card I suspect) to highlight areas of interest (through statistical analysis) rather than provide water-tight evidence to nail to the door of the FA.

43. @dogface, Laundry Ender: I hope that the referees that volunteer/are recruited will have demonstrable reputations for being straight. To me that is also essential and should be transparent, or the study would be open to challenge, and research ethics require it. It is not a given that volunteering for a worthwhile project means that one’s motives are on behalf of objectivity. Is this a hard principle to try and ensure? Yes, of course. Is it worth trying to vet those whose input is so essential to a good result? Yes, again. So, please give this some thought as this proceeds. Transparency always.

44. @toby: yes, what you outline will be the way the FA postpones action on a sincere, sober and conclusive report. So, while this goes on on one track, there is a need to try and level the pitch on other fronts. It can’t be a process of waiting for the needed report only. There already are commonsense understandings and knowledge, report or not, that video replay, radios on the refs, one-day pre-match ref choice, post-match ref press conference, publication of the PGMOL official ref report – all reasonable reforms – that enough fans from most teams (I do not say all) could press for. And a basis for these entirely reasonable (dare I say) demands, could well be Walter’s print and his and others’ companion videos. These are actionable. Would you be on-side with this, or is a not-as-yet designed report that may be attended to by the FA 3-4 years down the road the only actionable strategy. I’m not saying you’re saying this, but I’d welcome you’re further thinking.

45. p.s. sorry, meant “Walter’s print report”

46. @ bob –

    I am totally on board with getting ‘the truth’ out there in a timely a manner as possible, but advise on caution as this project proceeds.

    I am also pragmatic in that haste makes waste: unless this is set up right from the get-go, the messenger will be shot, and the message will be lost.

47. @toby: yes. but that’s one track. there is no waste to demanding any of the list of practical reforms that I and many others long before me have argued for. there’s no need for delaying anything that guarantees a level playing field for all. not “pragmatic” not “haste makes waste” Yes, the method needs to be set up right. but the method and its report cannot be the ONLY game in town or you have put out the fire that people feel on behalf of reform. Do you honestly feel that asking for video replay and the rest need the well-designed report to be a legitimate request/demand? If you think it is illegitimate, and should be put on hold until this report is ready, then you are on a track that plays directly into the hands of the very delaying-tactics that you rightly outline above. The point is to have more than one activity go on at the same time and to give people that pleasure in participating (as you wisely called for earlier). I hate to say it, but this isn’t really a tea party. Are you saying that ONLY this report is the way ahead? If you are, you’ve made your argument plain. But I’d welcome knowing if that’s your point of view – only work on the report and nothing else?

48. No – you and I are on the same page. It should be an assault on all fronts, but we should just be careful not to shoot ourselves in the foot.

49. To win the support of not just neutrals but of other clubs’ fans, I think it will be necessary to focus on referees’ inconsistency and incompetence. If instead the data is presented as proof of deliberate bias (i.e. corruption – let’s not pretend there is a significant difference), it will turn people off, and even worse, it will allow the people facing criticism to just call it conspiracy theories. No amount of data can ever prove corruption to a criminal standard, and in the absence of proof they will just say “lies, dammed lies and statistics”.

    The intention is surely to create such a momentum that all football fans come to recognise that the standard of EFL refereeing is unacceptably bad, and that something must be done. I think that far more people will reach that conclusion far more quickly if the data is presented as demonstrating that referees are ‘unacceptably incompetent’ rather than ‘unacceptably corrupt’. Ridicule the refs, or they will ridicule you.

50. One day not near to a computer and a very important and great article by Laundryender and lots of great comments to make it even more complete.

    I just would like to add a few thoughts.

    1) Recruting refs: we could ask the refs a few questions like which team they support/hate (almost as important!) -We need also to be sure that they are willing to use the law book as the instructions say it should be used! (but this would cancel for a bit if you read 3)
    2) Protecting refs: we should make it clear that the refs who will participate have the right to be unknown. You know my name but for me it is easy as I am not in England and the FA or any other organisation in England cannot touch me in any way. But I know how the football world works in Belgium and if I would be doing something like this in Belgium it could mean that I would be thrown out as a ref. I don’t have any idea if this would be the same in England to be honest but we cannot take any risk. Because the thing we want to start can bring the game in a bad daylight and then the people in charge will be looking for easy victims. To throw refs out who work on this would be a way in trying to shut us up.
    So I think we should make it sure that we don’t use the real names but should use names like refreviewer01, refreviewer02,…..
    Only the people who do the administration of the site will know who this refreviewer01 is.

    3) I was thinking that if we could do the top 6 each week this would mean that roughly calculated we would do half of the games in the EPL. This would meant that at the end of the season we woud have 38 games of each top 6 team and 19 games of each other 14 teams. And it would mean that we would only be doing 5 or 6 games each weekend. And could work wit fewer people. But if we could find let us say 20 refs who are willing to work along we could do each game by 2 refs and we could have an average score which should reduces bias even more?

    4) Dogface I had my review scheme ready for next season but some remarks have made me wonder if I could still implement them further. So please wait a few days/weeks before I will send you the empty schedule I use.

    For the rest: not enough time for the moment… damned…. Will try to get back to this.

51. Tram, I think you are right about this.

    We should take care that our message is about the low level of the referees in the EPL. And this message should be well made public at the start.

    We could mention my ref review in which I can show that the level of the refs in just Arsenal games was poor. And then we can say that we want to examine this further and see if this is the same for the other games.

    Let us at the start not say: we want to prove that the refs are biased (against or for a team). No the message should be: we think the refs are not up to it, we want to examine this and then come to the conclusion which can be: oh, blimey the refs are good enough or they are not good enough and let the people see where and when they have failed.

52. ….And if the numbers show a certain bias against or for a certain or more teams we can show them the numbers and then we could talk about possible something corrupt

53. Great article, and a lot of great suggestions.

    I agree with those saying that it would be wiser to begin with reviewing just the ‘top six’ matches. The focus should be on quality rather than quantity, if the quality of the material is there people will flock to it and recruiting more referees to become assessors should become easy. Perfecting the methodology is far more important than covering every game, at least for the time being. Also, even by only covering the top six, every other team will still see reviews of 12 of their 38 league games, almost a third of the season. That should still be plenty of interesting material to attract non-top-six fans.

    What I believe is really, REALLY crucial is that the whole process is as impartial as possible. I understand Bob’s request for something like “Vidic watch” (I would be interested in that analysis myself) but to include such a thing on the Untold Referees website would only show its contributors to have pre-conceived ideas about certain players/clubs/referees and possibly a hidden agenda. The perception of a responsible and impartial process would be severely damaged.

    If done properly, I could see the website carrying this information becoming not just something that football fans will take a strong interest in and hopefully become better informed and educated about referees and the laws of the game, but a precious resource that many a blogger and professional journalist may use in their work, therefore spreading the good word for you! But for that to happen the method must be rock-solid and impartial, or the findings will not be seen as sufficiently trustworthy for the word to spread, for others to use your findings for their own analysis and link back. And that’s precisely how Untold Refs can drive a huge change in public opinion.

54. @Walter: Would you have data on hand to consider – say for sometime after your initial report is issued – taken on a comparison of calls made/not made re. Vidic (SAF’s captain)and Rooney versus Cesc and RvP and see where it gets us? I think it would be useful – if the numbers prove out – in showing bias (incompetence or worse) regarding ManUre’s and our best players, this season. If you had access to more of MU’s matches than only our ties, that would of course be more than welcome. I think that we could see how refs like Webb and others have treated these four this season. I know there were injuries that may well have knocked them out of various competitions; but it might yield something important. I’m sorry to add to the pile, but perhaps the summer will allow time for this, or perhaps a better framed request than this one is at the moment. To me, such an exercise could achieve either just a one-off result, or a model for a Vidic Watch, Cesc Watch, Rooney Watch, RvP Watch, Webb Watch (as the top ref), and any others deemed worth following. My argument is that these Watches could in themselves be of short term help to fans and inevitably for the media to consider (openly or under the covers), and which also might percolate to the FA/PGMOL and, with evaluative eyes on them, might help level (or at least reduce the number or blatant bad calls on) the pitch. And just as an afterthought, I wonder how many of Cesc’s injuries can reasonably be attributed to a cumulative result of X number of non-called fouls (for which there is a count in your reviews). Anyway, I think you’ve developed a trove of important data this season that could provide greater insight into several questions that have mattered this season and will matter next season as well.

55. Untold Refs could be the resource that empowers dozens, maybe hundreds of bloggers and journalists who have been itching to speak about the refs, to finally broach the subject with confidence.

56. @Wrenny, Walter: yes, a Vidic Watch, etc., with comparisons to our players, would be for Untold Arsenal – but not Untold Refs. There’s many a good reason to continue some of the more far reaching or controversial points on UA – wouldn’t you say?

57. Bob, for this season a lot of your questions cannot be answered from the datat which I gathered from the reviews.

    But I was thinking of streamlining the data thanks to your suggestion. But it could go at the cost of the comments I usually give.
    I could make it 2 columns saying foul from XXX on YYY and then add if the ref gave it correct or not. Maybe a more clever guy then I can make some kind of instruction at the end of the game that could say Player XXX committed 10 fouls and the ref has punished 6 of them and 4 not.
    So we could maybe at a foul statistic at the end of each game with this data on each player that was involved in a foul given or not.

    This sounds like a great idea and at the end of a season we could come to some kind of “Vidic-watch” or “whoever player from the top 6 teams-watch”

    I will see if I can get this implemented in my final ref review of our game against Fulham… Please give me another day I would say….

58. @wrenny, walter: Also, the Vidic Watch, Webb Watch, etc., could be done in an ongoing way on UA as the next season proceeds. The Ref Watch is an admittedly long-term (1-3 years, depending, as toby indicated) project and could provide the reference point that we desire. The selected (and comparative) Player/Ref Watch (Vidic, Cesc, Webb, other refs) on UA would be an immediately available review and cumulative reference point from start to finish of the next season. I’d like our information to be brought to bear; as much as I couldn’t bear witness to the Fergie Feast of a 20th glorious championship. At UA, the passion of the case study, with good reasoning and immediate data; at Untold Ref, the cumulative process of expanding data-gathering for the broader public marketplace.

59. @Walter: yes, I really like the player-count. I’d also add that you could reserve a qualitative comment for when a specific calls/non-call on a player was a game-changer – the special call that clinches the match or clearly turns the tide. This would be a call in context – a qualitative call. That’s what I’ve meant by the marriage of quantitative and qualitative analysis. Together they take us far into the truth of the match and more, into hearts of darkness and light, so to speak, methinks.

60. @Bob
    Yes absolutely, I think those analyses should continue, but obviously on Untold Arsenal. To have such things on Untold Refs, in my opinion, would be to kill the site on arrival for the reasons I explained in my post – hidden agenda, preconceptions, etc. It would never garner the support it requires for it to grow and be accepted into the mainstream, to become a trusted resource for bloggers, the statisticians, the journalists and bring the issue to a huge number of people.

    @Walter
    I would think that this would be an aim of the site, to then build a database of stats on each individual PL referee. Just like with players we can find their games played, goals, assists, yellow and red cards, etc., the same could be done for the refs, bringing together the work of the reviews and Dogface into a deep set of statistics to be probed and examined by all.

61. Laundry Ender, thanks for a great post.

    The Calciopoli scandal is something I can’t comment on, as I don’t know much about it. According to reports, it was very difficult to accumalate the evidence that they had/used.

    My thought is, have we been witnessing variable tactics over the last few seasons?
    e.g. pelanty embargo last season.
    This season, plenty of pelanties given against for, say, falling over and having someone kick the ball against your hand (um, ‘hand to ball’?)
    I guess I am stating the obvious. That people involved in such antics are clever.
    But then, are they?

    I’m still waiting for the fake sheikh to do a story on the cricket spot fixer’s associations with a non-league football club. Why a non-league club? Questions, questions…

    When even Arsenal fans, who like to sing about signings and money and stuff, sing about a cheating referee in a game that Arsenal are winning (Everton Home), what does that mean?
    Well, I guess it could mean that Home support in such circumstances matters. A lot.

    In my humble opnion, it also means that they have ‘lost the plot’ a little, and are not very subtle.

62. I’ve had the pleasure of reading these posts all day and I have to say, some of the people on this site are fearsomely intelligent, the fa will be quaking in their boots when they get wind of this (and they will). One thing I will say and feel free to knock me down but, things are getting very in depth and potentially complicated, if there’s one thing I’ve learned is to KISS (keep it simple stupid)… Over to you 😉

63. Great Idea, power to the people!

    Examples of double standards are much harder to defend than individualdecisions, and I bet the games are rife with these, Especially ones for manu / arse ( seperately) reffed by Webb. Could that be why we didn’t get him much last season??

64. the truth is that there are other fine blogs out there but none match Untold Arsenal in the blogosphere. Thank you Laundry Ender for this great post and kudos to the Walter and Tony and the rest of the team for starting this. I’d like to say that a though i agree with the suggested idea of Player Watch. I see the point being made about conveying perceived bias when particular players from say the Top 6 teams are formed. My suggestion would be to make the watch system a sort of ‘Watch league table’ with the most favored player rising to the top and least moving down Using data assembled from the ‘Untold Referees’ ongoing research. This would make it a more streamlined process. So that if player xxx has been adjudged to commit a foul and escapes punishment then he gets a certain (weighted) value of points. anyway, this is a fantastic idea and we owe it to the game we all obviously love to make the changes we can make.

65. @obichibz: the logistics of the data sharing would have to be worked out, and plans made to syn up the data. But I think that the UA data compilations on Player Watch would likely be underway before the league-wide effort gets underway. Either way, I think your idea of a league-wide “Most Favored Player Table” is brilliant. Because of the star-power and celebrity that top players command, it would likely grab lots of public/media attention and probably have a positive impact.

66. I would think that some interesting player statistics would shake out of the data – assuming that the player (relating to whatever incident) was recorded as part of the dataset. this can be done in an intuative way that will be simple to use – we need a clean interface with incident types and a DB of player names, etc. What I really need to think about is this kind of standardisation of the data ant take it to walter.

67. I was wondering if it would be possible to bring in more non-British assessors for this project as getting the games abroad is easier (3pm kick offs)and they might actually highlight the tendency for British refs to “let things go”, As we know the advantage rule is used inconsistently over here and leads to bad blood on the pitch. Another point i would like to make is with regards to off-side, this is not solely the refs decision so should be omitted from the refs final score and maybe looked at separately, Where as playing advantage and stopping play tells you so much about the way in which a ref likes to control a game or disrupt a game. Having the linesmen’s decisions in the mix only distorts the refs score.

68. @dogface, walter: One interesting stat that might emerge from player-watch compilations is whether or not the oft-stated perception of anti-foreigner bias is actually being enacted on the pitch, and perhaps some correlations with which referees are making those calls/non-calls. A potentially very rich vein to tap. As you say, there’s more that player data could yield…

69. See sirs, you can start with top four teams of the current league say Manchester United, Chelsea, Manchester City and Arsenal and also try doing something with the other two teams which invariably affects the results.

    Please understand its always these top four team results that affect the one standing on top and one in the middle. I have not seen a Liverpool vs Tottenham match result really affecting the desired result at top. So it is this top 4.

    Also when you are talking for top 4 teams in reality you are doing 4 x 38 = 152 matches which is roughly 40% of the matches and can give you a fair view of the proceedings.

    I can only pinpoint in one single incident that will make people understand the clout Manchester United carries. EPL is telecast in India by the ESPNSTAR group. This sunday the only EPL game that was shown was the Manchester United vs Blackpool match in ESPN. Star showed French Open first round.

70. While it will be interesting to have PlayerWatches and RefWatches – only on UA for reasons mentioned above – care needs to be taken not to be overambitious. Don’t bite off more than you can chew.
    For a start we might want to limit our research to writing the ref reports for matches across the Premier League without drawing any overall conclusions before the season is over. That way you will prevent the often touted remark: “It will even itself out over the season.” First gather all the required data and then draw your conclusions. It is how academics works. You have a preconceived idea which validates your study: refs in the EPL are of a low standard. Then you adjust your idea according to your findings throughout your research: refs in the EPL are very poor. But only once all the relevant data has been obtainedyou you form a definite conclusion: refs in the EPL are biased.
    The matches under scrutiny can be all the EPL matches or limited to only the matches of the top six clubs of the 2010-11 season. This depends on how big a ref pool we can have and how many refs are assigned to review each match.

    With reference to Zebidiah’s comment: in fact it does not make a difference whether a decision is objective or subjective. Of course ref A can deem a certain foul worthy of a red card where ten other refs would only give a yellow, but as long as ref A is consistent in his decisionmaking during that match – and hopefully over the whole season – than there is little problem in that. I would no mind (much) if the ref lets a player from team A score a goal while being ten yards offside, as long as he does not blow his whistle if a player from team B ten yards offside and scores. Consistency is the key. This is something Walter has been hammering on all season long.
    Likewise, the refs for Untold Referees need to show the same kind of consistency. I suppose Walter can be in charge of quality control – if he can find time next to doing his reviews and articles, although the readership will keep a close eye on every ref as has been done with Walter all season.

    If done well then we will have something big in our hands and at some point the media will notice. Once they do the floodgate opens.

71. Wow. What a debate and some great suggestions following a fantastic article. I’d love to see it come to reality and it is encouraging that we are looking to move on and take this whole thing a step further. Who knows, we might even scare the refs into doing the right thing.

    I liked bob’s idea of Vidic Watch etc, but I too think the way to do it is to just list all fouls called or not called with the players fouling and fouled against. That data can be used to draw conclusions on any player. (Apart from Vidic, who gets away with awful fouls, what of Rio, who has played less matches but has given up only 3 fouls?)

    @Prasanna Veeraraghavan
    I agree with your overall point completely, about ManU being the brand name that the EPL uses to further their own brand, but Star Sports showed the ManCity-Bolton game live, followed by a telecast of our match at Fulham.

72. This is an excellent idea and a wonderful discussion.

    I have a slightly different way of looking at this.

    Money, IMO, is not the main issue here. We have some qualified referees on this forum apart from Walter. And we can be fairly sure there will be similar refs on forums of other clubs as well. I don’t think anyone who contributes here will ask for money to do reviews (I might be wrong). Similarly it should not be hard to find refs supporting other clubs who will be happy to do the reviews.

    As others have mentioned the structure is more important. It has to come across as a high quality, respectable, credible, and unbiased site.

    I believe this can be achieved by giving the refs on board the authority to question each others work.

    Suppose there are two refs from each of the top 6 clubs who volunteer. Now each one gets to do the game for one team (it could be organized in a manner that each ref does not do reviews for one team that he supports and one that he hates – there can be more hatred involved but the system should cover for that) This way there will be two official reviews of each game. Some contradictions will automatically come out. In addition, the refs supporting a team, who cannot officially review those games, will have the right to question the reviewers about certain decisions.

    This will bring the subjective element out and we can clearly see a discussion developing on judgement calls. Only the refs on the panel should be allowed to participate in this process to keep the clutter/noise out. If the decision is still dubious after contributions from multiple refs, readers can make a lot more informed choice for themselves.

    In effect, we could have an open group discussion amongst highly qualified people who aren’t discussing to prove a point but to achieve a common goal. The data (match videos) they use for the discussion would be available to most readers. So there will be a very open and honest debate on subjective issues and concrete, reliable analysis on other calls. Self-policing (so to speak) and a commitment to objective, unbiased, credible analysis can lead to incredible quality.

    It is important that the mission of this website be clearly stated. It should be about honestly assessing the performance of referees in the games involving the top 6 in the Premiership and should not include any theories that Untold has already developed. The theories must evolve from the work done on this website.

    I believe if the work has high quality and credibility, people will support it and raising money to expand it will not be an issue.

    I am sure there are enough techies in the football community who will help create the website and those costs should not be too high either. hosting, etc. might need some money initially but over time ads and donations should cover it as well.

    The key is in finding the right people, developing a credible and easy to understand system, and in ensuring quality and lack of bias.

    It can be done and I will be glad to contribute time and/or money. Someone in England will probably have to take charge. Finding the refs who are willing to contribute would be the first major step. In parallel the structure can be developed through discussions on this forum.

73. Count me in for Ref Watch- lets start with the top 6 clubs or 6 games per week.Refs of different nationalities ,preferably those recently retired,should be considered.
    To avoid bias ,each game could be seperately reviewed by 3 different refs,and their reports can be given a thumbs up/down
    by the readers of the site.

74. Top article, great ideas and I think this could really be a start for something that will truly make a change.

    Just a reply to Jebadaya regarding the subjective vs. objective decisions – I think that it’s not really necessary to make the distinction, for several reasons:

    1) Excercising discretion can almost always be regarded as subjective. Let’s take card as an example. Did a foul merit a yellow card, or non, or a read card. It’s true that sometimes the situation and circumstances are fluid, and the decision can depend on the nature of ref and the stage of the game is at and whether it was the first or fifth foul the player commited, etc. I think that in those cases, and Walter will correct me if I’m wrong, as long as the decision was totally irreasonable, the ref will get a positive score even if Walter (or any other reviewer) might have decided differently. That is: the reviewer is not replacing his discretion with the discretion of the ref, and as long the decision lies within some sort of “area of reasonbility” – it will count as good decision. BUT when a decsion NOT to issue a card makes no sense, and you’d prboably have to work very hard to find ANY ref who’d decided the same – then yes, that decision must count, even if it’s classified as “subjective”.

    2) getting objective decisions wrong can actualy be attributed to pure mistake rather than bias, so I don’t even think that in themselves are more signifcant than those “subjective” decisions.

75. @Dogface I think that consistency and style of refs would be very difficult to define from a research point of view. If you have 3 refs reviewing a game would their opinion be more valid than the 1 top official already reffing the game? The problem with the most commentators is that they want to this single data set(walters) to prove a hypothesis (biased refs) this is called post hoc theorizing and cannot be taken seriously by people with the half a brain. If you concentrate on a new system where only objective calls are studied intensely you can say look this ref makes a lot of mistakes a lot of the time…if you use walters you can only say we THINK that this ref makes a lot of mistakes a lot of the time. Just to be clear..walter’s system is the very interesting and excellent as long as it is just for fun and general interest. If you want to put money and make a difference then system is no good.

76. I think an interesting idea would be to have a thumbs up/thumbs down feature next to each individual call on the ref report (potentially in another column if they were laid out in Walter’s style). It would not be that hard to implement and would allow for a wikipedia-style empowerment of the reader. It would be prone to spam from strongly opinionated fans, so maybe a registration should be required. I just think that the ability for the user to rate the referee (and in a way, the ref reviewer) on a decision by decision basis would be a good addition to the system.

77. zebidiah – I think that your premises are wrong. some decisions – you are correct – are matter of opinion and are very hard to say “well he got it all wrong” and in those instances the ref should be given full points (despite the reviewer thinking differently). But why not take into account ludicrus, baselss decisions? Do you think that every so-called subjective decision, when reviewed by 3 different refs, will result in 3 differnt opinions?

78. @Zebidiah – this all depends on the analysis and how it is presented. At present our aim (if I may speak for the others) is to capture as much data as possible. The method and breadth of that is still to be defined but it will capture the data that is currently captured as part of Walter’s review, probably more. I am not interested in the scoring system at present, only the data.

    What I think might be the way forward is to get the data as per the referee’s log book as a starting point for the match analysis i.e. the page will be populated as it [the match] was called – these calls can be contested (if deemed incorrect) and events added that were missed i.e. it will take the load of the examiner in that if the referee has a perfect match he need not add or contest any events. Can it really be subjective to call a trip or shirt tug a foul or are we dealing with shades of gray? Maybe a % weighting slider could be used, a ‘nailed-on-ometer’ pehaps? I do understand that these things are not as black & white as the ball crossing the line or the question of if it was a corner or a goal kick.

    The ‘class’ of the types of incident would no doubt be different in regards to its objective weight and that can be debated ad-infinitum as and when it is analysed. A say this as you are projecting or extrapolating a ‘system’ (that as yet does not exist) to the data capture phase of this project.

    I am going to be doing my own analysis of the types of incident and lists of infringements that can be called in order to enforce a template for this data i.e. 90% of the capture will hopefully be picked from drop-down lists.

    My goals from this phase of the project are not to prove anything is amiss, this is an investigation and an exercise in data gathering – once that is done I will be looking for an statistical significance or data relationships relating to consistency as this is a concern that I have about the refereeing standard of the PGMOL. It could turn out that, once we look at the data, there will be nothing of much significance to gleam from it – but even if that were the case, it would still be a worthwhile exercise in my opinion.

79. Just to add my suggestions:

    Important to name the new site in a manner which includes all EPL teams, Untold EPL or Untold RefWatch would be ok and it will need a heading paragraph/stated goal that says that it’s goal is something like “to improve refereeing in the EPL to make sure it is an even playing field for ALL teams in the competition”

    You may need to find a Lawyer to check if the EPL won’t be able to shut you down if you are paying people to review their games (you wouldn’t think so, but if it gets too embarrassing they might try with some sort of copyright law), maybe choose a hosting country that has a history of standing up for their citizens in these types of disputes.

    Maintain a list of the RefReviewers that everyone can see and can thus keep in mind when reading their reviews,
    RefReviewer01 (not their real name as mentioned previously) – supports Arsenal
    Likes – Blackpool
    Dislikes – Stoke
    Has reviewed these games (link to all games they have reviewed).

    While I am not a Ref myself, I may know one or two that might be interested and I would certainly be willing to contribute some money to the project.

    It would be a good idea to set up the site as soon as possible with the stated goal and an explanation of how and what you are trying to achieve and a registration page for interested Refs as well as a donation link (using paypal or whatever), stating that if the project doesn’t get off the ground or the funds are not required that it will be donated to X charity.

    This way you can work on the methods of how the reviews are to be done knowing how many referees you will have and how much money you have to work with, which would make it easier to know how many games you are going to be able to cover.

    To get people to the site, link it on newsnow for each club, have people like desigunner mention it on his great blog, see if you can get the BBC writer to do a follow up article on this new project (once the site is up) as well as writing up a formal ‘press release’ and pass it around to all the local papers (you never know with smaller papers they might run it just because they have no other stories to write on that day).

80. This is going to hurt…
    How do we ensure that OUR refs aren’t bent; once we gain credibility it will be tempting for someone to bribe them…especially if they only make 20 quid per match

81. By keeping our refs, except for Walter, anonymous.
    Even if one of our refs appears bent or just incompetent, on Untold Arsenal now and Untold Referees in the near future people are allowed to raise their voice and, more importantly, are listened to. Unlike the PGMOL, Untold Referees will take appropriate action if necessary.

    For the Untold Referees website I suggest hosting outside of England. That way it will be much more difficult for the EPL to shut down the website.

82. @Dogface…yes I see now that maybe I misunderstood the project a little. I thought that it would just be the defective walter system but on the large scale.

    Yet the more I think on this the more I see that only conclusion will be: ref with video better than ref without.Hard to see past this problem?

83. @Zebidiah – that conclusion is obvious, a referee with the benefit of a slow motion replay and close-ups from multiple angles will always be able to call a game better than one on the pitch – but the use of video technology is another subject entirely. We know that human error exists in this game and it is a game played at some pace in which, as a referee, you have to make calls under pressure and face the constant deception or ‘gamesmanship’ of the players to gain an advantage. These sorts of calls should all even out – unless, that is, some teams are far better at cheating than others… and even if we do find that – it will provide a fascinating team-centric study. What will be interesting (to me initially) is to try and model relationships of inconsistency i.e. one referee may have a north/south bias – where another might favour one particular team. Another referee might go easy on English players – there are many referee-centric studies that can be driven from the ‘untold’ calls he neglects or makes in error.

    The database generated from this will also provide detail to a level unprecedented in any other study I am aware of and every item of the referee’s data will be documented and can be presented and argued/amended, if needs be, through subsequent re-examination.

84. @Dogface Yes I agree but there is no reason that these random mistakes must even out over a season or even over 100 seasons. If they are truly random of course! 😉 please dont get me wrong I like what you are trying to do…but I just want to point out that the next level step of your project must be carefully considered.

    Personally I think that media darlings like terry et al get the better treatment but I also think that there are always the aces on the river…Stick to the objective mistakes of refs if you want the f.a or anyone to take notice my man…it may even form part of a drive for video technology. Otherwise ur in a quagmire.

85. If they are random then there will be no statistical significance QED it all evens out.

    If there is a statistical significance then we give assign a probability of of bias – never 100% but we could say statistically we are 99.9999% certain that A is B or there is more chance of you winning the national lottery 3 weeks on the trot than this conclusion being erroneous.

    I’m sure you understand how it works.

    Do not worry – objective ‘mistakes’ will be captured as a part of the data so any studies derived thereof can be fit for purpose (whatever that purpose may be).

    At present I only wish to enlighten the fanbase – the FA/PGMOL and EPL can go hang, I’m sure that they already know the score.