The MSA GB has always fought against the publication of individual Approved Driving Instructor (ADI) pass rates, on the basis that the data differences make both correlation and therefore sensible comparison, of data, virtually impossible.
Considering the relatively small number of tests per instructor it is unlikely, from a statistical point of view, that it would be possible to produce reliable statistics even if a complicated algorithm were to be developed to try to cover all the variables.
As I understand it the chi-squared test, used to keep a check on examiner variances relies on a large sample. In a year a driving examiner probably conducts around one thousand five hundred driving tests. A driver trainer is unlikely to take more than fifty pupils for test in a year and probably less.
Age of candidate, gender of candidate, auto or manual gears, location of test centre; all these make for large differences in the stats recorded – as do unknown variables such as a pupil who ISN’T ready for the test exercising their democratic right to put themselves in for it anyway, as they are off to uni in a couple of weeks and they’d like to get it sorted out now, thanks very much.
Computer geeks – pull together an algorithm for that one
In the examples below I have rounded the decimals. The data is all extracted from Department for Transport (DfT) and Driver and Vehicle Testing Agency (DVSA) data sets and from a Freedom of Information (FOI) request. Each example illustrates the difficulty of providing a fair rating for an instructor based on available data.
I have also made some suggestions (IN BOLD) as to actions an instructor might take to improve their pupil pass rate.
Current overall pass rate 47% Male 51% Female 44%
Current overall pass rate (automatic) 40% Male 44% Female 38%
So, don’t teach Female pupils in automatics.
Current overall pass rate 17 year olds 57% Male 58% Female 55%
Current overall pass rate 25 year olds 46% Male 51% Female 43%
So, don’t teach anyone over 17 years old.
Highest overall pass rates for one group – 17-year olds taking their fourth test.
So, advertise for 17 year old clients who have already failed three driving tests.
These centres are all in North London and are within about 5 miles of each other
So, if you live in North London take your pupils to Mill Hill.
These centres in Greater Manchester are either inside the M60 or less than a couple of miles outside it.
So, if you live in Greater Manchester and usually use Cheetham Hill stop, take your pupils for test in Hyde they are twice as likely to pass first time.
Although there is nothing the ADI can do about it, the route taken for each test can affect the pass rate by as much as 13% as this information extracted from an FOI request shows.
The table below shows the pass and fail rates by test route for car practical driving tests conducted at Darlington driving test centre between 1 January 2012 and 31 December 2014. This data is only for routes where more than 10 tests have been conducted.
|No. tests conducted||Failed tests||%Pass|
Then of course there is the luck of the draw as to which examiner you take as this second extract from the same FOI request indicates
The table below shows the pass rate statistics for examiners who are permanently based at Darlington test centre between 1 January 2012 and 31 December 2014.
This has been supplied in an anonymised format as we do not release pass and fail rates by named examiner. Information about named individuals is exempt from release under section 36(2)(c) on the grounds that its disclosure would prejudice the effective conduct of public affairs, a view certified by the Minister and then subject to the public interest test as to whether the balance lies in favour of withholding or disclosing the information. We have concluded, applying a presumption in favour of disclosure, and in all the circumstances of the case, the public interest in maintaining the exemption outweighs the public interest in disclosing the information.
|Examiner||No. tests conducted||Failed tests||%Pass|
Publication of ADI pupil pass rates would be as meaningless as publishing individual examiner pass rates or the pass rate per route at a particular test center.
Now would be a good time for the DVSA to make it very clear that they do not intend to publish ADI pass rates.