A growing number of flying schools, instructors, examiners and student pilots are expressing exasperation over the new UK PPL ground exams, after witnessing unprecedented numbers of students failing their exams in recent months. The problem has become so bad that instructors and students alike are describing some of the exam questions as ‘exotic and unrelated to the course content’, with one well-respected instructor/examiner publicly stating that the PPL exams are ‘unfit for purpose’.
UK PPL and LAPL students are required to pass ground exams in nine subject areas. Exam content should be based on the UK CAA approved PPL and LAPL syllabi and so sufficient study of the relevant subject matter, coupled with taking practice exams, is normally enough preparation to ensure that students should gain the minimum 75% pass mark. The problem would appear to be, however, that the current PPL exam question bank has been modified by a non-CAA organisation and now contains a significant number of questions that many believe have been drawn from a commercial pilot ‘question bank’. The result is that exam candidates are faced with questions which fall far outside the scope of the PPL syllabus, bearing no relevance to the course content they have been studying nor the practical aspects of PPL operations.
Why this has happened is a mystery. Back in 2015, the CAA issued a tender for instructors and flying schools to write fresh questions and answers for the PPL exams, in order to create a new ‘question bank’ for PPL exams. When the CAA had received and studied these new questions, they referred them to the ‘PPL Exams Working Group’, which comprised of CAA staff and industry experts – included highly experienced instructors, examiners and PPL course material providers. This industry group was tasked with reviewing the new questions to ensure that they were suitable for use in terms of knowledge required, fairness and clarity. This Working Group first met at the CAA to review the new questions in mid-2017 and over the course of around a dozen face-to-face meetings at the CAA, the Working Group amended, and in some cases completely re-wrote, the original material to bring it up to standard. It is understood that the Working Group ended-up re-writing or amended at least 50% of the questions produced by the CAA contract, before signing them off in mid-2019 as being ready to use in new PPL exams. At a late stage in the process, the CAA announced that it was considering producing some additional questions of its own for the exams. However, it was agreed that as a matter of CAA policy, none of these CAA questions would be added to the ‘live’ question bank until they had been reviewed by the Working Group, just as all the other industry-produced questions had been.
In October 2020 the PPL exams went online for the first time, using the new question bank. Although there have been complaints about the ‘clunky’ nature of the on-line exams system, initially the overall industry reaction was favourable. However, in the following months there has been a growing chorus of complaints over ‘rogue’ questions appearing in the exams. It has become clear that contrary to the policy agreed by the CAA, extra questions have been added to the question bank which were not reviewed by the PPL Exams Working Group, and many of these questions are well-outside the scope of the PPL/LAPL Theoretical Knowledge syllabi.
The end result has been a record number of exam failures and PPL schools and their students are beginning to lose patience. Christabelle Munford is the owner of Blackbushe Flying School, which has been training pilots at Blackbushe Airport since 1985. The school has an enviable reputation for delivering first class instruction and Christabelle reports that historically at least 85% of their students gain first time passes in their ground exams. But this isn’t the case any longer. Christabelle told FTN that many of her students are now failing multiple exam sittings, despite demonstrating sound knowledge of the relevant subject matter, due to being asked questions that do not appear anywhere in the PPL syllabus. One of her current students, who holds a Physics PhD, and who Christabelle reports demonstrates a sound knowledge of the PPL syllabus, has now failed the Flight Performance and Planning exam three times. Another customer of the school is an experienced helicopter pilot, currently studying to gain a fixed-wing PPL, who likewise has failed multiple exam sittings despite having a level of knowledge and experienced that few PPL students can match.
Over at Wellesbourne Mountford Aerodrome, Rodney Galiffe, owner and operator of South Warwickshire Flying School is equally exasperated at the state of the current e-Exam system.
“I can confirm that pass marks have dropped and we have now had some failures which we had not had previously …. It would seem that many of the students that passed have made guesses on certain answers as they had never come across the subject material in their textbooks,” Rodney told FTN. “I have also written to AOPA [the Aircraft Owners and Pilots Association] previously which is all I can do as the main problem seems to be a clear lack of consultation between the CAA department and the two approved syllabus suppliers. There is also quite clearly a reminder that there is little consultation with the majority of flying schools regarding many such matters. It will be interesting to note what remedial and correcting action the CAA intend to take or does someone have to take the first legal action against the Authority for further forced corrective action to be taken!”
The Independent Flight Examiners and Instructors Association (IFEIA) are equally unhappy with the current state of the e-Exam question bank. UK senior examiner David Hoy, co-chairman of IFEIA, told FTN: “IFEIA has been approached by a number of instructors, examiners and flight schools with concerns over the content and the delivery of the PPL examination papers and the new system of administering the examinations.
“As most will know the CAA recently went over to an on-line system of issuing examination papers. This has proven cumbersome and difficult to access, a total failure as one school described it. The CAA appears to have outsourced the bank of questions to a company previously involved in commercial examinations; a company which has failed to align the PPL questions to the PPL syllabus and learning objectives.
“IFEIA has been told that the outsourcing was needed as the CAA no longer has the internal expertise or ability to produce its own bank of questions. Nor, it seems, does it appear to employ a staff which understands its own learning objectives. The consequence of this is that a number of questions in both the PPL (A) and (H) examinations are totally unsuited to wannabe private pilots and have not been covered in the textbooks produced by Pooleys or Airplan Flight Equipment. Moreover, unlike before when examiners were encouraged to discuss incorrect answers with candidates who had even passed the examinations, this ability has now disappeared. Students can fail 25% of their papers, walk away with a pass but will have had no discussion regarding the questions they failed to answer [correctly].
“IFEIA will be encouraging the CAA to hold an urgent review on both the content and the delivery of these examinations. Whilst IFEIA applauds encouraging a wider understanding of the syllabus, industry should be allowed to see and vet questions before they are approved for use.”
And it is not only flying schools, students and industry bodies who are up in arms. When the first reports surfaced about unsuitable questions appearing in the PPL exams, individual members of the PPL Exams Working Group contacted CAA staff to remind them that no new questions were to be added to the question bank without being reviewed by the Working Group. Despite repeated assurances that this had not happened, it is clear that in fact a substantial number of new questions have been added to the PPL exams without going through the agreed review process. FTN understands that the industry representatives of the Working Group have now written to the CAA, seeking an urgent meeting with CAA staff and strongly recommending that all questions which have not been reviewed by the Working Group should be removed from the ‘live’ question bank immediately.
The new PPL exams are administered by CAA International (CAAi), a limited company wholly owned by the CAA and tasked with making profits which are paid back to the CAA. It appears that individual CAA staff can be found to be working for both the CAA and CAAi. Thus, an individual could conceivably be both regulating a flying school, and tasked with making a profit from it at the same time. Given that CAAi is believed to be making a profit of at least £20,000 to £25,000 a month from providing the CAA PPL exams, cynics see an incentive for the PPL exams to be made more difficult – as the more often a candidate fails an exam – and therefore has to re-take it – the more profit CAAi makes.
The on-line exam delivery system has been outsourced to a New Zealand company called Aspeq, although how Aspeq were awarded this contract is not clear. It is believed within the industry that either CAAi or Aspeq have added the additional rogue questions to the PPL exams, and these questions may well have come from an existing professional pilot (CPL or ATPL) question databank.
In addition to mounting concern over the rogue questions in the PPL exams, the lack of a ‘debriefing’ facility in the e-exams has also been heavily criticised. This failing was raised with senior CAA personnel over a year before the on-line exams were implemented, but the CAA response was in essence that it was that the delivery system did not have the capability to offer this vital service. Thus, an exam candidate does not know which questions they have failed, and instructors and examiners cannot offer remedial or follow-up training. A further consequence is that it is impossible to know if the answer choices for a question are correct, and very difficult to ‘flag up’ rogue questions.
So far the CAA has appeared impervious to the concerns of individuals and the industry about the state of the PPL exams, and it remains to be seen whether the approaches being made by industry bodies will be received more constructively. In the meantime, PPL students are faced with a computer-generated lottery as to whether they take an exam which reflects the PPL syllabus they have trained for, or questions which are well-outside the knowledge they should be reasonably expected to have.
FTN approached both the CAA and Aspeq for comment, but at the time of going to press had not received a response.
