Washback analysis

Washback, in the sense established by Alderson and Wall (1993) and elaborated by Bailey (1996), denotes the influence a test exerts on teaching and learning; Messick (1996) situates it within consequential validity as the systemic effect of test use on participants, processes and products (Hughes, 2003). For ELPAC, the construct of interest is Aviation English proficiency as specified in ICAO Doc 9835 and rated on the six skills of the ICAO Language Proficiency Rating Scale (pronunciation, structure, vocabulary, fluency, comprehension and interactions).

Intended washback comprises higher curricular priority for Aviation English in flight crew and air traffic controller training, stricter radiotelephony discipline, and better plain-language strategies for non-routine and emergency situations. Unintended washback comprises narrow teaching-to-the-test, neglect of plain-language ability beyond the topics most frequently sampled, and over-reliance on memorised phraseology drills at the expense of genuine interactional fluency. A balanced consequential-validity account requires both to be reported.

A methodological caveat applies throughout this chapter. ELPAC operates inside a compulsory licensing framework — the ICAO language proficiency requirements (ICAO Annex 1; ICAO Annex 10 Volume II; ICAO Doc 9835), transposed in the EUR region through Commission Regulation (EU) 2015/340 and Commission Regulation (EU) No 1178/2011 (Part-FCL FCL.055). Many observed washback effects are produced by that regime as a whole and cannot be cleanly disaggregated from the effects of any single instrument used to discharge it. The chapter therefore distinguishes regime-level washback (attributable to the ICAO LPR regime, to which ELPAC contributes as one referenced instrument) from instrument-level washback (attributable to specific features of ELPAC test design or governance). Where attribution is genuinely mixed, this is stated explicitly rather than overclaimed for ELPAC.

Since the entry into force of the ICAO language proficiency requirements (ICAO Annex 1; ICAO Annex 10 Volume II) and their transposition through Commission Regulation (EU) 2015/340 (ATCO training organisations, Subpart D) and Commission Regulation (EU) No 1178/2011 (Part-FCL, FCL.055), Aviation English has acquired explicit and verifiable curricular standing in ab-initio and recurrent training. This is regime-level washback: it is produced by the licensing obligation, not by any single test instrument. ELPAC contributes to it as one of the established instruments used to discharge that obligation in the EUR region, but equivalent curricular weight is observable wherever the ICAO LPR regime applies.

The observable consequences include a sustained market in dedicated Aviation English courses delivered by approved training organisations and ATCO training organisations, the alignment of course syllabi with the ICAO rating scale, and the routine integration of language proficiency milestones into recurrent training cycles. The honest attribution is that the regime made Aviation English non-optional; ELPAC and comparable instruments operationalise what compliance looks like.

Beyond the regime-level effect, specific features of the ELPAC instrument shape how training organisations prepare candidates. Paper 2's task-based design, anchored to operational role-play and interaction with an interlocutor, requires candidates to demonstrate comprehension, interactions and plain-language strategy under realistic conditions. This pushes preparation away from decontextualised grammar drills and towards scenario-based practice aligned with the six rated skills.

Two further design features have a measurable effect on what training organisations can usefully rehearse. The use of unseen scenario prompts means that verbatim rehearsal of past tasks does not transfer; periodic form rotation prevents preparation collapsing onto a stylised version of the test, addressing the construct-narrowing risk identified by Messick (1996). The instrument-level claim is therefore narrower but defensible: ELPAC's design discourages the most damaging form of teaching-to-the-test and rewards interactional preparation.

ICAO Annex 10, Volume II, §5.1 establishes standard phraseology as the default medium of pilot–controller communication, with plain language used only where phraseology does not suffice. The most consistent reported washback of the ICAO language proficiency regime is reinforcement of this discipline: clearer code-switching between phraseology and plain English, more reliable readback, and improved handling of non-routine and emergency situations. Attribution here is mixed — the regime sets the expectation, while ELPAC Paper 2 contributes by explicitly sampling these scenarios through its operational role-play and interaction tasks, so that preparation for the test and preparation for the operational behaviour reinforce one another.

The applied-linguistic literature on aviation communication (Estival, Farris & Molesworth, 2016; Kim & Elder, 2009; Read & Knoch, 2009) documents the interactional and lingua-franca demands of pilot–ATC talk and the role of language proficiency in miscommunication. ELPAC's construct definition and its rating scale, anchored to ICAO Doc 9835, are consistent with those demands, which supports the inference that improvements in measured ELPAC performance correspond to operationally meaningful gains in radiotelephony discipline.

A further strand of washback operates through the global professional bodies. The International Federation of Air Traffic Controllers' Associations (IFATCA) and the International Federation of Air Line Pilots' Associations (IFALPA) — the representative voices of controllers and pilots respectively — increasingly treat English communicative competence as a substantive safety issue rather than as an administrative licensing formality. Position material and conference output from both federations frame language proficiency as integral to threat and error management, to crew–ATC coordination in abnormal situations, and to the resilience of the global ATM system, rather than as a box ticked once per validity period.

This is regime-level washback in the strict sense: the stance is taken in response to the ICAO LPR regime and to operational experience, not in response to any single test. ELPAC's relationship to it is twofold. First, ELPAC benefits from it: when the professional bodies endorse communicative competence as a safety concern, the legitimacy of a serious language proficiency assessment is strengthened, and pressure for lenient or perfunctory testing is reduced. Second, ELPAC contributes to it: as one of the established instruments referenced in EUR-region implementation, ELPAC provides the kind of construct definition, rating scale anchoring and operational task sampling that allows professional-body advocacy to point to a concrete standard rather than to a general aspiration.

The ELPAC operational regime produces positive systemic washback on the assessment workforce. Centralised examiner training, the requirement for double-rating of Paper 2 with defined reconciliation, and the third-assessor escalation process (see Inference 4 of the validity argument) jointly enforce a shared interpretation of the ICAO rating scale and calibration against benchmarked performances. Examiners and raters therefore operate within a single interpretive community rather than as isolated practitioners, which is the principal organisational safeguard against rater drift.

At test-centre level, accreditation under EUROCONTROL governance promotes standardised administration conditions, audit trails and recording requirements. These reduce the systemic risk — flagged by Alderson (2009, 2010) for the ICAO regime as a whole — of opportunistic test-centre shopping for lenient ratings, and they make the test's operational quality assurance evidenced rather than asserted.

A balanced washback analysis must record the negative effects that the literature associates with high-stakes language proficiency testing in aviation: narrow teaching-to-the-test, over-rehearsal of stock phraseology at the expense of interactional fluency, candidate anxiety arising from the minimum-across-sub-ratings convention, and the structural economic incentive — documented by Alderson (2009) and Read & Knoch (2009) — for lenient or non-rigorous testing where test providers compete on pass rates rather than on construct coverage. Several of these risks are regime-level (they arise wherever the ICAO LPR regime creates a high-stakes market for language assessment); the mitigations described below are instrument-level and governance-level features specific to ELPAC.

ELPAC mitigates these risks through several converging features: independent double-rating of Paper 2 with a third-assessor escalation, unseen scenario prompts, periodic form rotation and centralised quality assurance over forms and examiners, the audit trail provided by Paper 2 recordings, and the institutional separation of the test owner (EUROCONTROL) from commercial test-centre interests. None of these eliminates negative washback; together they bound it, and they make residual negative effects observable.

Taken together, the ICAO LPR regime and the design and governance of ELPAC exert predominantly positive washback on Aviation English training and on radiotelephony discipline in the EUR region. The regime-level contribution — compulsory curricular standing for Aviation English, reinforced radiotelephony discipline, and the elevated stance of IFATCA and IFALPA on communicative competence as a safety issue — is what ELPAC participates in rather than what ELPAC alone produces. The instrument-level contribution that ELPAC can claim on its own account is the operational realisation of these expectations through task-based Paper 2 design, unseen scenarios and form rotation, double-rating with third-assessor escalation, centralised examiner calibration, and EUROCONTROL test-centre governance.

Overall, this supports the claim that ELPAC meets the consequential-validity criterion on impact on training and operational language use, with attribution honestly partitioned between the regime and the instrument. Identified negative-washback risks are bounded by the test's quality-assurance architecture and by the surrounding regulatory regime — ICAO Annex 1, ICAO Annex 10 Volume II, Commission Regulation (EU) 2015/340 and Commission Regulation (EU) No 1178/2011 (Part-FCL FCL.055) — which aligns licensing decisions with safety rather than with score economy. Residual monitoring needs are explicitly acknowledged and form part of the continuing validation programme: longitudinal evidence on operational language incidents, periodic review of training organisation curricula, and a criterion study linking ELPAC scores to operational radiotelephony performance.