Washback analysis

Washback, in the sense established by Alderson and Wall (1993) and elaborated by Bailey (1996), describes the influence a test exerts on teaching and learning; Messick (1996) situates it within consequential validity as the systemic effect of test use on participants, processes and products (Hughes, 2003). For ELPAC, the construct of interest is Aeronautical Radiotelephony Communication as specified in the ICAO language proficiency requirements in ICAO Annex 1, ICAO Doc 9835 and ICAO Doc 10197.

Intended washback comprises higher curricular priority for Aeronautical Radiotelephony Communication in flight crew and air traffic controller training, stricter radiotelephony discipline, and better plain-language strategies for non-routine and emergency situations. Unintended washback comprises narrow teaching-to-the-test, neglect of plain-language ability beyond the topics most frequently sampled, and over-reliance on memorised phraseology drills at the expense of genuine interactional fluency. A balanced consequential-validity account requires both to be reported.

Since the ICAO language proficiency requirements (ICAO Annex 1; ICAO Annex 10 Volume II) and their transposition through Commission Regulation (EU) 2015/340 (ATCO training organisations, Subpart D) and Commission Regulation (EU) No 1178/2011 (Part-FCL, FCL.055) came into effect, Aeronautical Radiotelephony Communication has received increased interest in ab-initio and recurrent training. This is regime-level washback: it is produced by the licensing obligation. ELPAC contributes to it as one of the recognised tests used by organisations to implement and comply with the ICAO language proficiency requirements.

The observable consequences include a sustained market in dedicated Aeronautical Radiotelephony Communication courses delivered by approved training organisations and ATCO training organisations, the alignment of course syllabi with the ICAO rating scale, and the routine integration of language proficiency milestones into recurrent training cycles.

Specific aspects of the ELPAC test shape how training organisations prepare test takers. Paper 2's task-based design, based on operational role-play and interaction with an interlocutor, requires test takers to demonstrate communicative and interactional skills that reflect the real-world situations they are likely to encounter on the radiotelephony frequency. This pushes preparation away from decontextualised grammar drills and towards scenario-based practice aligned with the six rated skills.

Two further design features have a measurable effect on what training organisations can usefully rehearse. The use of unseen scenario prompts means that verbatim rehearsal of past tasks does not transfer; and Paper 1 forms are assembled from a calibrated item bank, so preparation cannot collapse onto a stylised version of the test. This addresses the construct-narrowing risk identified by Messick (1996). The test-level claim is therefore narrower but defensible: ELPAC's design discourages the most damaging form of teaching-to-the-test and rewards interactional preparation.

ICAO Annex 10, Volume II, §5.1 identifies standard phraseology as the default register for communication between air traffic controllers and pilots, with plain language used only where phraseology does not suffice. The most consistent reported washback of the ICAO language proficiency regime is reinforcement of this discipline, through stricter control on code-switching between phraseology and plain English. Attribution here is mixed: the ICAO language proficiency requirements set the expectation, while ELPAC Paper 2 contributes by explicitly sampling these scenarios through its operational role-play and interaction tasks, so that preparation for the test and preparation for the operational behaviour reinforce one another.

The applied-linguistic literature on aviation communication (Estival, Farris & Molesworth, 2016; Kim & Elder, 2009; Read & Knoch, 2009) documents the interactional and lingua franca demands of communication between air traffic controllers and pilots, and the role of language proficiency in miscommunication. ELPAC's construct definition and its assessment scheme, based on ICAO Doc 9835 and ICAO Annex 1, are consistent with those demands, which supports the inference that improvements in measured ELPAC performance correspond to operationally meaningful gains in radiotelephony discipline.

The ELPAC test produces positive systemic washback on the assessment workforce. Centralised assessor training, a structured assessment procedure in Paper 2, and the third-assessor escalation process (see Inference 4 of the validity argument) jointly enforce a shared interpretation and calibration of the ELPAC assessment scheme against benchmarked performances. Assessors operate within a single interpretive community rather than as isolated practitioners, which is the principal organisational safeguard against rater drift.

A balanced washback analysis must record the negative effects that the literature associates with high-stakes language proficiency testing in aviation: narrow teaching-to-the-test, over-rehearsal of stock phraseology at the expense of interactional fluency, test taker anxiety arising from the lowest-of-six convention, and the structural economic incentive, documented by Alderson (2009) and Read & Knoch (2009), for lenient or non-rigorous testing where test providers compete on pass rates rather than on construct coverage. Several of these risks arise from the way the ICAO language proficiency requirements have been set up.

ELPAC mitigates these risks through several converging features: independent double-rating of Paper 2 with a third-assessor escalation, unseen scenario prompts, calibrated item banking and centralised analysis of test items and assessor performance, the audit trail provided by Paper 2 recordings, and the institutional separation of the test owner (EUROCONTROL) from commercial test-centre interests. None of these eliminates negative washback; together they bound it, and they make residual negative effects observable.

Taken together, the ICAO language proficiency requirements and the design and governance of ELPAC exert predominantly positive washback. The contribution that ELPAC can claim on its own account is the operationalisation of the target language use domain through task-based Paper 2 design, calibrated item banking, double-rating with third-assessor escalation, centralised assessor calibration, and EUROCONTROL test-centre governance.

Overall, this supports the claim that ELPAC meets the consequential-validity criterion on impact on training and operational language use. Identified negative-washback risks are bounded by the monitoring mechanisms for the test's performance and that of its assessors, and by the surrounding regulatory framework (ICAO Annex 1, ICAO Annex 10 Volume II, Commission Regulation (EU) 2015/340 and Commission Regulation (EU) No 1178/2011, Part-FCL FCL.055), which aligns licensing decisions with safety rather than with score economy. Residual monitoring needs are explicitly acknowledged and form part of the continuing validation programme: longitudinal evidence on operational language incidents, periodic review of training organisation curricula, and a criterion study linking ELPAC scores to operational radiotelephony performance.