ELPAC and ICAO Doc 10197

Paper 1 presents recordings of radiotelephony exchanges and longer operational audio (briefings, ATIS-style messages, incident reports), with comprehension items keyed to prescribed correct responses. Paper 2 Task 1a places the candidate in their own operational role while the Operational Expert plays the counterpart from a script, mixing routine messages that call for standard phraseology with developing non-routine events that require plain English.

Listening and speaking in radiotelephony contexts are assessed in separate tasks, in separate papers, and the material for each role is drawn from that role's own radio work.

ELPAC ATC and ELPAC Pilot are two separate instruments. They assess the same construct, but the audio, charts, scenarios and role-play counterparts are drawn from the candidate's own environment: air traffic control radiotelephony for controllers, flight deck communication for pilots. Paper 1 is delivered in four parts for the ATC version and five parts for the Pilot version. A controller is not asked to handle flight deck tasks, and a pilot is not asked to work traffic.

Role matching extends to the rating panel: the Operational Expert is an operational or former air traffic controller or pilot, matched to the candidate's role.

Paper 1 assesses listening comprehension on its own, with 45 computer-marked items across four (ATC) or five (Pilot) parts and a documented item-level psychometric record. No speaking is required, so comprehension is measured directly rather than being read off the interaction in Paper 2.

The recordings vary in type and length: short radiotelephony transmissions in one set of parts, and longer operational audio such as briefings, ATIS-style messages and incident reports in others. Comprehension observed inside the Paper 2 interaction functions as additional supporting evidence within the Comprehension sub-rating, not as the primary measure.

The instrument has two clearly separated papers. Paper 1 moves from recognition items on short radiotelephony transmissions to multiple-choice comprehension on longer operational audio, across four (ATC) or five (Pilot) parts. Paper 2 has three tasks with different demands: an operational role play with an informal debriefing to a supervisor (Task 1a and 1b), a task on unusual or non-routine situations (Task 2), and extended interaction involving hypothesising and problem-solving (Task 3).

The content types differ across parts as well as the task types: standard phraseology in routine traffic, plain language under a developing complication, and plain language in a work-related but non-radiotelephony context (the debriefing). A candidate cannot reach a high overall level on the strength of one narrow skill.

Paper 2 is conducted face to face, not through recorded prompts. The Operational Expert works from an interlocutor script that is written to be answered rather than merely delivered, and the traffic situation develops in response to what the candidate does. The supervisor debriefing in Task 1b asks targeted follow-up questions whenever detail is missing, so the exchange builds on what the candidate has already said.

Task 3 extends the interaction into hypothesising and problem-solving, which produces longer stretches of speech than a question-and-answer format would. Across the three tasks the dialogue is long enough and varied enough to rate all six sub-ratings, which is what makes the Interaction sub-rating of the ICAO rating scale directly observable.

Differentiation is built into the task design first. Paper 1 progresses from short transmissions, where comprehension of a single instruction or request is enough, to longer and denser operational audio that carries more information than a Level 4 listener will reliably hold. In Paper 2, routine traffic in Task 1a is where Level 4 performance becomes visible, while Task 2 (unusual and non-routine situations) and Task 3 (hypothesising and problem-solving) require the flexibility, sensitivity to nuance and handling of the unexpected that separate Levels 5 and 6. Both papers therefore contain content deliberately pitched above the licensing threshold.

That design intention is then checked empirically rather than assumed. Paper 2 is rated live by two accredited examiners, an English Language Expert and an Operational Expert, who independently apply the ICAO holistic descriptors and the six ICAO rating scale criteria; the overall level awarded is the lowest of the six sub-ratings. Rating scale functioning and examiner severity are monitored with many-facet Rasch measurement on the Paper 2 evidence page. For Paper 1, item facility and discrimination, reliability and standard error of measurement are published per form under classical test statistics and Rasch analysis, which show how precisely the paper separates candidates around the decision points that matter.

On ICAO's point about the source of difficulty: instructions and rubrics are short and are given before the audio or the task begins, and the Paper 2 interlocutor script is written in operational language, so difficulty comes from the operational content rather than from working out what is being asked.

Paper 2 is not an interview about aviation. The candidate works a traffic situation from a chart in their own operational role, handles routine traffic and then a developing complication, and afterwards reports the event to a supervisor. That covers both halves of the criterion: the radiotelephony context, and a work-related situation away from the radio.

Paper 1 uses genuine radiotelephony-style audio and operational material such as briefings and incident reports rather than general-English listening passages, so listening as well as speaking is assessed in contexts the candidate recognises from work, and in the contexts specific to their own role.

Doc 10197's guidance chapter on this criterion identifies role play as the speaking task type that maximises authenticity, and names debriefings and reporting on events as appropriate work-related tasks outside radiotelephony. Both are central to Paper 2 rather than peripheral to it.

Several operational Paper 1 versions are in circulation for each role, all built to the same specification: identical structure, number of parts and 45 items, so each version represents the construct in the same way. Comparability between versions is not assumed but checked: per-version reliability, mean score, standard error of measurement and item quality are published side by side under classical test statistics, and item difficulties are placed on a common scale in the Rasch analysis. Paper 2 uses several scripted scenarios per role, written to a common task specification; scenario allocation and rotation are controlled under the ELPAC Handbook and Assessment Scheme.