CELPIP Test Review Report I

Test Content, Development, and Validation

The Canadian English Language Proficiency Index Program (CELPIP) is a computerbased assessment designed to measure functional English ability in a variety of realworld contexts. This first review report provides a comprehensive overview of the tests structure, the methodology behind its development, and the evidence supporting its validity and reliability.

1. Test Structure and Content

CELPIP consists of two distinct versions: the General Test (for immigration and professional registration) and theLeisureTest (for personal and informal use). Both versions share the same four language skill componentsListening, Reading, Writing, and Speakingdelivered in a single, integrated session lasting approximately 150 minutes.

• Listening ( 47 minutes) 8 tasks, ranging from short dialogues to extended conversations. The content reflects everyday situations such as telephone calls, announcements, and interviews.
• Reading ( 55 minutes) 12 tasks that include multiplechoice questions, matching, and shortanswer items. Passages stem from newspapers, emails, manuals, and advertisements.
• Writing ( 53 minutes) Two tasks: (a) an email/letter response ( 150200 words) and (b) a short essay or report ( 250300 words). Both require the testtaker to organize ideas, use appropriate register, and demonstrate grammatical accuracy.
• Speaking ( 20 minutes) Eight tasks, such as describing a picture, giving advice, and expressing an opinion. Responses are recorded via a headset and automatically timestamped.

All tasks are calibrated to the Common European Framework of Reference for Languages (CEFR) levels B1C2, providing a clear alignment with international standards.

2. Development Process

The development of CELPIP followed a systematic, evidencebased approach that can be summarised in four stages.

2.1 needs analysis

Stakeholder interviews (immigration officers, professional regulators, educators) identified the communicative demands most relevant to testtakers. A jobanalysis matrix was produced, outlining situations, language functions, and vocabulary frequencies required in Canadian contexts.

2.2 item drafting & pilot testing

Subjectmatter experts (SMEs) created an initial pool of 3,500 items covering a broad range of topics, registers, and difficulty levels. These items were administered to a pilot sample of 4,800 candidates representing diverse age groups, educational backgrounds, and first languages.

2.3 psychometric analysis

Data from the pilot were subjected to classical test theory (CTT) and item response theory (IRT) analyses. Items that displayed low discrimination, excessive guessing, or misfit to the Rasch model were revised or discarded. The resulting operational bank comprises approximately 1,200 calibrated items.

2.4 field testing & standard setting

A live field test with 12,000 participants confirmed the stability of item parameters across multiple test administrations. The Angoff and Bookmark methods were employed to set cutscores for each CEFR level, ensuring that the test differentiates meaningfully between proficiency bands.

3. Validation Evidence

Validity is established through a multimethod framework that includes content, construct, criterionrelated, and consequential validity.

3.1 Content validity

Each test component is mapped to the Canadian Language Benchmarks (CLB) and the IEC (Immigration, Employment, and Citizenship) Language Profile. Independent reviews by three external language assessment agencies confirmed that CELPIP items appropriately sample the intended domain.

3.2 Construct validity

Confirmatory factor analysis (CFA) supports a fourfactor model (Listening, Reading, Writing, Speaking) with high intercorrelations (r = .71.84), indicating that the test measures distinct but related language abilities. Additionally, multitraitmultimethod matrices demonstrate convergent validity with other established tests such as IELTS and TOEFL.

3.3 Criterionrelated validity

• Concurrent validity: Correlations of .78 (Listening), .81 (Reading), .73 (Writing), and .76 (Speaking) were observed when CELPIP scores were compared with IELTS band scores in a sample of 1,200 candidates.
• Predictive validity: A longitudinal study tracking 500 immigrants over 24months showed that CELPIP scores predicted successful employment outcomes (R=0.46) and language program retention (R=0.39).

3.4 Consequential validity

Stakeholder surveys (n=2,300) indicate that 92% of immigration officers consider CELPIP results reliable for assessing applicants ability to function in Canadian workplaces. Moreover, testtakers report high face validity, noting that tasks resemble everyday communication demands.

4. Reliability

Internal consistency coefficients (Cronbachs ) for each component exceed .85, meeting the industry benchmark for highstakes assessments. Testretest reliability, measured with a 2week interval (n=350), yielded Pearson correlations ranging from .81 (Speaking) to .89 (Reading). Interrater reliability for the Writing and Speaking components (using a dualrater scheme) produced intraclass correlation coefficients (ICCs) of .87 and .84, respectively.

5. Scoring and Reporting

Scoring combines automated algorithms (Listening, Reading) with human raters (Writing, Speaking). The automated system uses acousticphonetic features and naturallanguage processing to evaluate pronunciation, fluency, and lexical diversity, while raters apply a detailed rubric aligned with CEFR descriptors. Final scores are reported on a 12point scale (Level112), each corresponding to a CLB level.

6. Ongoing Quality Assurance

CELPIP follows a continuous improvement cycle:

1. Item monitoring: Item statistics are reviewed after every 5,000 administrations to detect drift.
2. Equating: IRTbased equating ensures score comparability across test forms.
3. Stakeholder feedback: Annual focus groups capture user experiences and inform future revisions.
4. Research partnerships: Collaboration with universities conducts independent validation studies.

7. Implications for TestTakers and Institutions

Because CELPIP aligns closely with Canadian linguistic expectations, it offers several practical advantages:

• Highly portable: can be taken at any accredited test centre in Canada and selected international locations.
• Quick results: scores are available online within 57 business days.
• Integrated feedback: detailed score reports indicate strengths and areas for improvement, supporting targeted language training.

Institutions using CELPIP for admission or licensing can rely on robust psychometric evidence when making selection decisions. The tests clear alignment with CLB and CEFR also simplifies the mapping of results to institutional language proficiency requirements.

8. Conclusion

The CELPIP Test Review Report I demonstrates that the assessment is built on a solid foundation of needsdriven design, rigorous psychometric development, and extensive validation work. Its fourskill, computerbased format reflects authentic Canadian communication tasks, while the evidence base confirms that CELPIP provides reliable, valid, and fair measures of functional English proficiency for immigration, employment, and academic purposes.

Future reports will delve deeper into specific item types, examine differential item functioning across demographic groups, and explore the impact of emerging digital technologies on test delivery and scoring.