Assessing English-Medium Instruction Lecturer Language Proficiency Across Disciplines

TY - JOUR

T1 - Assessing English-Medium Instruction Lecturer Language Proficiency Across Disciplines

AU - Dimova, Slobodanka

AU - Kling, Joyce

PY - 2018/9

Y1 - 2018/9

N2 - The rapid increase of English-medium instruction (EMI) programs across Europe has raised concerns regarding the oral competencies of nonnative English speaking lecturers and the implications for the quality of teaching. Consequently, lecturers’ English proficiency is under scrutiny and universities are implementing internal assessment procedures. Given the complexity of the local teaching and learning contexts in which these assessments are administered and used, answering the questions about whether and how to address the interface between language, disciplinary content, and pedagogy in the assessment procedure has been a struggle. This study is based on an oral English certification test for university EMI lecturers. Holistic scores and formative feedback reports (N = 400) from six raters, and interviews with lecturers (N = 10) were used to analyze questions related to (1) rater bias, (2) references to pedagogy, (3) reported lexical content, and (4) EMI lecturers’ perceptions about their disciplinary knowledge and vocabulary use. Rating data were examined using multifacet Rasch measurement (MFRM), while formative feedback reports and interview data were analyzed in NVivo10. MFRM results suggested no significant bias, or interaction, between raters and departments. In the written formative feedback, raters referred to linguistic aspects of pedagogy (e.g., “utilization of stress and intonation to convey pragmatic meaning”) rather than to lecturers’ classroom behavior. As for vocabulary references, results suggest that the identified problems align with general rather than domain-specific vocabulary. Interview findings suggest that, despite awareness of their lack of nuanced vocabulary, lecturers’ content knowledge and teaching experience facilitate their language performance.

AB - The rapid increase of English-medium instruction (EMI) programs across Europe has raised concerns regarding the oral competencies of nonnative English speaking lecturers and the implications for the quality of teaching. Consequently, lecturers’ English proficiency is under scrutiny and universities are implementing internal assessment procedures. Given the complexity of the local teaching and learning contexts in which these assessments are administered and used, answering the questions about whether and how to address the interface between language, disciplinary content, and pedagogy in the assessment procedure has been a struggle. This study is based on an oral English certification test for university EMI lecturers. Holistic scores and formative feedback reports (N = 400) from six raters, and interviews with lecturers (N = 10) were used to analyze questions related to (1) rater bias, (2) references to pedagogy, (3) reported lexical content, and (4) EMI lecturers’ perceptions about their disciplinary knowledge and vocabulary use. Rating data were examined using multifacet Rasch measurement (MFRM), while formative feedback reports and interview data were analyzed in NVivo10. MFRM results suggested no significant bias, or interaction, between raters and departments. In the written formative feedback, raters referred to linguistic aspects of pedagogy (e.g., “utilization of stress and intonation to convey pragmatic meaning”) rather than to lecturers’ classroom behavior. As for vocabulary references, results suggest that the identified problems align with general rather than domain-specific vocabulary. Interview findings suggest that, despite awareness of their lack of nuanced vocabulary, lecturers’ content knowledge and teaching experience facilitate their language performance.

U2 - 10.1002/tesq.454

DO - 10.1002/tesq.454

M3 - Journal article

SN - 0039-8322

VL - 52

SP - 634

EP - 656

JO - TESOL Quarterly

JF - TESOL Quarterly

IS - 3

ER -