Beyond AI Familiarity: Explaining EFL Instructors’ Professional Development Needs in AI-Mediated Assessment

Abstract

Professional-development need in generative-AI (GenAI) assessment is often treated as a deficit that should decline as teachers become more familiar and confident. This secondary analysis tests a different possibility: professional-development need may reflect a differentiated awareness of the demands of AI-mediated assessment rather than simple unfamiliarity. Using the verified anonymised responses of 216 GenAI-experienced EFL instructors in Saudi higher education, we modelled a three-item professional-development-need score from a previously validated instrument. Conditional quantile regression examined the 25th, 50th, and 75th percentiles of need, with 1000 case-bootstrap resamples. Familiarity/use alone explained 11.6%, 8.5%, and 3.3% of quantile variation, whereas the full competency model increased pseudo-R² by 12.6, 11.4, and 10.6 percentage points, respectively. The results revealed that familiarity/use was positively associated with need at the lower and median quantiles but not at the upper quantile. The results also showed that assessment literacy was negatively associated with need across all three quantiles, whereas ethical awareness and self-efficacy were consistently positive; assessment application contributed no independent association after adjustment. HC3 mean regression and item-specific ordinal-logit models reproduced the broad pattern. After demographic adjustment, ethical awareness and self-efficacy remained the most robust competency correlates. The findings challenge a simple deficit interpretation of training need and support differentiated professional development that combines assessment literacy, ethical judgement, and advanced evaluation of AI-generated assessment materials.