How can scaled scores be compared across time?

This article is intended only for customers and partners using the Assessment Series.

Item Response Theory (IRT) methods allow assessment developers to map assessments within a grade back to the same scaled score metric. The assessments take into account the difficulty level of each assessment, so students’ scores on the Pre-Assessments and Post-Assessments can be directly compared. In other words, if a student’s score increases significantly from the Pre-Assessment to the Post-Assessment and the assessments were given under consistent conditions, that change can be more directly attributed to an increase in the student’s reading ability. 

Student scores can also fluctuate slightly between each administration of an assessment because of a number of factors (e.g., their level of concentration on a given day), so small changes in student scores from the Pre- to the Post-Assessment are not indicative of either growth or decline in student ability. Due to these factors along with variations in test questions and time between tests, a student's underlying reading ability at the time of the assessment may vary on average by up to a certain number of points above or below their reported score. This number varies by assessment, and it is listed on the score report for each assessment. Changes in scores are considered meaningful if they are beyond that threshold for each grade level. CommonLit’s data displays indicate whether or not a student’s change in score was large enough to be considered meaningful for that grade level. For instance, if the threshold is 5 points, and a student scored 8 points higher on the Post-Assessment than the Pre-Assessment, then that represents meaningful growth. However, if the student only scored 3 points higher on the Post-Assessment, then that change would not be significant.

Students who score at the high end of the scale on the Pre-Assessment may not see big changes in their scores simply because they already demonstrated a strong understanding of the material measured by the assessment. It is also important to take into account the time between test administrations. If students take two assessments very close together, they might not have learned enough between tests for their new skills to be reflected in their scores. CommonLit recommends Post-Assessments be scheduled at least three months after Pre-Assessments for full-year classes and 1.5 months after Pre-Assessments for semester-long classes. Post-Assessments cannot be scheduled less than one month after the Pre-Assessment.