Dataset Information

Four Empirically Based Reasons Not to Administer Time-Limited Tests.

ABSTRACT: For more than a century, measurement experts have distinguished between time-limited tests and untimed power tests, which are administered without time limits or with time limits so generous that all students are assured of completing all items. On untimed power tests, students can differ in their propensity to correctly respond to every item, and items should differ in how many correct responses they elicit. However, differences among students' speed of responding do not confound untimed power tests; therefore, untimed power tests ensure more accurate assessment. In this article, we present four empirically based reasons to administer untimed power tests rather than time-limited tests in educational settings. (1) Time-limited tests are less valid; students' test-taking pace is not a valid reflection of their knowledge and mastery. (2) Time-limited tests are less reliable; estimates of time-limited tests' reliability are artificially inflated due to artifactual consistency in students' rate of work rather than authentic consistency in students' level of knowledge. (3) Time-limited tests are less inclusive; time-limited tests exclude students with documented disabilities who, because they are legally allowed additional test-taking time, are often literally excluded from test-taking classrooms. (4) Time-limited tests are less equitable; in addition to excluding students with documented disabilities, time-limited tests can also impede students who are learning English, students from underrepresented backgrounds, students who are older than average, and students with disabilities who encounter barriers (e.g., stigma and financial expense) in obtaining disability documentation and legally mandated accommodations. We conclude by offering recommendations for avoiding time-limited testing in higher educational assessment.

SUBMITTER: Gernsbacher MA

PROVIDER: S-EPMC7314377 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Four Empirically Based Reasons Not to Administer Time-Limited Tests.

Gernsbacher Morton Ann MA Soicher Raechel N RN Becker-Blease Kathryn A KA

Translational issues in psychological science 20200601 2

For more than a century, measurement experts have distinguished between time-limited tests and untimed power tests, which are administered without time limits or with time limits so generous that all students are assured of completing all items. On untimed power tests, students can differ in their propensity to correctly respond to every item, and items should differ in how many correct responses they elicit. However, differences among students' speed of responding do not confound untimed power ...[more]

PMID: 32582819

Similar Datasets

Project description:Administering a sustained attention test often takes a lengthy time, which can hamper routine assessments in clinical settings. Therefore, we first proposed a method to reduce the time needed for administering a sustained attention test (the Computerized Digit Vigilance Test, C-DVT). The method was to retrieve 5 segments from different trial positions of the original C-DVT testing. Then we compared the concurrent validity, convergent validity, and random measurement error of the examinees' performance on these segments to find the segment with better psychometric properties. The 5 segments were as follows: the first 50% of testing, the 21st~50th percentile of testing, the first 60% of testing, the 31st~60th percentile of testing, and the 36th~65th percentile of testing. Then we compared the validities and random measurement error of the examinees' performance on these segments. Ninety patients with stroke participated in the validity study, and 44 of them participated in the random measurement error study. The patients' scores on the 5 segments were highly correlated with those of the C-DVT (Pearson's r ≥ 0.98), indicating excellent concurrent validity. The patients' scores on the 5 segments were moderately correlated with those of the Tablet-based Symbol Digit Modalities Test (Pearson's r = -0.51~-0.48), indicating sufficient convergent validity. The amounts of random measurement error (percent standard error of measurement) were all limited: 5.1% for the C-DVT, 6.6% for the first 50% of testing, 6.0% for the 21st~50th percentile of testing, 6.1% for the first 60% of testing, 6.0% for the 31st~60th percentile of testing, and 6.1% for the 36th~65th percentile of testing. The patients needed on average 3~4 minutes to complete all the aforementioned testing. The patients' scores on the 5 segments showed excellent concurrent validity, sufficient convergent validity, and limited amounts of random measurement error in patients with stroke. We suggest the 31st~60th percentile of testing segment for users because it had the lowest amount of random measurement error and can reduce the time needed for formal testing by about 40%.

Project description:IntroductionThe development of costs-effective and sensitive screening solutions to prevent amblyopia and identify its risk factors (strabismus, refractive problems or mixed) is a significant priority of pediatric ophthalmology. The main objective of our study was to compare the classification performance of various vision screening tests, including classic, stereoacuity-based tests (Lang II, TNO, Stereo Fly, and Frisby), and non-stereoacuity-based, low-density static, dynamic, and noisy anaglyphic random dot stereograms. We determined whether the combination of non-stereoacuity-based tests integrated in the simplest artificial intelligence (AI) model could be an alternative method for vision screening.MethodsOur study, conducted in Spain and Hungary, is a non-experimental, cross-sectional diagnostic test assessment focused on pediatric eye conditions. Using convenience sampling, we enrolled 423 children aged 3.6-14 years, diagnosed with amblyopia, strabismus, or refractive errors, and compared them to age-matched emmetropic controls. Comprehensive pediatric ophthalmologic examinations ascertained diagnoses. Participants used filter glasses for stereovision tests and red-green goggles for an AI-based test over their prescribed glasses. Sensitivity, specificity, and the area under the ROC curve (AUC) were our metrics, with sensitivity being the primary endpoint. AUCs were analyzed using DeLong's method, and binary classifications (pathologic vs. normal) were evaluated using McNemar's matched pair and Fisher's nonparametric tests.ResultsFour non-overlapping groups were studied: (1) amblyopia (n = 46), (2) amblyogenic (n = 55), (3) non-amblyogenic (n = 128), and (4) emmetropic (n = 194), and a fifth group that was a combination of the amblyopia and amblyogenic groups. Based on AUCs, the AI combination of non-stereoacuity-based tests showed significantly better performance 0.908, 95% CI: (0.829-0.958) for detecting amblyopia and its risk factors than most classical tests: Lang II: 0.704, (0.648-0.755), Stereo Fly: 0.780, (0.714-0.837), Frisby: 0.754 (0.688-0.812), p < 0.02, n = 91, DeLong's method). At the optimum ROC point, McNemar's test indicated significantly higher sensitivity in accord with AUCs. Moreover, the AI solution had significantly higher sensitivity than TNO (p = 0.046, N = 134, Fisher's test), as well, while the specificity did not differ.DiscussionThe combination of multiple tests utilizing anaglyphic random dot stereograms with varying parameters (density, noise, dynamism) in AI leads to the most advanced and sensitive screening test for identifying amblyopia and amblyogenic conditions compared to all the other tests studied.

Dataset Information

Four Empirically Based Reasons Not to Administer Time-Limited Tests.

Publications

Four Empirically Based Reasons Not to Administer Time-Limited Tests.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets