Dataset Information

Norming a VALUE rubric to assess graduate information literacy skills.

ABSTRACT:

Objective

The study evaluated whether a modified version of the information literacy Valid Assessment of Learning in Undergraduate Education (VALUE) rubric would be useful for assessing the information literacy skills of graduate health sciences students.

Methods

Through facilitated calibration workshops, an interdepartmental six-person team of librarians and faculty engaged in guided discussion about the meaning of the rubric criteria. They applied the rubric to score student work for a peer-review essay assignment in the "Information Literacy for Evidence-Based Practice" course. To determine inter-rater reliability, the raters participated in a follow-up exercise in which they independently applied the rubric to ten samples of work from a research project in the doctor of physical therapy program: the patient case report assignment.

Results

For the peer-review essay, a high level of consistency in scoring was achieved for the second workshop, with statistically significant intra-class correlation coefficients above 0.8 for 3 criteria: "Determine the extent of evidence needed," "Use evidence effectively to accomplish a specific purpose," and "Access the needed evidence." Participants concurred that the essay prompt and rubric criteria adequately discriminated the quality of student work for the peer-review essay assignment. When raters independently scored the patient case report assignment, inter-rater agreement was low and statistically insignificant for all rubric criteria (kappa=-0.16, p>0.05-kappa=0.12, p>0.05).

Conclusions

While the peer-review essay assignment lent itself well to rubric calibration, scorers had a difficult time with the patient case report. Lack of familiarity among some raters with the specifics of the patient case report assignment and subject matter might have accounted for low inter-rater reliability. When norming, it is important to hold conversations about search strategies and expectations of performance. Overall, the authors found the rubric to be appropriate for assessing information literacy skills of graduate health sciences students.

SUBMITTER: Turbow DJ

PROVIDER: S-EPMC4915638 | biostudies-literature | 2016 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Norming a VALUE rubric to assess graduate information literacy skills.

Turbow David J DJ Evener Julie J

Journal of the Medical Library Association : JMLA 20160701 3

<h4>Objective</h4>The study evaluated whether a modified version of the information literacy Valid Assessment of Learning in Undergraduate Education (VALUE) rubric would be useful for assessing the information literacy skills of graduate health sciences students.<h4>Methods</h4>Through facilitated calibration workshops, an interdepartmental six-person team of librarians and faculty engaged in guided discussion about the meaning of the rubric criteria. They applied the rubric to score student wor ...[more]

PMID: 27366121

Similar Datasets

Project description:BackgroundHealth literacy among Japanese is often low, making it difficult for them to evaluate health information and make informed decisions. However, the health literacy scales applied measure the perceived difficulty of health-related tasks; they do not directly assess the specific skills needed to perform the tasks: the skills to judge the reliability of diverse information using evaluation criteria and implement rational decision-making. Therefore, the study objectives were to investigate the following issues using a nationwide survey in Japan. (1) When obtaining information, to what extent do people apply criteria for evaluating information to confirm its reliability; when making decisions, to what extent do they seek out available options and compare pros and cons based on their own values? (2) How strongly are such skills associated with health literacy and demographic characteristics? (3) What opportunities are available to learn these skills?MethodsWe conducted an online questionnaire survey using a Japanese Internet research company; 3,914 valid responses were received. The measures comprised health literacy (European Health Literacy Survey Questionnaire), five items on information evaluation, four items on decision-making, and items on the availability and location of learning opportunities. We calculated Pearson correlations to explore the association of health literacy with information-evaluation and decision-making skills. Multivariate analyses were also conducted using these factors as dependent variables.ResultsFewer than half (30%-50%) of respondents reported always or often evaluating information and engaging in decision-making. Health literacy was significantly and positively correlated with the specific skills of information evaluation and decision-making (r = .26 and .30, respectively) as were multivariate analyses (beta = .15 and .22, respectively). Over 40% of respondents had never learned those skills. The most common resources for learning the skills were the Internet and television; less-used resources were schools and workplaces.ConclusionsBoth information-evaluation and decision-making skills were associated with health literacy. However, these skills are not sufficiently widespread in Japan because there are few opportunities to acquire them. More research is needed to raise awareness of the importance of such skills for improving health literacy and providing learning opportunities.

Dataset Information

Norming a VALUE rubric to assess graduate information literacy skills.

Objective

Methods

Results

Conclusions

Publications

Norming a VALUE rubric to assess graduate information literacy skills.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets