Dataset Information

An evidence-based method for assessing the value of a search tool: a pilot study.

ABSTRACT:

Objective

The objective of this study was to develop an evidence-based method with a set of metrics that could be used to assess an information search tool.

Methods

This pilot study analyzed a two-group convenience sample of graduate nursing students and resident physicians. The intervention group received ten minutes of instruction on a familiar search tool (eSearcher). Each group was provided one prompt to search for clinical guidelines on a given topic within their scope of practice and asked to find the best result using only eSearcher (intervention group) or specifically excluding eSearcher (comparison group). Three measurements of search results were employed: time elapsed to complete the search, an accuracy score, and a participant-reported score of confidence in the result.

Results

Forty-two students participated in this study (23 graduate nursing students and 19 resident physicians). The intervention group consisted of 22 participants (12 graduate nursing students and 10 resident physicians), and the comparison group consisted of 20 participants (11 graduate nursing students and 9 resident physicians). The intervention group had lower mean ranks in both accuracy and confidence compared to the comparison (not eSearcher) group, although these differences were not significant. However, the intervention (eSearcher) group had significantly longer search times compared to the comparison (not eSearcher) group.

Discussion

These findings provided new insights into the performance of the search tool and how users felt about their search experience. The quantitative evidence gained from this study led directly to an informed decision to explore other options for search tools. The evidence-based methods and process developed in this pilot study will enable similar studies to test other student groups and other search tools, leading to better informed purchasing and instructional decisions.

SUBMITTER: Pearson DS

PROVIDER: S-EPMC6148619 | biostudies-literature | 2018 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An evidence-based method for assessing the value of a search tool: a pilot study.

Pearson Donald Stanley DS Roksandic Stevo S Kilanowski Jill J

Journal of the Medical Library Association : JMLA 20181001 4

<h4>Objective</h4>The objective of this study was to develop an evidence-based method with a set of metrics that could be used to assess an information search tool.<h4>Methods</h4>This pilot study analyzed a two-group convenience sample of graduate nursing students and resident physicians. The intervention group received ten minutes of instruction on a familiar search tool (eSearcher). Each group was provided one prompt to search for clinical guidelines on a given topic within their scope of pra ...[more]

PMID: 30271288

Similar Datasets

Project description:IntroductionThe CHARISMA intervention, nested within the MTN-025/HOPE vaginal ring trial in Johannesburg, South Africa, seeks to facilitate women's use of HIV prevention products by promoting partner dialogue and mitigating intimate partner violence (IPV). We developed "HEART", a lay counselor-administered relationship assessment tool, for the CHARISMA pilot. The five-scale tool assesses participants' endorsement of Traditional Values (TV), her HIV Prevention Readiness (HPR) and levels of partner support (PS), abuse and control (PAC), and resistance to HIV prevention (PR), guiding decisions about which of three counselling modules to offer (partner communication/A; ring disclosure/B; and IPV prevention/C).MethodsWe correlated baseline scores on HEART subscales with a) independent measures of relationship stability, disclosure and IPV to assess construct validity, and b) with specific modules offered to determine how HEART was used in the pilot. We examined changes in HEART scores at three and six months. Finally, we ran separate growth models for each subscale to examine changes in scores, accounting for partnership changes and counseling module(s) received.ResultsBaseline HEART scores correlated as predicted among subscales and with other measures. Reliabilities for four subscales were 0.75 or higher. Women who disclosed study participation and ring use scored higher on PS and lower on PR. Women experiencing IPV scored lower on PS, and higher on PAC and PR. During the pilot, 82% of women received one and 17% received two or more modules; over half received the IPV module. Women with higher PAC and PR scores were more likely to receive the IPV than the communication or disclosure modules. Over time, the TV, PAC and PR scores decreased, and PS score increased. Receiving the IPV module was associated with a decreased PAC score.ConclusionsThese data offer preliminary evidence for HEART construct and predictive validity and support its further evaluation to guide implementation and monitor the impact of the CHARISMA intervention in a randomized controlled evaluation.

Dataset Information

An evidence-based method for assessing the value of a search tool: a pilot study.

Objective

Methods

Results

Discussion

Publications

An evidence-based method for assessing the value of a search tool: a pilot study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets