Using the IRB Researcher Assessment Tool to Guide Quality Improvement.
Ontology highlight
ABSTRACT:
SUBMITTER: Hall DE
PROVIDER: S-EPMC4644456 | biostudies-literature | 2015 Dec
REPOSITORIES: biostudies-literature
Publications
Using the IRB Researcher Assessment Tool to Guide Quality Improvement.
Hall Daniel E DE Hanusa Barbara H BH Ling Bruce S BS Stone Roslyn A RA Switzer Galen E GE Fine Michael J MJ Arnold Robert M RM
Journal of empirical research on human research ethics : JERHRE 20151102 5
Institutional Review Boards (IRBs) are intended to protect those who participate in research. However, because there is no established measure of IRB quality, it is unclear whether these committees achieve their goal. The IRB Researcher Assessment Tool is a previously validated, internally normed, proxy measure of IRB quality that assesses 45 distinct IRB activities and functions. We administered this instrument to a sample of investigators and IRB members at a large urban VA Medical Center. We ...[more]
Similar Datasets
Project description:BackgroundInstitutional Review Boards (IRBs) have been criticized for inconsistency, delay, and bias, suggesting an opportunity for quality improvement. To aid such quality improvement, this study aimed at determining IRB members' and investigators' priorities regarding IRB review at 10 Veterans Affairs (VA) IRBs.Methods680 anonymous internet surveys were sent to 252 IRB members and staff, and 428 principal investigators and project coordinators at 9 VA Medical Centers and the VA Central IRB. Surveys included 27 statements adapted from Koocher and Kieth-Spiegel's IRB-RAT describing IRB activities or functions (e.g., "An IRB that is open to reversing its earlier decisions"). Respondents indicated how each statement described both their "ideal" and "actual" IRBs. The difference between the ratings of the actual and ideal IRBs was calculated for each item along with estimated 95% confidence intervals. Ratings outside those intervals indicated activities or functions with relatively good or poor performance compared to the ideal IRB.Results390 (57.4%) responses from 165 IRB members and staff (65.5%) and 225 investigators and project coordinators (52.6%) demonstrated that these IRBs were closest to the ideal when protecting human subjects, treating investigators with respect, and taking appropriate action for alleged scientific misconduct. The IRBs were furthest from the ideal regarding duplicative forms, timeliness of review, and provision of complete rationales for decisions. Although IRB members reported near-ideal willingness to reverse earlier decisions, investigators rated this capacity far from ideal. Investigators rated IRB members' knowledge about procedures and policy as nearly ideal, but IRB members themselves rated this aspect far from ideal. Noteworthy site-level differences in the ratings of IRB functions and activities were also identified.ConclusionsAlthough these VA IRBs perform well in some areas, these data support the need for ongoing quality improvement. The described method of administering and analyzing the IRB-RAT may help identify and monitor site- and activity- specific initiatives for quality improvement.
| S-EPMC7409436 | biostudies-literature
Project description:IntroductionFailure to achieve accrual goals is a common problem in health-related research. Electronic health records represent a promising resource, offering the ability to identify a precisely defined cohort of patients who meet inclusion/exclusion criteria. However, challenges associated with the recruitment process remain and institutional policies vary.MethodsWe interviewed researchers, institutional review board chairs, and primary care physicians in North Carolina and Tennessee. Questions focused on strategies for initiating contact with potentially eligible patients, as well as recruitment letters asking recipients to opt in versus opt out of further communication.ResultsWhen we asked about initiating contact with prospective participants, qualitative themes included trust, credibility, and established relationships; research efficiency and validity; privacy and autonomy; the intersection between research and clinical care; and disruption to physician-researcher and physician-patient relationships. All interviewees said it was acceptable for researchers to contact patients through their physicians; most said it was acceptable for researchers to contact patients directly. Over half chose contact through physicians as more appropriate. Regarding recruitment letters, qualitative themes included the quality of the participant pool; privacy and control; research efficiency and representativeness; and patients' opportunity to make their own decisions. All interviewees said asking recipients to opt in to further communication was acceptable; nearly all said opt out was acceptable. Similar proportions chose each approach as more appropriate.ConclusionsComparing these results to our previous research with patients reveals potential differences in stakeholder perspectives. We offer suggestions for developing balanced approaches that respect patients and facilitate the advancement of science.
| S-EPMC8057488 | biostudies-literature
Project description:Evidence based medicine tells us that we should not accept published research at face value. Even research from established teams published in the highest impact journals can have methodological flaws, biases and limited generalisability. The critical appraisal of research studies can seem daunting, but tools are available to make the process easier for the non-specialist. Understanding the language and process of quality assessment is essential when considering or conducting research, and is also valuable for all clinicians who use published research to inform their clinical practice.We present a review written specifically for the practising geriatrician. This considers how quality is defined in relation to the methodological conduct and reporting of research. Having established why quality assessment is important, we present and critique tools which are available to standardise quality assessment. We consider five study designs: RCTs, non-randomised studies, observational studies, systematic reviews and diagnostic test accuracy studies. Quality assessment for each of these study designs is illustrated with an example of published cognitive research. The practical applications of the tools are highlighted, with guidance on their strengths and limitations. We signpost educational resources and offer specific advice for use of these tools.We hope that all geriatricians become comfortable with critical appraisal of published research and that use of the tools described in this review - along with awareness of their strengths and limitations - become a part of teaching, journal clubs and practice.
| S-EPMC5405751 | biostudies-literature
Project description:IntroductionPost-stroke dysphagia is common and associated with significant morbidity and mortality, rendering bedside screening of significant clinical importance. Using voice as a biomarker coupled with deep learning has the potential to improve patient access to screening and mitigate the subjectivity associated with detecting voice change, a component of several validated screening protocols.MethodsIn this single-center study, we developed a proof-of-concept model for automated dysphagia screening and evaluated the performance of this model on training and testing cohorts. Patients were admitted to a comprehensive stroke center, where primary English speakers could follow commands without significant aphasia and participated on a rolling basis. The primary outcome was classification either as a pass or fail equivalent using a dysphagia screening test as a label. Voice data was recorded from patients who spoke a standardized set of vowels, words, and sentences from the National Institute of Health Stroke Scale. Seventy patients were recruited and 68 were included in the analysis, with 40 in training and 28 in testing cohorts, respectively. Speech from patients was segmented into 1,579 audio clips, from which 6,655 Mel-spectrogram images were computed and used as inputs for deep-learning models (DenseNet and ConvNext, separately and together). Clip-level and participant-level swallowing status predictions were obtained through a voting method.ResultsThe models demonstrated clip-level dysphagia screening sensitivity of 71% and specificity of 77% (F1 = 0.73, AUC = 0.80 [95% CI: 0.78-0.82]). At the participant level, the sensitivity and specificity were 89 and 79%, respectively (F1 = 0.81, AUC = 0.91 [95% CI: 0.77-1.05]).DiscussionThis study is the first to demonstrate the feasibility of applying deep learning to classify vocalizations to detect post-stroke dysphagia. Our findings suggest potential for enhancing dysphagia screening in clinical settings. https://github.com/UofTNeurology/masa-open-source.
| S-EPMC10734030 | biostudies-literature
Project description:ObjectiveValid, reliable critical appraisal tools advance quality improvement (QI) intervention impacts by helping stakeholders identify higher quality studies. QI approaches are diverse and differ from clinical interventions. Widely used critical appraisal instruments do not take unique QI features into account and existing QI tools (eg, Standards for QI Reporting Excellence) are intended for publication guidance rather than critical appraisal. This study developed and psychometrically tested a critical appraisal instrument, the QI Minimum Quality Criteria Set (QI-MQCS) for assessing QI-specific features of QI publications.MethodsApproaches to developing the tool and ensuring validity included a literature review, in-person and online survey expert panel input, and application to empirical examples. We investigated psychometric properties in a set of diverse QI publications (N=54) by analysing reliability measures and item endorsement rates and explored sources of disagreement between reviewers.ResultsThe QI-MQCS includes 16 content domains to evaluate QI intervention publications: Organisational Motivation, Intervention Rationale, Intervention Description, Organisational Characteristics, Implementation, Study Design, Comparator Description, Data Sources, Timing, Adherence/Fidelity, Health Outcomes, Organisational Readiness, Penetration/Reach, Sustainability, Spread and Limitations. Median inter-rater agreement for QI-MQCS items was κ 0.57 (83% agreement). Item statistics indicated sufficient ability to differentiate between publications (median quality criteria met 67%). Internal consistency measures indicated coherence without excessive conceptual overlap (absolute mean interitem correlation=0.19). The critical appraisal instrument is accompanied by a user manual detailing What to consider, Where to look and How to rate.ConclusionsWe developed a ready-to-use, valid and reliable critical appraisal instrument applicable to healthcare QI intervention publications, but recognise scope for continuing refinement.
| S-EPMC4680162 | biostudies-literature
Project description:SummaryLimitations of genome sequencing techniques have led to dozens of assembly algorithms, none of which is perfect. A number of methods for comparing assemblers have been developed, but none is yet a recognized benchmark. Further, most existing methods for comparing assemblies are only applicable to new assemblies of finished genomes; the problem of evaluating assemblies of previously unsequenced species has not been adequately considered. Here, we present QUAST-a quality assessment tool for evaluating and comparing genome assemblies. This tool improves on leading assembly comparison software with new ideas and quality metrics. QUAST can evaluate assemblies both with a reference genome, as well as without a reference. QUAST produces many reports, summary tables and plots to help scientists in their research and in their publications. In this study, we used QUAST to compare several genome assemblers on three datasets. QUAST tables and plots for all of them are available in the Supplementary Material, and interactive versions of these reports are on the QUAST website.Availabilityhttp://bioinf.spbau.ru/quast .Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC3624806 | biostudies-literature
Project description:Over the past few years, a large number of prediction models have been published, often of poor methodological quality. Seemingly objective and straightforward, prediction models provide a risk estimate for the outcome of interest, usually based on readily available clinical information. Yet, using models of substandard methodological rigour, especially without external validation, may result in incorrect risk estimates and consequently misclassification. To assess and combat bias in prediction research the prediction model risk of bias assessment tool (PROBAST) was published in 2019. This risk of bias (ROB) tool includes four domains and 20 signalling questions highlighting methodological flaws, and provides guidance in assessing the applicability of the model. In this paper, the PROBAST will be discussed, along with an in-depth review of two commonly encountered pitfalls in prediction modelling that may induce bias: overfitting and composite endpoints. We illustrate the prevalence of potential bias in prediction models with a meta-review of 50 systematic reviews that used the PROBAST to appraise their included studies, thus including 1510 different studies on 2104 prediction models. All domains showed an unclear or high ROB; these results were markedly stable over time, highlighting the urgent need for attention on bias in prediction research. This article aims to do just that by providing (1) the clinician with tools to evaluate the (methodological) quality of a clinical prediction model, (2) the researcher working on a review with methods to appraise the included models, and (3) the researcher developing a model with suggestions to improve model quality.
| S-EPMC9291738 | biostudies-literature
Project description:Many hospitals wish to improve their patients' experience of care. To learn whether social media could be used as a tool to engage patients and to identify opportunities for hospital quality improvement (QI), we solicited patients' narrative feedback on the Baystate Medical Center Facebook page during a 3-week period in 2014. Two investigators used directed qualitative content analysis to code comments and descriptive statistics to assess the frequency of selected codes and themes. We identified common themes, including: (1) comments about staff (17/37 respondents, 45.9%); (2) comments about specific departments (22/37, 59.5%); (3) comments on technical aspects of care, including perceived errors and inattention to pain control (9/37, 24.3%); and (4) comments describing the hospital physical plant, parking, and amenities (9/37, 24.3%). A small number (n?=?3) of patients repeatedly responded, accounting for 30% (45/148) of narratives. Although patient feedback on social media could help to drive hospital QI efforts, any potential benefits must be weighed against the reputational risks, the lack of representativeness among respondents, and the volume of responses needed to identify areas of improvement.
| S-EPMC4926770 | biostudies-literature
Project description:Ventilator-associated pneumonia (VAP) is a frequent complication of mechanical ventilation and is associated with substantial morbidity and mortality. Accurate diagnosis of VAP relies in part on subjective diagnostic criteria. Surveillance according to ventilator-associated event (VAE) criteria may allow quick and objective benchmarking. Our objective was to create an automated surveillance tool for VAE tiers I and II on a large data collection, evaluate its diagnostic accuracy and retrospectively determine the yearly baseline VAE incidence. We included all consecutive intensive care unit admissions of patients with mechanical ventilation at Bern University Hospital, a tertiary referral center, from January 2008 to July 2016. Data was automatically extracted from the patient data management system and automatically processed. We created and implemented an application able to automatically analyze respiratory and relevant medication data according to the Centers for Disease Control protocol for VAE-surveillance. In a subset of patients, we compared the accuracy of automated VAE surveillance according to CDC criteria to a gold standard (a composite of automated and manual evaluation with mediation for discrepancies) and evaluated the evolution of the baseline incidence. The study included 22'442 ventilated admissions with a total of 37'221 ventilator days. 592 ventilator-associated events (tier I) occurred; of these 194 (34%) were of potentially infectious origin (tier II). In our validation sample, automated surveillance had a sensitivity of 98% and specificity of 100% in detecting VAE compared to the gold standard. The yearly VAE incidence rate ranged from 10.1-22.1 per 1000 device days and trend showed a decrease in the yearly incidence rate ratio of 0.96 (95% CI, 0.93-1.00, p = 0.03). This study demonstrated that automated VAE detection is feasible, accurate and reliable and may be applied on a large, retrospective sample and provided insight into long-term institutional VAE incidences. The surveillance tool can be extended to other centres and provides VAE incidences for performing quality control and intervention studies.
| S-EPMC8593155 | biostudies-literature
Project description:BACKGROUND: Understanding quality improvement (QI) is an important skill for physicians, yet educational interventions focused on teaching QI to residents are relatively rare. Web-based training may be an effective teaching tool in time-limited and expertise-limited settings. INTERVENTION: We developed a web-based curriculum in QI and evaluated its effectiveness. METHODS: During the 2011-2012 academic year, we enrolled 53 first-year internal medicine residents to complete the online training. Residents were provided an average of 6 hours of protected time during a 1-month geriatrics rotation to sequentially complete 8 online modules on QI. A pre-post design was used to measure changes in knowledge of the QI principles and self-assessed competence in the objectives of the course. RESULTS: Of the residents, 72% percent (37 of 51) completed all of the modules and pretests and posttests. Immediate pre-post knowledge improved from 6 to 8.5 for a total score of 15 (P < .001) and pre-post self-assessed competence in QI principles on paired t test analysis improved from 1.7 to 2.7 on a scale of 5 for residents who completed all of the components of the course. CONCLUSIONS: Web-based training of QI in this study was comparable to other existing non-web-based curricula in improving learner confidence and knowledge in QI principles. Web-based training can be an efficient and effective mode of content delivery.
| S-EPMC3963773 | biostudies-other