Dataset Information

No short-cut in assessing trial quality: a case study.

ABSTRACT: BACKGROUND: Assessing the quality of included trials is a central part of a systematic review. Many check-list type of instruments for doing this exist. Using a trial of antibiotic treatment for acute otitis media, Burke et al., BMJ, 1991, as the case study, this paper illustrates some limitations of the check-list approach to trial quality assessment. RESULTS: The general verdict from the check list type evaluations in nine relevant systematic reviews was that Burke et al. (1991) is a good quality trial. All relevant meta-analyses extensively used its data to formulate therapeutic evidence. My comprehensive evaluation, on the other hand, brought to the surface a series of serious problems in the design, conduct, analysis and report of this trial that were missed by the earlier evaluations. CONCLUSION: A check-list or instrument based approach, if used as a short-cut, may at times rate deeply flawed trials as good quality trials. Check lists are crucial but they need to be augmented with an in-depth review, and where possible, a scrutiny of the protocol, trial records, and original data. The extent and severity of the problems I uncovered for this particular trial warrant an independent audit before it is included in a systematic review.

SUBMITTER: Hirji KF

PROVIDER: S-EPMC2636799 | biostudies-other | 2009

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

No short-cut in assessing trial quality: a case study.

Hirji Karim F KF

Trials 20090107

<h4>Background</h4>Assessing the quality of included trials is a central part of a systematic review. Many check-list type of instruments for doing this exist. Using a trial of antibiotic treatment for acute otitis media, Burke et al., BMJ, 1991, as the case study, this paper illustrates some limitations of the check-list approach to trial quality assessment.<h4>Results</h4>The general verdict from the check list type evaluations in nine relevant systematic reviews was that Burke et al. (1991) i ...[more]

PMID: 19128475

Similar Datasets

Project description:Birth centres are described as settings where women with uncomplicated pregnancies can give birth in a home-like environment assisted by midwives and maternity care assistants. If complications arise or threaten, the woman is referred to a maternity unit of a hospital where an obstetrician will take over responsibility. In the last decade, a number of new birth centres have been established in the Netherlands, based on the assumption that birth centres provide better quality of care since they offer a better opportunity for more integrated care than the existing system with independent primary and secondary care providers. At present, there is no evidence for this assumption. The Dutch Birth Centre Study is designed to present evidence-based recommendations for organization and functioning of future birth centres in the Netherlands. A necessary first step in this evaluation is the development of indicators for measuring the quality of the care delivered in birth centres in the Netherlands. The aim of this study is to identify a comprehensive set of structure and process indicators to assess quality of birth centre care.We used mixed methods to develop a set of structure and process quality indicators for evaluating birth centre care. Beginning with a literature review, we developed an exhaustive list of determinants. We then used a Delphi study to narrow this list, calling on experts to rate the determinants for relevance and feasibility. A multidisciplinary expert panel of 63 experts, directly or indirectly involved with birth centre care, was invited to participate.A panel of 42 experts completed two Delphi rounds rating determinants of the quality of birth centre care based on their relevance (to the setting) and feasibility (of use). A set of 30 determinants for structure and process quality indicators was identified to assess the quality of birth centre care in the Netherlands.We identified 30 determinants for structure and process quality indicators concerning birth centre care. This set will be validated during the evaluation of birth centres in the Dutch Birth Centre Study.

Project description:Chronic lymphocytic leukaemia (CLL) is renowned for its variable clinical course and response to therapy, suggesting a role for precision medicine. However, the molecular basis of CLL variability remains incompletely understood. Recent developments in data independent acquisition (DIA) -MS technologies, such as SWATH (Sequential Windowed Acquisition of all THeoretical fragments), provide an opportunity to study the pathophysiology of CLL at the proteome level. This report describes the sample replication and data handling requirements for SWATH-MS analysis of clinical samples. A CLL-specific spectral library compromising over 1,500,000 spectra with digital information for over 150,000 peptides has been generated to enable the quantification of up to 7736 proteins. To assess the reproducibility of the assay, SWATH-MS analysis was performed on 6 cryopreserved CLL patient samples, incorporating biological (IGHV mutational status), sample preparation and MS technical replicates. Quantitative information was obtained for 5169 proteins (<1% FDR) across 54 SWATH-MS acquisitions. These analyses showed that SWATH-MS is a highly reproducible technique. However, replicate sample preparations on different days was identified as the main source of variation. To overcome the effect of variation between sample preparation batches, different computational approaches for batch correction were tested. All approaches successfully removed technical variability whilst retaining proteomic differences between clinical subgroups. Functional enrichment analysis of proteins associated to IGHV mutational status highlighted the importance of metabolic remodelling in the biology of CLL and overlapped significantly with previous studies based on gene expression profiling. Finally, an approach to perform statistical power analysis in proteomics studies was developed. This approach could be used to drive the design of future CLL SWATH-MS studies. These fundamental requirements for DIA approaches should be widely applicable to other clinical proteomics studies.

Dataset Information

No short-cut in assessing trial quality: a case study.

Publications

No short-cut in assessing trial quality: a case study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure