Development and reliability assessment of a new quality appraisal tool for cross-sectional studies using biomarker data (BIOCROSS).
Ontology highlight
ABSTRACT: Background
Biomarker-based analyses are commonly reported in observational epidemiological studies; however currently there are no specific study quality assessment tools to assist evaluation of conducted research. Accounting for study design and biomarker measurement would be important for deriving valid conclusions when conducting systematic data evaluation.Methods
We developed a study quality assessment tool designed specifically to assess biomarker-based cross-sectional studies (BIOCROSS) and evaluated its inter-rater reliability. The tool includes 10-items covering 5 domains: 'Study rational', 'Design/Methods', 'Data analysis', 'Data interpretation' and 'Biomarker measurement', aiming to assess different quality features of biomarker cross-sectional studies. To evaluate the inter-rater reliability, 30 studies were distributed among 5 raters and intraclass correlation coefficients (ICC-s) were derived from respective ratings.Results
The estimated overall ICC between the 5 raters was 0.57 (95% Confidence Interval (CI): 0.38-0.74) indicating a good inter-rater reliability. The ICC-s ranged from 0.11 (95% CI: 0.01-0.27) for the domain 'Study rational' to 0.56 (95% CI: 0.40-0.72) for the domain 'Data interpretation'.Conclusion
BIOCROSS is a new study quality assessment tool suitable for evaluation of reporting quality from cross-sectional epidemiological studies employing biomarker data. The tool proved to be reliable for use by biomedical scientists with diverse backgrounds and could facilitate comprehensive review of biomarker studies in human research.
SUBMITTER: Wirsching J
PROVIDER: S-EPMC6219097 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundIn the context of the volume of mixed- and multi-methods studies in health services research, the present study sought to develop an appraisal tool to determine the methodological and reporting quality of such studies when included in systematic reviews. Evaluative evidence regarding the design and use of our existing Quality Assessment Tool for Studies with Diverse Designs (QATSDD) was synthesised to enhance and refine it for application across health services research.MethodsSecondary data were collected through a literature review of all articles identified using Google Scholar that had cited the QATSDD tool from its inception in 2012 to December 2019. First authors of all papers that had cited the QATSDD (n=197) were also invited to provide further evaluative data via a qualitative online survey. Evaluative findings from the survey and literature review were synthesised narratively and these data used to identify areas requiring refinement. The refined tool was subject to inter-rater reliability, face and content validity analyses.ResultsKey limitations of the QATSDD tool identified related to a lack of clarity regarding scope of use of the tool and in the ease of application of criteria beyond experimental psychological research. The Quality Appraisal for Diverse Studies (QuADS) tool emerged as a revised tool to address the limitations of the QATSDD. The QuADS tool demonstrated substantial inter-rater reliability (k=0.66), face and content validity for application in systematic reviews with mixed, or multi-methods health services research.ConclusionOur findings highlight the perceived value of appraisal tools to determine the methodological and reporting quality of studies in reviews that include heterogeneous studies. The QuADS tool demonstrates strong reliability and ease of use for application to multi or mixed-methods health services research.
| S-EPMC7885606 | biostudies-literature
Project description:Objectives: Rupture of a brain arteriovenous malformation (bAVM) can cause intracranial hemorrhage and severe clinical outcomes. At present, the mechanisms of bAVM-related hemorrhage are poorly understood. This study aimed to summarize the potential genetic risk factors for bAVM-related hemorrhage and appraise the methodological quality of existing genetic studies on bAVM-related hemorrhage using a cross-sectional design. Methods: A systematic literature search was conducted on genetic studies associated with bAVM-related hemorrhage published in PubMed, Embase, Web of Science, China National Knowledge Internet, and Wangfang databases, up to November 2022. Subsequently, a cross-sectional study was performed to describe the potential candidate genetic variants of bAVM associated with risk of hemorrhage and to evaluate the methodological quality of the identified studies using the Newcastle-Ottawa quality assessment scale and Q-genie tool. Results: Of the 1811 records identified in the initial search, nine studies met the filtering criteria and were included. Twelve single nucleotide polymorphisms (SNPs), including IL6 rs1800795, IL17A rs2275913, MMP9 rs9509, VEGFA rs1547651, and EPHB4 rs314353, rs314308, and rs314313, were associated with bAVM-related hemorrhage. However, only 12.5% of the evaluated SNPs showed statistical power> 0.80 (α = 0.05). Methodological quality assessment revealed significant flaws in the designs of the included studies, such as less reliable representativeness of recruited individuals, short follow-up periods in cohort studies, and less comparability between groups of hemorrhagic and non-hemorrhagic patients. Conclusion: IL1B, IL6, IL17A, APOE, MMP9, VEGFA and EPHB4 were potentially associated with bAVM-related hemorrhage. The methodological designs of the analyzed studies required improvement in order to obtain more reliable results. Regional alliances and rare disease banks need to be established to recruit large numbers of bAVM patients (especially familial and extreme-trait cases) in a multicenter, prospective cohort study with an adequate follow-up period. Furthermore, it is important to use advanced sequencing techniques and efficient measures to filter candidate genetic variants.
| S-EPMC10099571 | biostudies-literature
Project description:BackgroundAs the most common malignant tumor in the world, breast cancer also brings a huge disease burden to China. Ordinary people are increasingly inclined to use the Internet, especially video social platforms, as a source of health information. Educating the public to obtain correct information is important to reduce the incidence of breast cancer and improve the prognosis. However, the quality and reliability of breast cancer-related video content have not been fully studied.ObjectiveThis study aims to evaluate the quality of the information of breast cancer-related videos on TikTok and Bilibili video sharing platforms and factors related to video quality.MethodsWe collected the top 100 videos about breast cancer on TikTok and Bilibili, respectively. Categorize videos according to video source and video content. Video quality and reliability were assessed using Global Quality Score (GQS) and modified DISCERN (mDISCERN) tools. We also analyzed the correlation between video quality and video likes, comments, saves, and shares.ResultsAlthough the quality and reliability of Bilibili's breast cancer videos were higher than TikTok (p = 0.002 and p = 0.001, respectively), the video quality of both video sharing platforms was not satisfactory, with a median GQS scores of 2.00 and 3.00 and mDISCERN scores of 1.00 and 2.00, respectively. In general, the quality and reliability of videos released by medical practitioners were higher than those of non-medical practitioners, and the quality and reliability of videos covering disease-related knowledge were higher than those of news reports (all p < 0.001). Among medical practitioners, the quality of videos uploaded by doctors in breast disease was significantly lower than that of doctors in other areas (p < 0.05). There was a significant positive correlation between video quality and duration (r = 0.240, p < 0.001), a weak negative correlation between video quality and likes (r = 0.191, p < 0.01), video quality and comments (r = 0.256, p < 0.001), video reliability and likes (r = 0.198, p < 0.001), video reliability and comments (r = 0.243, p < 0.01).ConclusionOur study shows that the quality and reliability of breast cancer-related videos on TikTok and Bilibili are poor, and the overall quality is unsatisfactory. But videos uploaded by medical practitioners covering disease knowledge, prevention and treatment are of higher quality. Medical practitioners are encouraged to publish more high-quality videos, while video social platforms should formulate relevant policies to censor and supervise health education videos, so as to enable the public to obtain reliable health information.
| S-EPMC10839971 | biostudies-literature
Project description:BackgroundObservational comparative effectiveness studies in allergen immunotherapy (AIT) represent an important evidence source answering research questions that can be challenging to obtain from randomized controlled trials (RCTs), such as long-term benefits of AIT, the effects on asthma prevention and the onset of new allergen sensitizations. However, observational studies are prone to several sources of bias, which limit their reliability.The REal Life Evidence AssessmeNt Tool (RELEVANT) was recently developed to assist in quality appraisal of observational comparative research to enable identification of useful nonrandomized studies to be considered within guideline development.ObjectiveTo systematically appraise the quality of published observational comparative AIT studies using RELEVANT.MethodsObservational studies comparing AIT to pharmacotherapy for respiratory allergies, assessing as outcome measures reduction of symptoms and/or medication use reduction, were retrieved by computerized bibliographic searches. According to RELEVANT, a failure to meet any one of primary items (background, design, measures, analysis, results, discussion/interpretation, and conflict of interest) represents a critical flaw, significantly undermining the validity of the study results.ResultsThe 14 studies identified supported the benefit of AIT in real-life, which persists after treatment discontinuation. However, none of them met all the 7 primary RELEVANT criteria. The main defects were reported in the design (28.6% of studies), measures and analysis (64.3% of studies), and results (78.6% of studies) items, due to selection bias and lack of methods for adjusting controls. Half of the studies did not report on conflict of interest.ConclusionThere is a need for more robust observational research in AIT. RELEVANT appears as an easy-to-use and sensitive tool for quality appraisal in AIT studies.
| S-EPMC8203181 | biostudies-literature
Project description:ObjectivesTo describe how the methodological quality of primary studies is assessed in systematic reviews and whether the quality assessment is taken into account in the interpretation of results.Data sourcesCochrane systematic reviews and systematic reviews in paper based journals.Study selection965 systematic reviews (809 Cochrane reviews and 156 paper based reviews) published between 1995 and 2002.Data synthesisThe methodological quality of primary studies was assessed in 854 of the 965 systematic reviews (88.5%). This occurred more often in Cochrane reviews than in paper based reviews (93.9% v 60.3%, P < 0.0001). Overall, only 496 (51.4%) used the quality assessment in the analysis and interpretation of the results or in their discussion, with no significant differences between Cochrane reviews and paper based reviews (52% v 49%, P = 0.58). The tools and methods used for quality assessment varied widely.ConclusionsCochrane reviews fared better than systematic reviews published in paper based journals in terms of assessment of methodological quality of primary studies, although they both largely failed to take it into account in the interpretation of results. Methods for assessment of methodological quality by systematic reviews are still in their infancy and there is substantial room for improvement.
| S-EPMC557223 | biostudies-literature
Project description:BackgroundWith the rapid increase in genome sequencing projects for non-model organisms, numerous genome assemblies are currently in progress or available as drafts, but not made available as satisfactory, usable genomes. Data quality assessment of genome assemblies is gaining importance not only for people who perform the assembly/re-assembly processes, but also for those who attempt to use assemblies as maps in downstream analyses. Recent studies of the quality control, quality evaluation/ assessment of genome assemblies have focused on either quality control of reads before assemblies or evaluation of the assemblies with respect to their contiguity and correctness. However, correctness assessment depends on a reference and is not applicable for de novo assembly projects. Hence, development of methods providing both post-assembly and pre-assembly quality assessment reports for examining the quality/correctness of de novo assemblies and the input reads is worth studying.ResultsWe present SQUAT, an efficient tool for both pre-assembly and post-assembly quality assessment of de novo genome assemblies. The pre-assembly module of SQUAT computes quality statistics of reads and presents the analysis in a well-designed interface to visualize the distribution of high- and poor-quality reads in a portable HTML report. The post-assembly module of SQUAT provides read mapping analytics in an HTML format. We categorized reads into several groups including uniquely mapped reads, multiply mapped, unmapped reads; for uniquely mapped reads, we further categorized them into perfectly matched, with substitutions, containing clips, and the others. We carefully defined the poorly mapped (PM) reads into several groups to prevent the underestimation of unmapped reads; indeed, a high PM% would be a sign of a poor assembly that requires researchers' attention for further examination or improvements before using the assembly. Finally, we evaluate SQUAT with six datasets, including the genome assemblies for eel, worm, mushroom, and three bacteria. The results show that SQUAT reports provide useful information with details for assessing the quality of assemblies and reads.AvailabilityThe SQUAT software with links to both its docker image and the on-line manual is freely available at https://github.com/luke831215/SQUAT .
| S-EPMC7402383 | biostudies-literature
Project description:BackgroundPopulation-based surveys collect crucial data on anthropometric measures to track trends in stunting [height-for-age z score (HAZ) < -2SD] and wasting [weight-for-height z score (WHZ) < -2SD] prevalence among young children globally. However, the quality of the anthropometric data varies between surveys, which may affect population-based estimates of malnutrition.ObjectivesWe aimed to develop composite indices of anthropometric data quality for use in multisurvey analysis of child health and nutritional status.MethodsWe used anthropometric data for children 0-59 mo of age from all publicly available Demographic and Health Surveys (DHS) from 2000 onwards. We derived 6 indicators of anthropometric data quality at the survey level, including 1) date of birth completeness, 2) anthropometric measure completeness, 3) digit preference for height and age, 4) difference in mean HAZ by month of birth, 5) proportion of biologically implausible values, and 6) dispersion of HAZ and WHZ distribution. Principal component factor analysis was used to generate a composite index of anthropometric data quality for HAZ and WHZ separately. Surveys were ranked from the highest (best) to the lowest (worst) index values in anthropometric quality across countries and over time.ResultsOf the 145 DHS included, the majority (83 of 145; 57%) were conducted in Sub-Saharan Africa. Surveys were ranked from highest to lowest anthropometric data quality relative to other surveys using the composite index for HAZ. Although slightly higher values in recent DHS suggest potential improvements in anthropometric data quality over time, there continues to be substantial heterogeneity in the quality of anthropometric data across surveys. Results were similar for the WHZ data quality index.ConclusionsA composite index of anthropometric data quality using a parsimonious set of individual indicators can effectively discriminate among surveys with excellent and poor data quality. Such indices can be used to account for variations in anthropometric data quality in multisurvey epidemiologic analyses of child health.
| S-EPMC7487428 | biostudies-literature
Project description:The appropriate acquisition and processing of water quality data are crucial for water resource management. As such, published articles on water quality monitoring and assessment are meant to convey essential and reliable information to water quality experts, decision-makers, researchers, students, and the public. The implication is that such information must emanate from data obtained and analysed in an up-to-date, scientifically sound manner. Thus, inappropriate data analysis and reporting techniques could yield misleading results and mar the endeavours of achieving error-free conclusions. This study utilises the findings on water quality assessment in Nigeria over the last 20 years to reveal the likely trends in water quality research regarding data collection, data analysis, and reporting for physicochemical, bacteriological parameters, and trace organics. A total of 123 Web of Science and quartile ranked (Q1-Q4) published articles involving water quality assessment in Nigeria were analysed. Results indicated shortcomings in various aspects of data analysis and reporting. Consequently, we use simulated heatmaps and graphs to illustrate preferred ways of analysing, reporting, and visualising some regularly used descriptive and inferential statistics of water quality variables. Finally, we highlight alternative approaches to the customarily applied water quality assessment methods in Nigeria and emphasise other areas of deficiency that need attention for improved water quality research.
| S-EPMC8450204 | biostudies-literature
Project description:BackgroundWith the growing impact of observational research studies, there is also a growing focus on data quality (DQ). As opposed to experimental study designs, observational research studies are performed using data mostly collected in a non-research context (secondary use). Depending on the number of data elements to be analyzed, DQ reports of data stored within research networks can grow very large. They might be cumbersome to read and important information could be overseen quickly. To address this issue, a DQ assessment (DQA) tool with a graphical user interface (GUI) was developed and provided as a web application.MethodsThe aim was to provide an easy-to-use interface for users without prior programming knowledge to carry out DQ checks and to present the results in a clearly structured way. This interface serves as a starting point for a more detailed investigation of possible DQ irregularities. A user-centered development process ensured the practical feasibility of the interactive GUI. The interface was implemented in the R programming language and aligned to Kahn et al.'s DQ categories conformance, completeness and plausibility.ResultsWith DQAgui, an R package with a web-app frontend for DQ assessment was developed. The GUI allows users to perform DQ analyses of tabular data sets and to systematically evaluate the results. During the development of the GUI, additional features were implemented, such as analyzing a subset of the data by defining time periods and restricting the analyses to certain data elements.ConclusionsAs part of the MIRACUM project, DQAgui is now being used at ten German university hospitals for DQ assessment and to provide a central overview of the availability of important data elements in a datamap over 2 years. Future development efforts should focus on design optimization and include a usability evaluation.
| S-EPMC9367129 | biostudies-literature
Project description:OBJECTIVES:The foot posture index (FPI) is an observational tool designed to measure the position of the foot. The objective of this study was to establish international reference data for foot posture across childhood, and influence of body mass index (BMI) on paediatric foot posture. DESIGN:Cross-sectional study. SETTING AND PARTICIPANTS:The dataset comprised 3217 healthy children, aged from 3 to 15 years. Contributing data were acquired from Spain, UK and Australia. INTERVENTIONS:Foot posture was described by means and z-score of the FPI and the height and weight of each subject was measured and the BMI was calculated. RESULTS:The foot posture of 3217 children were reviewed. A pronated (FPI ≥+6) foot posture was found in 960 (29.8%) children, a normal (FPI 0 to +6) foot posture in 1776 (55.2%) and a highly pronated (FPI +10) foot posture was found in 127 children (3.9%) (range -4 to +12 FPI). Less than 11% were found to have a supinated foot type (n=354). Approximately 20% of children were overweight/obese, but correlation between BMI and FPI was weak and inverse (r=-0.066, p<0.01), refuting the relationship between increased body mass and flatfeet. CONCLUSIONS:This study confirms that the 'flat' or pronated foot is the common foot posture of childhood, with FPI score of +4 (3) the average finding. Trend indicated a less flatfoot with age, although non-linear. A wide normal range of foot posture across childhood is confirmed.
| S-EPMC6500282 | biostudies-other