Dataset Information

RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

ABSTRACT:

Background

The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates). For this, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors will be conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methods is to assess risk of bias (RoB) of individual studies. In this article, we present and describe the development of such a tool, called the Risk of Bias in Studies estimating Prevalence of Exposure to Occupational risk factors (RoB-SPEO) tool; report results from RoB-SPEO's pilot testing; note RoB-SPEO's limitations; and suggest how the tool might be tested and developed further.

Methods

Selected existing RoB tools used in environmental and occupational health systematic reviews were reviewed and analysed. From existing tools, we identified domains for the new tool and, if necessary, added new domains. For each domain, we then identified and integrated components from the existing tools (i.e. instructions, domains, guiding questions, considerations, ratings and rating criteria), and, if necessary, we developed new components. Finally, we elicited feedback from other systematic review methodologists and exposure scientists and agreed upon RoB-SPEO. Nine experts pilot tested RoB-SPEO, and we calculated a raw measure of inter-rater agreement (P_i) for each of its domain, rating P_i?i???0.8 as substantial and P_i?>?0.80 as almost perfect agreement.

Results

Our review found no standard tool for assessing RoB in prevalence studies of exposure to occupational risk factors. We identified six existing tools for environmental and occupational health systematic reviews and found that their components for assessing RoB differ considerably. With the new RoB-SPEO tool, assessors judge RoB for each of eight domains: (1) bias in selection of participants into the study; (2) bias due to a lack of blinding of study personnel; (3) bias due to exposure misclassification; (4) bias due to incomplete exposure data; (5) bias due to conflict of interest; (6) bias due to selective reporting of exposures; (7) bias due to difference in numerator and denominator; and (8) other bias. The RoB-SPEO's ratings are low, probably low, probably high, high or no information. Pilot testing of the RoB-SPEO tool found substantial inter-rater agreement for six domains (range of P_i for these domains: 0.51-0.80), but poor agreement for two domains (i.e. P_i of 0.31 and 0.33 for biases due to incomplete exposure data and in selection of participants into the study, respectively). Limitations of RoB-SPEO include that it has not yet been fully performance-tested.

Conclusions

We developed the RoB-SPEO tool for assessing RoB in prevalence studies of exposure to occupational risk factors. The tool will be applied and its performance tested in the ongoing systematic reviews for the WHO/ILO Joint Estimates.

SUBMITTER: Pega F

PROVIDER: S-EPMC7479507 | biostudies-literature | 2020 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

Pega Frank F Norris Susan L SL Backes Claudine C Bero Lisa A LA Descatha Alexis A Gagliardi Diana D Godderis Lode L Loney Tom T Modenese Alberto A Morgan Rebecca L RL Pachito Daniela D Paulo Marilia B S MBS Scheepers Paul T J PTJ Schlünssen Vivi V Sgargi Daria D Silbergeld Ellen K EK Sørensen Kathrine K Sutton Patrice P Tenkate Thomas T Torreão Corrêa da Silva Denise D Ujita Yuka Y van Deventer Emilie E Woodruff Tracey J TJ Mandrioli Daniele D

Environment international 20191218

<h4>Background</h4>The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates). For this, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors will be conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methods is to assess risk of bias (RoB) of indi ...[more]

PMID: 31864023

Similar Datasets

Project description:BackgroundAs part of the development of the World Health Organization (WHO)/International Labour Organization (ILO) Joint Estimates of the Work-related Burden of Disease and Injury, WHO and ILO carried out several systematic reviews to determine the prevalence of exposure to selected occupational risk factors. Risk of bias assessment for individual studies is a critical step of a systematic review. No tool existed for assessing the risk of bias in prevalence studies of exposure to occupational risk factors, so WHO and ILO developed and pilot tested the RoB-SPEO tool for this purpose. Here, we investigate the assessor burden, inter-rater agreement, and user experience of this new instrument, based on the abovementioned WHO/ILO systematic reviews.MethodsTwenty-seven individual experts applied RoB-SPEO to assess risk of bias. Four systematic reviews provided a total of 283 individual assessments, carried out for 137 studies. For each study, two or more assessors independently assessed risk of bias across the eight RoB-SPEO domains selecting one of RoB-SPEO's six ratings (i.e., "low", "probably low", "probably high", "high", "unclear" or "cannot be determined"). Assessors were asked to report time taken (i.e. indicator of assessor burden) to complete each assessment and describe their user experience. To gauge assessor burden, we calculated the median and inter-quartile range of times taken per individual risk of bias assessment. To assess inter-rater reliability, we calculated a raw measure of inter-rater agreement (Pi) for each RoB-SPEO domain, between Pi = 0.00, indicating no agreement and Pi = 1.00, indicating perfect agreement. As subgroup analyses, Pi was also disaggregated by systematic review, assessor experience with RoB-SPEO (≤10 assessments versus > 10 assessments), and assessment time (tertiles: ≤25 min versus 26-66 min versus ≥ 67 min). To describe user experience, we synthesised the assessors' comments and recommendations.ResultsAssessors reported a median of 40 min to complete one assessment (interquartile range 21-120 min). For all domains, raw inter-rater agreement ranged from 0.54 to 0.82. Agreement varied by systematic review and assessor experience with RoB-SPEO between domains, and increased with increasing assessment time. A small number of users recommended further development of instructions for selected RoB-SPEO domains, especially bias in selection of participants into the study (domain 1) and bias due to differences in numerator and denominator (domain 7).DiscussionOverall, our results indicated good agreement across the eight domains of the RoB-SPEO tool. The median assessment time was comparable to that of other risk of bias tools, indicating comparable assessor burden. However, there was considerable variation in time taken to complete assessments. Additional time spent on assessments may improve inter-rater agreement. Further development of the RoB-SPEO tool could focus on refining instructions for selected RoB-SPEO domains and additional testing to assess agreement for different topic areas and with a wider range of assessors from different research backgrounds.

Project description:ObjectivesBurden of disease estimation commonly requires estimates of the population exposed to a risk factor over a time window (yeart to yeart+n). We present a microsimulation modelling approach for producing such estimates and apply it to calculate the population exposed to long working hours for one country (Italy).MethodsWe developed a three-model approach: Model 1, a multilevel model, estimates exposure to the risk factor at the first year of the time window (yeart). Model 2, a regression model, estimates transition probabilities between exposure categories during the time window (yeart to yeart+n). Model 3, a microsimulation model, estimates the exposed population over the time window, using the Monte Carlo method. The microsimulation is carried out in three steps: (a) a representative synthetic population is initiated in the first year of the time window using prevalence estimates from Model 1, (b) the exposed population is simulated over the time window using the transition probabilities from Model 2; and (c) the population is censored for deaths during the time window.ResultsWe estimated the population exposed to long working hours (i.e. 41-48, 49-54 and ≥55 hours/week) over a 10-year time window (2002-11) in Italy. We populated all three models with official data from Labour Force Surveys, United Nations population estimates and World Health Organization life tables. Estimates were produced of populations exposed over the time window, disaggregated by sex and 5-year age group.ConclusionsOur modelling approach for estimating the population exposed to a risk factor over a time window is simple, versatile, and flexible. It however requires longitudinal exposure data and Model 3 (the microsimulation model) is stochastic. The approach can improve accuracy and transparency in exposure and burden of disease estimations. To improve the approach, a logical next step is changing Model 3 to a deterministic microsimulation method, such as modelling of microflows.

Project description:BackgroundThe World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates), with contributions from a large network of experts. Welding fumes have been classified as carcinogenic to humans (Group 1) by the International Agency for Research on Cancer (IARC); this assessment found sufficient evidence from studies in humans that welding fumes are a cause of lung cancer. In this article, we present the protocol for a systematic review of parameters for estimating the number of deaths and disability-adjusted life years from trachea, bronchus and lung cancer attributable to occupational exposure to welding fumes, to inform the development of the WHO/ILO Joint Estimates.ObjectivesWe aim to systematically review and meta-analyse estimates of the effect of occupational exposure to welding fumes on trachea, bronchus and lung cancer, applying the Navigation Guide systematic review methodology as an organizing framework.Data sourcesWe will search electronic bibliographic databases for potentially relevant records from published and unpublished studies, including Medline, EMBASE, Web of Science, and CISDOC. We will also search electronic grey literature databases, Internet search engines and organizational websites; hand search reference list of previous systematic reviews and included study records; and consult additional experts.Study eligibility and criteriaWe will include working-age (?15 years) workers in the formal and informal economy in any Member State of WHO and/or ILO but exclude children (<15 years) and unpaid domestic workers. The eligible risk factor will be occupational exposure to welding fumes, measured directly or indirectly (i.e., through proxy of relevant occupation, work task, job-exposure matrix, expert judgment or self-report). The eligible outcomes will be trachea, bronchus and lung cancer. We will include randomized controlled trials, cohort studies, case-control studies and other non-randomized intervention studies with an estimate of the relative effect of any occupational exposure to welding fumes on the prevalence of, incidence of or mortality from trachea, bronchus and lung cancer, compared with the theoretical minimum risk exposure level of no occupational exposure to welding fumes.Study appraisal and synthesis methodsAt least two review authors will independently screen titles and abstracts against the eligibility criteria at a first stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. Two or more review authors will assess risk of bias and the quality of evidence, using the Navigation Guide tool or approach. If feasible, we will combine relative risks using meta-analysis. We will report results using the preferred reporting items for systematic reviews and meta-analyses guidelines (PRISMA).

Project description:BackgroundThe World Health Organization (WHO) and the International Labour Organization (ILO) are developing a joint methodology for estimating the national and global work-related burden of disease and injury (WHO/ILO joint methodology), with contributions from a large network of experts. In this paper, we present the protocol for two systematic reviews of parameters for estimating the number of disability-adjusted life years from osteoarthritis of hip or knee, and selected other musculoskeletal diseases respectively, attributable to exposure to occupational ergonomic risk factors to inform the development of the WHO/ILO joint methodology.ObjectivesWe aim to systematically review studies on exposure to occupational ergonomic risk factors (Systematic Review 1) and systematically review and meta-analyze estimates of the effect of exposure to occupational ergonomic risk factors on osteoarthritis of the hip or knee, and selected other musculoskeletal diseases respectively (Systematic Review 2), applying the Navigation Guide systematic review methodology as an organizing framework, conducting both systematic reviews in tandem and in a harmonized way.Data sourcesSeparately for Systematic Reviews 1 and 2, we will search electronic academic databases for potentially relevant records from published and unpublished studies, including Medline, EMBASE, Web of Science and CISDOC. We will also search electronic grey literature databases, Internet search engines and organizational websites; hand-search reference lists of previous systematic reviews and included study records; and consult additional experts.Study eligibility and criteriaWe will include working-age (≥15 years) workers in the formal and informal economy in any WHO and/or ILO Member State, but exclude children (<15 years) and unpaid domestic workers. The included occupational ergonomic risk factors will be any exposure to one or more of: force exertion; demanding posture; repetitiveness; hand-arm vibration; lifting; kneeling and/or squatting; and climbing. Included outcomes will be (i) osteoarthritis and (ii) other musculoskeletal diseases (i.e., one or more of: rotator cuff syndrome; bicipital tendinitis; calcific tendinitis; shoulder impingement; bursitis shoulder; epicondylitis medialis; epicondylitis lateralis; bursitis elbow; bursitis hip; chondromalacia patellae; meniscus disorders; and/or bursitis knee). For Systematic Review 1, we will include quantitative prevalence studies of any exposure to occupational ergonomic risk factors stratified by country, gender, age and industrial sector or occupation. For Systematic Review 2, we will include randomized controlled trials, cohort studies, case-control-studies and other non-randomized intervention studies with an estimate of the relative effect of any exposure with occupational ergonomic risk factors on the prevalence or incidence of osteoarthritis and/or selected musculoskeletal diseases, compared with the theoretical minimum risk exposure level (i.e., no exposure).Study appraisal and synthesis methodsAt least two review authors will independently screen titles and abstracts against the eligibility criteria at a first stage and full texts of potentially eligible records at a second stage, followed by extraction of data from qualifying studies. At least two review authors will assess risk of bias and the quality of evidence, using the most suited tools currently available. For Systematic Review 2, if feasible, we will combine relative risks using meta-analysis. We will report results using the guidelines for accurate and transparent health estimates reporting (GATHER) for Systematic Review 1 and the preferred reporting items for systematic reviews and meta-analyses guidelines (PRISMA) for Systematic Review 2. PROSPERO registration number: CRD42018102631.

Dataset Information

RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

Background

Methods

Results

Conclusions

Publications

RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets