Methods in health services research. Interpreting the evidence: choosing between randomised and non-randomised studies.
Ontology highlight
ABSTRACT:
SUBMITTER: McKee M
PROVIDER: S-EPMC1126943 | biostudies-literature | 1999 Jul
REPOSITORIES: biostudies-literature
Publications
Methods in health services research. Interpreting the evidence: choosing between randomised and non-randomised studies.
McKee M M Britton A A Black N N McPherson K K Sanderson C C Bain C C
BMJ (Clinical research ed.) 19990701 7205
Similar Datasets
Project description:BackgroundParticipant recruitment is an ongoing challenge in health research. Recruitment may be especially difficult for studies of access to health care because, even among those who are in care, people using services least often also may be hardest to contact and recruit. Opt-out recruitment methods (in which potential participants are given the opportunity to decline further contact about the study (opt out) following an initial mailing, and are then contacted directly if they have not opted out within a specified period) can be used for such studies. However, there is a dearth of literature on the effort needed for effective opt-out recruitment.MethodsIn this paper we describe opt-out recruitment procedures for two studies on access to health care within the U.S. Department of Veterans Affairs. We report resource requirements for recruitment efforts (number of opt-out packets mailed and number of phone calls made). We also compare the characteristics of study participants to potential participants via t-tests, Fisher's exact tests, and chi-squared tests.ResultsRecruitment rates for our two studies were 12 and 21%, respectively. Across multiple study sites, we had to send between 4.3 and 9.2 opt-out packets to recruit one participant. The number of phone calls required to arrive at a final status for each potentially eligible Veteran (i.e. study participation or the termination of recruitment efforts) were 2.9 and 6.1 in the two studies, respectively. Study participants differed as expected from the population of potentially eligible Veterans based on planned oversampling of certain subpopulations. The final samples of participants did not differ statistically from those who were mailed opt-out packets, with one exception: in one of our two studies, participants had higher rates of mental health service use in the past year than did those mailed opt-out packets (64 vs. 47%).ConclusionsOur results emphasize the practicality of using opt-out methods for studies of access to health care. Despite the benefits of these methods, opt-out alone may be insufficient to eliminate non-response bias on key variables. Researchers will need to balance considerations of sample representativeness and feasibility when designing studies investigating access to care.
| S-EPMC5391553 | biostudies-literature
Project description:ObjectiveTo use the experience from a health services research evaluation to provide guidance in team development for mixed methods research.MethodsThe Research Initiative Valuing Eldercare (THRIVE) team was organized by the Robert Wood Johnson Foundation to evaluate The Green House nursing home culture change program. This article describes the development of the research team and provides insights into how funders might engage with mixed methods research teams to maximize the value of the team.ResultsLike many mixed methods collaborations, the THRIVE team consisted of researchers from diverse disciplines, embracing diverse methodologies, and operating under a framework of nonhierarchical, shared leadership that required new collaborations, engagement, and commitment in the context of finite resources. Strategies to overcome these potential obstacles and achieve success included implementation of a Coordinating Center, dedicated time for planning and collaborating across researchers and methodologies, funded support for in-person meetings, and creative optimization of resources.ConclusionsChallenges are inevitably present in the formation and operation of effective mixed methods research teams. However, funders and research teams can implement strategies to promote success.
| S-EPMC3870895 | biostudies-literature
Project description:BackgroundTo describe the ethical issues and experiences of scientists conducting mixed methods health services research and to advance empirical and conceptual discussion on ethical integrity in mixed methods health research.MethodsThe study was conducted with 64 scholars, faculty and consultants from the NIH-funded Mixed Methods Research Training Program (MMRTP) for the Health Sciences. This was a cross-sectional study. Survey results were analyzed using descriptive statistics to characterize responses and open coding to summarize strategies about eight ethical mixed methods research issues. Respondents completed an online survey to elicit experiences related to eight ethical issues (informed consent, confidentiality, data management, burden, safety, equitable recruitment, communication, and dissemination) and strategies for addressing them.ResultsOnly about one-third of respondents thought their research ethics training helped them plan, conduct, or report mixed methods research. The most frequently occurring ethical issues were participant burden, dissemination and equitable recruitment (> 70% endorsement). Despite occurring frequently, < 50% of respondents rated each ethical issue as challenging. The most challenging ethical issues were related to managing participant burden, communication, and dissemination. Strategies reported to address ethical issues were largely not specific or unique to mixed methods with the exception of strategies to mitigate participant burden and, to a lesser degree, to facilitate equitable recruitment and promote dissemination of project results.ConclusionsMixed methods health researchers reported encountering ethical issues often yet varying levels of difficulty and effectiveness in the strategies used to mitigate ethical issues. This study highlights some of the unique challenges faced by mixed methods researchers to plan for and appropriately respond to arising ethical issues such as managing participant burden and confidentiality across data sources and utilizing effective communication and dissemination strategies particularly when working with a multidisciplinary research team. As one of the first empirical studies to examine mixed methods research ethics, our findings highlight the need for greater attention to ethics in health services mixed methods research and training.
| S-EPMC8204431 | biostudies-literature
Project description:Poverty is positively associated with poor health; thus, some healthcare commissioners in the UK have pioneered the introduction of advice services in health service locations. Previous systematic reviews have found little direct evidence for a causal relationship between the provision of advice and physical health and limited evidence for mental health improvement. This paper reports a study using a broader range of types of research evidence to construct a conceptual (logic) model of the wider evidence underpinning potential (rather than only proven) causal pathways between the provision of advice services and improvements in health. Data and discussion from 87 documents were used to construct a model describing interventions, primary outcomes, secondary and tertiary outcomes following advice interventions. The model portrays complex causal pathways between the intervention and various health outcomes; it also indicates the level of evidence for each pathway. It can be used to inform the development of research designed to evaluate the pathways between interventions and health outcomes, which will determine the impact on health outcomes and may explain inconsistencies in previous research findings. It may also be useful to commissioners and practitioners in making decisions regarding development and commissioning of advice services.
| S-EPMC3557712 | biostudies-literature
Project description:BackgroundEvidence from health services research (HSR) is currently thinly spread through many journals, making it difficult for health services researchers, managers and policy-makers to find research on clinical practice guidelines and the appropriateness, process, outcomes, cost and economics of health care services. We undertook to develop and test search terms to retrieve from the MEDLINE database HSR articles meeting minimum quality standards.MethodsThe retrieval performance of 7445 methodologic search terms and phrases in MEDLINE (the test) were compared with a hand search of the literature (the gold standard) for each issue of 68 journal titles for the year 2000 (a total of 25,936 articles). We determined sensitivity, specificity and precision (the positive predictive value) of the MEDLINE search strategies.ResultsA majority of the articles that were classified as outcome assessment, but fewer than half of those in the other categories, were considered methodologically acceptable (no methodologic criteria were applied for cost studies). Combining individual search terms to maximize sensitivity, while keeping specificity at 50% or more, led to sensitivities in the range of 88.1% to 100% for several categories (specificities ranged from 52.9% to 97.4%). When terms were combined to maximize specificity while keeping sensitivity at 50% or more, specificities of 88.8% to 99.8% were achieved. When terms were combined to maximize sensitivity and specificity while minimizing the differences between the 2 measurements, most strategies for HSR categories achieved sensitivity and specificity of at least 80%.InterpretationSensitive and specific search strategies were validated for retrieval of HSR literature from MEDLINE. These strategies have been made available for public use by the US National Library of Medicine at www.nlm.nih.gov/nichsr/hedges/search.html.
| S-EPMC524948 | biostudies-literature
Project description:Methodologically sound mixed methods research can improve our understanding of health services by providing a more comprehensive picture of health services than either method can alone. This study describes the frequency of mixed methods in published health services research and compares the presence of methodological components indicative of rigorous approaches across mixed methods, qualitative, and quantitative articles.All empirical articles (n = 1,651) published between 2003 and 2007 from four top-ranked health services journals.All mixed methods articles (n = 47) and random samples of qualitative and quantitative articles were evaluated to identify reporting of key components indicating rigor for each method, based on accepted standards for evaluating the quality of research reports (e.g., use of p-values in quantitative reports, description of context in qualitative reports, and integration in mixed method reports). We used chi-square tests to evaluate differences between article types for each component.Mixed methods articles comprised 2.85 percent (n = 47) of empirical articles, quantitative articles 90.98 percent (n = 1,502), and qualitative articles 6.18 percent (n = 102). There was a statistically significant difference (?(2) (1) = 12.20, p = .0005, Cramer's V = 0.09, odds ratio = 1.49 [95% confidence interval = 1,27, 1.74]) in the proportion of quantitative methodological components present in mixed methods compared to quantitative papers (21.94 versus 47.07 percent, respectively) but no statistically significant difference (?(2) (1) = 0.02, p = .89, Cramer's V = 0.01) in the proportion of qualitative methodological components in mixed methods compared to qualitative papers (21.34 versus 25.47 percent, respectively).Few published health services research articles use mixed methods. The frequency of key methodological components is variable. Suggestions are provided to increase the transparency of mixed methods studies and the presence of key methodological components in published reports.
| S-EPMC3419885 | biostudies-literature
Project description:BackgroundResearch on pressing health services and policy issues requires access to complete, accurate, and timely patient and organizational data.AimThis paper describes how administrative and health records (including electronic medical records) can be linked for comparative effectiveness and health services research.Materials and methodsWe categorize the major agents (i.e., who owns and controls data and who carries out the data linkage) into three areas: (1) individual investigators; (2) government sponsored linked data bases; and (3) public-private partnerships that facilitate linkage of data owned by private organizations. We describe challenges that may be encountered in the linkage process, and the benefits of combining secondary databases with primary qualitative and quantitative sources. We use cancer care research to illustrate our points.ResultsTo fill the gaps in the existing data infrastructure, additional steps are required to foster collaboration among institutions, researchers, and public and private components of the health care sector. Without such effort, independent researchers, governmental agencies, and nonprofit organizations are likely to continue building upon a fragmented and costly system with limited access. Discussion. Without the development and support for emerging information technologies across multiple health care settings, the potential for data collected for clinical and transactional purposes to benefit the research community and, ultimately, the patient population may go unrealized.ConclusionThe current environment is characterized by budget and technical challenges, but investments in data infrastructure are arguably cost-effective given the need to reform our health care system and to monitor the impact of health reform initiatives.
| S-EPMC2965887 | biostudies-literature
Project description:ObjectiveTo investigate potential bias in the use of the conventional linear instrumental variables (IV) method for the estimation of causal effects in inherently nonlinear regression settings.Data sourcesSmoking Supplement to the 1979 National Health Interview Survey, National Longitudinal Alcohol Epidemiologic Survey, and simulated data.Study designPotential bias from the use of the linear IV method in nonlinear models is assessed via simulation studies and real world data analyses in two commonly encountered regression setting: (1) models with a nonnegative outcome (e.g., a count) and a continuous endogenous regressor; and (2) models with a binary outcome and a binary endogenous regressor.Principal findingsThe simulation analyses show that substantial bias in the estimation of causal effects can result from applying the conventional IV method in inherently nonlinear regression settings. Moreover, the bias is not attenuated as the sample size increases. This point is further illustrated in the survey data analyses in which IV-based estimates of the relevant causal effects diverge substantially from those obtained with appropriate nonlinear estimation methods.ConclusionsWe offer this research as a cautionary note to those who would opt for the use of linear specifications in inherently nonlinear settings involving endogeneity.
| S-EPMC2442231 | biostudies-literature
Project description:As educators seek confirmation of successful trainee achievement, medical education must move toward a more evidence-based approach to teaching and evaluation. Although medical training often provides physicians with a general background in biostatistics, many are not prepared to apply these skills. This can hinder clinician educators as they wish to develop, analyze and disseminate their scholarly work. This paper is intended to be a concise educational tool and guide for choosing and interpreting statistical tests aimed toward medical education assessment. It includes guidelines and examples that clinician-educators can use when analyzing and interpreting studies and when writing methods and results sections of reports.
| S-EPMC1924630 | biostudies-literature
Project description:BackgroundPremature discontinuation of clinical studies affects about 25% of randomised controlled trials (RCTs) which raises concerns about waste of scarce resources for research. The risk of discontinuation of non-randomised prospective studies (NPSs) is yet unclear.ObjectivesTo compare the proportion of discontinued studies between NPSs and RCTs that received ethical approval.MethodsWe systematically surveyed prospective longitudinal clinical studies that were approved by a single REC in Freiburg, Germany between 2000 and 2002. We collected study characteristics, identified subsequent publications, and surveyed investigators to elucidate whether a study was discontinued and, if so, why.ResultsOf 917 approved studies, 547 were prospective longitudinal studies (306 RCTs and 241 NPSs). NPSs were on average smaller than RCTs, more frequently single centre and pilot studies, and less frequently funded by industry. NPSs were less frequently discontinued than RCTs: 32/221 (14%) versus 78/288 (27%, p<0.001, missing data excluded). Poor recruitment was the most frequent reason for discontinuation in both NPSs (36%) and RCTs (37%).ConclusionsCompared to RCTs, NPSs were at lower risk for discontinuation. Measures to reliably predict, sustain, and stimulate recruitment could prevent discontinuation of many RCTs but also of some NPSs.
| S-EPMC5085068 | biostudies-literature