Dataset Information

Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity.

ABSTRACT: BACKGROUND: Controlled clinical trials of health care interventions are either explanatory or pragmatic. Explanatory trials test whether an intervention is efficacious; that is, whether it can have a beneficial effect in an ideal situation. Pragmatic trials measure effectiveness; they measure the degree of beneficial effect in real clinical practice. In pragmatic trials, a balance between external validity (generalizability of the results) and internal validity (reliability or accuracy of the results) needs to be achieved. The explanatory trial seeks to maximize the internal validity by assuring rigorous control of all variables other than the intervention. The pragmatic trial seeks to maximize external validity to ensure that the results can be generalized. However the danger of pragmatic trials is that internal validity may be overly compromised in the effort to ensure generalizability. We are conducting two pragmatic randomized controlled trials on interventions in the management of hypertension in primary care. We describe the design of the trials and the steps taken to deal with the competing demands of external and internal validity. DISCUSSION: External validity is maximized by having few exclusion criteria and by allowing flexibility in the interpretation of the intervention and in management decisions. Internal validity is maximized by decreasing contamination bias through cluster randomization, and decreasing observer and assessment bias, in these non-blinded trials, through baseline data collection prior to randomization, automating the outcomes assessment with 24 hour ambulatory blood pressure monitors, and blinding the data analysis. SUMMARY: Clinical trials conducted in community practices present investigators with difficult methodological choices related to maintaining a balance between internal validity (reliability of the results) and external validity (generalizability). The attempt to achieve methodological purity can result in clinically meaningless results, while attempting to achieve full generalizability can result in invalid and unreliable results. Achieving a creative tension between the two is crucial.

SUBMITTER: Godwin M

PROVIDER: S-EPMC317298 | biostudies-literature | 2003 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity.

Godwin Marshall M Ruhland Lucia L Casson Ian I MacDonald Susan S Delva Dianne D Birtwhistle Richard R Lam Miu M Seguin Rachelle R

BMC medical research methodology 20031222

<h4>Background</h4>Controlled clinical trials of health care interventions are either explanatory or pragmatic. Explanatory trials test whether an intervention is efficacious; that is, whether it can have a beneficial effect in an ideal situation. Pragmatic trials measure effectiveness; they measure the degree of beneficial effect in real clinical practice. In pragmatic trials, a balance between external validity (generalizability of the results) and internal validity (reliability or accuracy of ...[more]

PMID: 14690550

Similar Datasets

Project description:ObjectivesTo assess aspects of the internal validity of recently published cluster randomised trials and explore the reporting of information useful in assessing the external validity of these trials.DesignReview of 34 cluster randomised trials in primary care published in 2004 and 2005 in seven journals (British Medical Journal, British Journal of General Practice, Family Practice, Preventive Medicine, Annals of Internal Medicine, Journal of General Internal Medicine, Pediatrics).Data sourcesNational Library of Medicine (Medline) via PubMed.Data extractionTo assess aspects of internal validity we extracted data on appropriateness of sample size calculations and analyses, methods of identifying and recruiting individual participants, and blinding. To explore reporting of information useful in assessing external validity we extracted data on cluster eligibility, cluster inclusion and retention, cluster generalisability, and the feasibility and acceptability of the intervention to health providers in clusters.Results21 (62%) trials accounted for clustering in sample size calculations and 30 (88%) in the analysis; about a quarter were potentially biased because of procedures surrounding recruitment and identification of patients; individual participants were blind to allocation status in 19 (56%) and outcome assessors were blind in 15 (44%). In almost half the reports, information relating to generalisability of clusters was poorly reported, and in two fifths there was no information about the feasibility and acceptability of the intervention.ConclusionsCluster randomised trials are essential for evaluating certain types of interventions. Issues affecting their internal validity, such as appropriate sample size calculations and analysis, have been widely disseminated and are now better addressed by researchers. Blinding of those identifying and recruiting patients to allocation status is recommended but is not always carried out. There may be fewer barriers to internal validity in trials in which individual participants are not recruited. External validity seems poorly addressed in many trials, yet is arguably as important as internal validity in judging quality as a basis for healthcare intervention.

Project description:Depressive disorders and symptoms affect more than one-third of primary care patients, many of whom do not receive or do not complete treatment. Internet-based social support from peers could sustain depression treatment engagement and adherence. We do not know whether primary care patients will accept referral to such websites nor do we know which methods of referral would be most effective.We conducted a randomized clinical trial to determine whether (1) a simple generic referral card (control), (2) a patient-oriented brochure that provided examples of online postings and experience (internal motivation), or (3) a physician letter of recommendation (external motivation) would generate the greatest participation in a primary care Internet depression treatment support portal focused around an Internet support group (ISG).We used 3 offline methods to identify potential participants who had not used an ISG in the past 6 months. Eligibility was determined in part by a brief structured psychiatric interview based on the Patient Health Questionnaire-9 (PHQ-9). After consent and enrollment, participants were randomly assigned to 1 of 3 groups (control, internal motivation, or external motivation). We constructed a portal to connect primary care patients to both fact-based information and an established ISG (Psycho-Babble). The ISG allowed participants to view messages and then decide if they actually wished to register there. Participation in the portal and the ISG was assessed via automated activity tracking.Fifty participants were assigned to the 3 groups: a motivation-neutral control group (n=18), an internal motivation group (n=19), and an external motivation group (n=13). Of these participants, 31 (62%) visited the portal; 27 (54%) visited the ISG itself. The internal motivation group showed significantly greater participation than the control group on several measures. The external motivation group spent significantly less time logged onto the portal than the control group. The internal motivation group showed significantly greater participation than the external motivation group on several measures.Referral of primary care patients with depressive disorders and symptoms to an ISG is feasible even if they have never previously used one. This may best be accomplished by enhancing their internal motivation.

Dataset Information

Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity.

Publications

Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets