Dataset Information

Dealing with false positive risk as an indicator of misperceived effectiveness of conservation interventions.

ABSTRACT: As human pressures on the environment continue to spread and intensify, effective conservation interventions are direly needed to prevent threats, reduce conflicts, and recover populations and landscapes in a liaison between science and conservation. It is practically important to discriminate between true and false (or misperceived) effectiveness of interventions as false perceptions may shape a wrong conservation agenda and lead to inappropriate decisions and management actions. This study used the false positive risk (FPR) to estimate the rates of misperceived effectiveness of electric fences (overstated if reported as effective but actually ineffective based on FPR; understated otherwise), explain their causes and propose recommendations on how to improve the representation of true effectiveness. Electric fences are widely applied to reduce damage to fenced assets, such as livestock and beehives, or increase survival of fenced populations. The analysis of 109 cases from 50 publications has shown that the effectiveness of electric fences was overstated in at least one-third of cases, from 31.8% at FPR = 0.2 (20% risk) to 51.1% at FPR = 0.05 (5% risk, true effectiveness). In contrast, understatement reduced from 23.8% to 9.5% at these thresholds of FPR. This means that truly effective applications of electric fences were only 48.9% of all cases reported as effective, but truly ineffective cases were 90.5%, implying that the effectiveness of electric fences was heavily overstated. The main reasons of this bias were the lack of statistical testing or improper reporting of test results (63.3% of cases) and interpretation of marginally significant results (p < 0.05, p < 0.1 and p around 0.05) as indicators of effectiveness (10.1%). In conclusion, FPR is an important tool for estimating true effectiveness of conservation interventions and its application is highly recommended to disentangle true and false effectiveness for planning appropriate conservation actions. Researchers are encouraged to calculate FPR, publish its constituent statistics (especially treatment and control sample sizes) and explicitly provide test results with p values. It is suggested to call the effectiveness "true" if FPR < 0.05, "suggestive" if 0.05 ≤ FPR < 0.2 and "false" if FPR ≥ 0.2.

SUBMITTER: Khorozyan I

PROVIDER: S-EPMC8342041 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundWhen evaluating cancer screening it is important to estimate the cumulative risk of false positives from periodic screening. Because the data typically come from studies in which the number of screenings varies by subject, estimation must take into account dropouts. A previous approach to estimate the probability of at least one false positive in n screenings unrealistically assumed that the probability of dropout does not depend on prior false positives.MethodBy redefining the random variables, we obviate the unrealistic dropout assumption. We also propose a relatively simple logistic regression and extend estimation to the expected number of false positives in n screenings.ResultsWe illustrate our methodology using data from women ages 40 to 64 who received up to four annual breast cancer screenings in the Health Insurance Program of Greater New York study, which began in 1963. Covariates were age, time since previous screening, screening number, and whether or not a previous false positive occurred. Defining a false positive as an unnecessary biopsy, the only statistically significant covariate was whether or not a previous false positive occurred. Because the effect of screening number was not statistically significant, extrapolation beyond 4 screenings was reasonable. The estimated mean number of unnecessary biopsies in 10 years per woman screened is.11 with 95% confidence interval of (.10,.12). Defining a false positive as an unnecessary work-up, all the covariates were statistically significant and the estimated mean number of unnecessary work-ups in 4 years per woman screened is.34 with 95% confidence interval (.32,.36).ConclusionUsing data from multiple cancer screenings with dropouts, and allowing dropout to depend on previous history of false positives, we propose a logistic regression model to estimate both the probability of at least one false positive and the expected number of false positives associated with n cancer screenings. The methodology can be used for both informed decision making at the individual level, as well as planning of health services.

Project description:BackgroundEvidence-based conservation is urgently needed to identify, apply and promote effective interventions for mitigation of threats and recovery of the natural environment. Estimation of intervention effectiveness is subject to robust study design and statistical analysis, and much progress is documented in these fields. In contrast, little is understood about the accuracy and biases (underestimation and overestimation) of different effectiveness metrics and how they are affected by sample size.MethodsIn this study, a dataset (n = 500 cases) consisting of random, positive, integer numbers was simulated to produce frequency input data for the 2 × 2 contingency table. For each case, three metrics of the relative risk, odds ratio and the magnitude of change were calculated, their disparity was estimated and the characteristics of treatment (with intervention) and control (without intervention) samples significantly affecting this disparity were studied by means of linear regression. Further, four case studies from different conservation interventions are provided to support the results.ResultsThe study has shown that the relative risk and the magnitude of change produce identical estimates of intervention effectiveness only when treatment and control samples are equal, or when the number of target outcomes (e.g., number of livestock killed by predators) in treatment sample reaches zero. In other situations, the magnitude of change gives overestimates or underestimates, depending on relationships between treatment and control sample sizes. The table summarizing the conditions of equalities and biases between these two metrics is provided. These conditions are valid for both reduction-aimed interventions reducing negative target outcomes (e.g., livestock protection to reduce livestock losses to predators) and for addition-aimed interventions increasing positive target outcomes (e.g., establishment of protected areas to increase species presence). No significant effects on the odds ratio were found.ConclusionResearchers should set equal treatment and control sample sizes so that to produce identical estimates of intervention effectiveness by the relative risk and the magnitude of change. Otherwise, these estimates are biased if produced by the magnitude of change and the relative risk should be used instead. As setting equal treatment and control samples can be impractical, I encourage researchers and practitioners to use the relative risk in estimation of intervention effectiveness. This will not take additional efforts as both metrics are calculated from the same contingency table. Treatment and control sample sizes, along with their sub-samples affected and not by an intervention, should be explicitly reported by researchers to allow independent evaluation of intervention effectiveness. This approach can help obtain more accurate information on intervention effectiveness for making better decisions in conservation actions.

Project description:Spatial priorities for the conservation of three key Mediterranean habitats, i.e. seagrass Posidonia oceanica meadows, coralligenous formations, and marine caves, were determined through a systematic planning approach. Available information on the distribution of these habitats across the entire Mediterranean Sea was compiled to produce basin-scale distribution maps. Conservation targets for each habitat type were set according to European Union guidelines. Surrogates were used to estimate the spatial variation of opportunity cost for commercial, non-commercial fishing, and aquaculture. Marxan conservation planning software was used to evaluate the comparative utility of two planning scenarios: (a) a whole-basin scenario, referring to selection of priority areas across the whole Mediterranean Sea, and (b) an ecoregional scenario, in which priority areas were selected within eight predefined ecoregions. Although both scenarios required approximately the same total area to be protected in order to achieve conservation targets, the opportunity cost differed between them. The whole-basin scenario yielded a lower opportunity cost, but the Alboran Sea ecoregion was not represented and priority areas were predominantly located in the Ionian, Aegean, and Adriatic Seas. In comparison, the ecoregional scenario resulted in a higher representation of ecoregions and a more even distribution of priority areas, albeit with a higher opportunity cost. We suggest that planning at the ecoregional level ensures better representativeness of the selected conservation features and adequate protection of species, functional, and genetic diversity across the basin. While there are several initiatives that identify priority areas in the Mediterranean Sea, our approach is novel as it combines three issues: (a) it is based on the distribution of habitats and not species, which was rarely the case in previous efforts, (b) it considers spatial variability of cost throughout this socioeconomically heterogeneous basin, and (c) it adopts ecoregions as the most appropriate level for large-scale planning.

Project description:Screening tests are widely recommended for the early detection of disease among asymptomatic individuals. While detecting disease at an earlier stage has the potential to improve outcomes, screening also has negative consequences, including false positive results which may lead to anxiety, unnecessary diagnostic procedures, and increased healthcare costs. In addition, multiple false positive results could discourage participating in subsequent screening rounds. Screening guidelines typically recommend repeated screening over a period of many years, but little prior research has investigated how often individuals receive multiple false positive test results. Estimating the cumulative risk of multiple false positive results over the course of multiple rounds of screening is challenging due to the presence of censoring and competing risks, which may depend on the false positive risk, screening round, and number of prior false positive results. To address the general challenge of estimating the cumulative risk of multiple false positive test results, we propose a nonhomogeneous multistate model to describe the screening process including competing events. We developed alternative approaches for estimating the cumulative risk of multiple false positive results using this multistate model based on existing estimators for the cumulative risk of a single false positive. We compared the performance of the newly proposed models through simulation studies and illustrate model performance using data on screening mammography from the Breast Cancer Surveillance Consortium. Across most simulation scenarios, the multistate extension of a censoring bias model demonstrated lower bias compared to other approaches. In the context of screening mammography, we found that the cumulative risk of multiple false positive results is high. For instance, based on the censoring bias model, for a high-risk individual, the cumulative probability of at least two false positive mammography results after 10 rounds of annual screening is 40.4.

Dataset Information

Dealing with false positive risk as an indicator of misperceived effectiveness of conservation interventions.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets