Dataset Information

How to detect high-performing individuals and groups: Decision similarity predicts accuracy.

ABSTRACT: Distinguishing between high- and low-performing individuals and groups is of prime importance in a wide range of high-stakes contexts. While this is straightforward when accurate records of past performance exist, these records are unavailable in most real-world contexts. Focusing on the class of binary decision problems, we use a combined theoretical and empirical approach to develop and test a approach to this important problem. First, we use a general mathematical argument and numerical simulations to show that the similarity of an individual's decisions to others is a powerful predictor of that individual's decision accuracy. Second, testing this prediction with several large datasets on breast and skin cancer diagnostics, geopolitical forecasting, and a general knowledge task, we find that decision similarity robustly permits the identification of high-performing individuals and groups. Our findings offer a simple, yet broadly applicable, heuristic for improving real-world decision-making systems.

SUBMITTER: Kurvers RHJM

PROVIDER: S-EPMC6957221 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

How to detect high-performing individuals and groups: Decision similarity predicts accuracy.

Kurvers R H J M RHJM Herzog S M SM Hertwig R R Krause J J Moussaid M M Argenziano G G Zalaudek I I Carney P A PA Wolf M M

Science advances 20191120 11

Distinguishing between high- and low-performing individuals and groups is of prime importance in a wide range of high-stakes contexts. While this is straightforward when accurate records of past performance exist, these records are unavailable in most real-world contexts. Focusing on the class of binary decision problems, we use a combined theoretical and empirical approach to develop and test a approach to this important problem. First, we use a general mathematical argument and numerical simul ...[more]

PMID: 31976366

Similar Datasets

Project description:BACKGROUND:Genotype imputation is a common technique in genetic research. Genetic similarity between target population and reference dataset is crucial for high-quality results. Although several reference panels are available, it is often not clear which is the most optimal for a particular target dataset to be imputed. Maximizing genetic similarity between study sample and intended reference panels may be the straight forward method for selecting the genetically best-matched reference. However, the impact of genetic similarity on imputation accuracy has not yet been studied in detail. RESULTS:We performed a simulation study in 20 ethnic groups obtained from POPRES. High-quality SNPs were masked and re-imputed with MaCH, MaCH-minimac and IMPUTE2 using four different HapMap reference panels (CEU, CHB-JPT, MEX and YRI). Imputation accuracy was assessed by different statistics. Genetic similarity between ethnic groups and reference populations were measured by F -statistics (F(ST)) originally proposed by Wright and G -statistics (G(ST)) introduced by Nei and others. To assess the predictive power of these measures regarding imputation accuracy, we analysed relations between them and corresponding imputation accuracy scores. We found that population genetic distances between homogeneous reference and target populations were strongly linearly correlated with resulting imputation accuracies irrespective of considered distance measure, imputation accuracy measure, missingness and imputation software used. Possible exception was African population. CONCLUSION:Usage of G(ST) or F(ST)-related measures for predicting the optimal reference panel for imputation frameworks relying on a specific reference is highly recommended. A cut-off of G(ST) < 0.01 is recommended to achieve good imputation results for high-frequency variants and small data sets. The linear relationship is less pronounced for low-frequency variants for which we also observed a dependence of imputation accuracy on the number of polymorphic sites in the reference. We also show that the software specific measures MaCH-Rsq and IMPUTE-info must be interpreted with caution if the genetic distance of target and reference population is high.

Project description:BackgroundTreatment with radiolabeled ligands to prostate-specific membrane antigen (PSMA) is gaining importance in the treatment of patients with advanced prostate carcinoma. Previous imaging with positron emission tomography/computed tomography (PET/CT) is mandatory. The aim of this study was to investigate the role of radiomics features in PSMA-PET/CT scans and clinical parameters to predict response to 177Lu-PSMA treatment given just baseline PSMA scans using state-of-the-art machine learning (ML) methods.MethodsA total of 2,070 pathological hotspots annotated in 83 prostate cancer patients undergoing PSMA therapy were analyzed. Two main tasks are performed: (I) analyzing correlation of averaged (per patient) values of radiomics features of individual hotspots and clinical parameters with difference in prostate specific antigen levels (ΔPSA) in pre- and post-therapy as a therapy response indicator. (II) ML-based classification of patients into responders and non-responders based on averaged features values and clinical parameters. To achieve this, machine learning (ML) algorithms and linear regression tests are applied. Grid search, cross validation (CV) and permutation test were performed to assure that the results were significant.ResultsRadiomics features (PET_Min, PET_Correlation, CT_Min, CT_Busyness and CT_Coarseness) and clinical parameters such as Alp1 and Gleason score showed best correlations with ΔPSA. For the treatment response prediction task, 80% area under the curve (AUC), 75% sensitivity (SE), and 75% specificity (SP) were obtained, applying ML support vector machine (SVM) classifier with radial basis function (RBF) kernel on a selection of radiomics features and clinical parameters with strong correlations with ΔPSA.ConclusionsMachine learning based on 68Ga-PSMA PET/CT radiomics features holds promise for the prediction of response to 177Lu-PSMA treatment, given only base-line 68Ga-PSMA scan. In addition, it was shown that, the best correlating set of radiomics features with ΔPSA are superior to clinical parameters for this therapy response prediction task using ML classifiers.

Project description:IntroductionPractice effects associated with the repeated administration of cognitive tests often confound true therapeutic or experimental effects. Alternate test forms help reduce practice effects, but generating stimulus sets with identical properties can be difficult. The main objective of this study was to disentangle practice and stimulus set effects for Cognition, a battery of 10 brief cognitive tests specifically designed for high-performing populations with 15 unique versions for repeated testing. A secondary objective was to investigate the effects of test-retest interval on practice effects.MethodsThe 15 versions of Cognition were administered in three groups of 15-16 subjects (total N = 46, mean±SD age 32.5 ± 7.2 years, range 25-54 years, 23 male) in a randomized but balanced fashion with administration intervals of ≥10 days, ≤5 days, or 4 times per day. Mixed effect models were used to investigate linear and logarithmic trends across repeated administrations in key speed and accuracy outcomes, whether these trends differed significantly between administration interval groups, and whether stimulus sets differed significantly in difficulty.ResultsProtracted, non-linear practice effects well beyond the second administration were observed for most of the 10 Cognition tests both in accuracy and speed, but test-retest administration interval significantly affected practice effects only for 3 out of the 10 tests and only in the speed domain. Stimulus set effects were observed for the 6 Cognition tests that use unique sets of stimuli. Factors were established that allow for correcting for both practice and stimulus set effects.ConclusionsPractice effects are pronounced and probably under-appreciated in cognitive testing. The correction factors established in this study are a unique feature of the Cognition battery that can help avoid masking practice effects, address noise generated by differences in stimulus set difficulty, and facilitate interpretation of results from studies with repeated assessments.

Dataset Information

How to detect high-performing individuals and groups: Decision similarity predicts accuracy.

Publications

How to detect high-performing individuals and groups: Decision similarity predicts accuracy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets