Item Responses in Quantity-Frequency Questionnaires: Implications for Data Generalizability.
Ontology highlight
ABSTRACT:
SUBMITTER: Stevens JE
PROVIDER: S-EPMC8351754 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:PurposeMeaningful thresholds are needed to interpret patient-reported outcome measure (PROM) results. This paper introduces a new method, based on item response theory (IRT), to estimate such thresholds. The performance of the method is examined in simulated datasets and two real datasets, and compared with other methods.MethodsThe IRT method involves fitting an IRT model to the PROM items and an anchor item indicating the criterion state of interest. The difficulty parameter of the anchor item represents the meaningful threshold on the latent trait. The latent threshold is then linked to the corresponding expected PROM score. We simulated 4500 item response datasets to a 10-item PROM, and an anchor item. The datasets varied with respect to the mean and standard deviation of the latent trait, and the reliability of the anchor item. The real datasets consisted of a depression scale with a clinical depression diagnosis as anchor variable and a pain scale with a patient acceptable symptom state (PASS) question as anchor variable.ResultsThe new IRT method recovered the true thresholds accurately across the simulated datasets. The other methods, except one, produced biased threshold estimates if the state prevalence was smaller or greater than 0.5. The adjusted predictive modeling method matched the new IRT method (also in the real datasets) but showed some residual bias if the prevalence was smaller than 0.3 or greater than 0.7.ConclusionsThe new IRT method perfectly recovers meaningful (interpretational) thresholds for multi-item questionnaires, provided that the data satisfy the assumptions for IRT analysis.
| S-EPMC10172229 | biostudies-literature
Project description:The Monetary Choice Questionnaire (MCQ) is a widely used behavioral task that measures the rate of delay discounting (i.e., k), the degree to which a delayed reward loses its present value as a function of the time to its receipt. Both 21- and 27-item MCQs have been extensively validated and proven valuable in research. Different methods have been developed to streamline MCQ scoring. However, existing scoring methods have yet to tackle the issue of missing responses or provide clear guidance on imputing such data. Due to this lack of knowledge, the present study developed and compared three imputation approaches that leverage the MCQ's structure and prioritize ease of implementation. Additionally, their performance was compared with mode imputation. A Monte Carlo simulation was conducted to evaluate the performance of these approaches in handling various missing responses in each observation across two datasets from prior studies that employed the 21- and 27-item MCQs. One of the three approaches consistently outperformed mode imputation across all performance measures. This approach involves imputing missing values using congruent non-missing responses to the items corresponding to the same k value or introducing random responses when congruent answers are unavailable. This investigation unveils a straightforward method for imputing missing data in the MCQ while ensuring unbiased estimates. Along with the investigation, an R tool was developed for researchers to implement this strategy while streamlining the MCQ scoring process.
| S-EPMC10578570 | biostudies-literature
Project description:Comparison-shopping applications are widespread and have been the subject of considerable research and development. There has also been widespread recognition that people are predictably irrational when making shopping decisions. In this work, we combine these two facts to propose a new type of predicable irrational behavior that has important implications for comparison-shopping applications that now utilize crowdsourcing to increase the information provided about sellers in these electronic marketplaces. In a series of three studies we demonstrate that, even after controlling for relative and absolute savings, the number of items in a shopping trip is an important consideration in the decision to make a trip to more than one store. This is true of both actual trips in physical shopping in the real world, and virtual trips to other vendors in online shopping. We term this effect quantity bias.
| S-EPMC8815983 | biostudies-literature
Project description:Objective: We examined the interrater reliability and generalizability of high-frequency oscillation (HFO) visual evaluations in the ripple (80-250 Hz) band, and established a framework for the transition of HFO analysis to routine clinical care. We were interested in the interrater reliability or epoch generalizability to describe how similar the evaluations were between reviewers, and in the reviewer generalizability to represent the consistency of the internal threshold each individual reviewer. Methods: We studied 41 adult epilepsy patients (mean age: 35.6 years) who underwent intracranial electroencephalography. A morphology detector was designed and used to detect candidate HFO events, lower-threshold events, and distractor events. These events were subsequently presented to six expert reviewers, who visually evaluated events for the presence of HFOs. Generalizability theory was used to characterize the epoch generalizability (interrater reliability) and reviewer generalizability (internal threshold consistency) of visual evaluations, as well as to project the numbers of epochs, reviewers, and datasets required to achieve strong generalizability (threshold of 0.8). Results: The reviewer generalizability was almost perfect (0.983), indicating there were sufficient evaluations to determine the internal threshold of each reviewer. However, the interrater reliability for 6 reviewers (0.588) and pairwise interrater reliability (0.322) were both poor, indicating that the agreement of 6 reviewers is insufficient to reliably establish the presence or absence of individual HFOs. Strong interrater reliability (≥0.8) was projected as requiring a minimum of 17 reviewers, while strong reviewer generalizability could be achieved with <30 epoch evaluations per reviewer. Significance: This study reaffirms the poor reliability of using small numbers of reviewers to identify HFOs, and projects the number of reviewers required to overcome this limitation. It also provides a set of tools which may be used for training reviewers, tracking changes to interrater reliability, and for constructing a benchmark set of epochs that can serve as a generalizable gold standard, against which other HFO detection algorithms may be compared. This study represents an important step toward the reconciliation of important but discordant findings from HFO studies undertaken with different sets of HFOs, and ultimately toward transitioning HFO analysis into a meaningful part of the clinical epilepsy workup.
| S-EPMC6031752 | biostudies-literature
Project description:Collecting accurate and detailed dietary intake data is costly at a national level. Accordingly, limited dietary assessment tools such as Short Food Questionnaires (SFQs) are increasingly used in large surveys. This paper describes a novel method linking matched datasets to improve the quality of dietary data collected. Growing Up in Ireland (GUI) is a nationally representative longitudinal study of infants in the Republic of Ireland which used a SFQ (with no portion sizes) to assess the intake of "healthy" and "unhealthy" food and drink by 3 years old preschool children. The National Preschool Nutrition Survey (NPNS) provides the most accurate estimates available for dietary intake of young children in Ireland using a detailed 4 days weighed food diary. A mapping algorithm was applied using food name, cooking method, and food description to fill all GUI food groups with information from the NPNS food datafile which included the target variables, frequency, and amount. The augmented data were analyzed to examine all food groups described in NPNS and GUI and what proportion of foods were covered, non-covered, or partially-covered by GUI food groups, as a percentage of the total number of consumptions. The term non-covered indicated a specific food consumption that could not be mapped using a GUI food group. "High sugar" food items that were non-covered included ready-to-eat breakfast cereals, fruit juice, sugars, syrups, preserves and sweeteners, and ice-cream. The average proportion of consumption frequency and amount of foods not covered by GUI was 44 and 34%, respectively. Through mapping food codes in this manner, it was possible, using density plots, to visualize the relative performance of the brief dietary instrument (SFQ) compared to the more detailed food diary (FD). The SFQ did not capture a substantial portion of habitual foods consumed by 3-year olds in Ireland. Researchers interested in focussing on specific foods, could use this approach to assess the proportion of foods covered, non-covered, or partially-covered by reference to the mapped food database. These results can be used to improve SFQs for future studies and improve the capacity to identify diet-disease relationships.
| S-EPMC6190565 | biostudies-literature
Project description:BackgroundWeb-based questionnaires are increasingly used in epidemiologic studies, as traditional methods are facing a decrease in response rates and an increase in costs. However, few studies have investigated factors related to the level of completion of internet-based epidemiologic questionnaires.ObjectiveOur objective was to identify person-level characteristics and item design factors associated with breakoff (not finishing the questionnaire) and item nonresponse in a Web-based questionnaire.MethodsThis study was a cross-sectional analysis of the baseline questionnaire, applied from 2005 to 2016, of the Italian NINFEA (Nascita e Infanzia: gli Effetti dell'Ambiente) birth cohort. The baseline questionnaire was administered to enrolled women, who could register at any time during pregnancy. We used logistic regression to analyze the influence of person-level factors on questionnaire breakoff, and a logistic multilevel model (first level: items of the questionnaire; second level: sections of the questionnaire; third level: study participants) to analyze the influence of person-level and item design factors on item nonresponse. Since the number of applicable items depended on the respondent's characteristics and breakoff, we used inverse probability weighting to deal with missing by design.ResultsOf 5970 women, 519 (8.69%) did not finish the questionnaire. Older age (adjusted odds ratio 1.40, 95% CI 1.05-1.88), lower educational level (adjusted odds ratio [OR] 1.53, 95% CI 1.23-1.90), and earlier stage of pregnancy (adjusted OR 3.01, 95% CI 2.31-3.92) were positively associated with questionnaire breakoff. Of the 1,062,519 applicable items displayed for the participants, 22,831 were not responded to (overall prevalence of item nonresponse 2.15%). Item nonresponse was positively associated with older age (adjusted OR 1.25, 95% CI 1.14-1.38), being in the first trimester of pregnancy (adjusted OR 1.18, 95% CI 1.06-1.31), and lower educational level (adjusted OR 1.23, 95% CI 1.14-1.33). Dropdown menu items (adjusted OR 1.77, 95% CI 1.56-2.00) and items organized in grids (adjusted OR 1.69, 95% CI 1.49-1.91) were positively associated with item nonresponse.ConclusionsIt is important to use targeted strategies to keep participants motivated to respond. Item nonresponse in internet-based questionnaires is affected by person-level and item design factors. Some item types should be limited to reduce item nonresponse.
| S-EPMC6303736 | biostudies-literature
Project description:Easily utilized questionnaires estimating dietary calcium intake would be a valuable asset to promoting skeletal health as well as a helpful research tool. Two calcium questionnaires, one online and one printed, were each compared with dietary calcium intake measured by a 3-day diet record. Women completed the questionnaires in a randomized order and kept a 3-day food record at home, returning it by mail. The ethnicity of the 140 female study participants was 102 white, 12 African American, 16 Asian, and 10 Latina. The mean age (+/-standard deviation) was 49+/-15 years, with a range of 22.7 to 89.9 years. Measures used included an online calcium quiz, a printed calcium food frequency questionnaire, and one 3-day diet record. Data analysis was done using Pearson correlation coefficients. Sub-analyses were conducted by ethnicity. The correlation between each questionnaire and the daily calcium intake from the diet records was 0.37, P<0.001. Among the ethnic subgroups, the correlations were highest for African-American women, followed by white women. There was also a trend toward higher correlations in younger women. These simple calcium assessment tools, taking fewer than 5 minutes to complete, have correlation values with diet records similar to more complex food frequency questionnaires reported in the literature. These questionnaires may provide valuable tools to estimate calcium intake in research, clinical, and community settings.
| S-EPMC2760411 | biostudies-literature
Project description:BackgroundUsing electronic health records data to predict events and onset of diseases is increasingly common. Relatively little is known, although, about the tradeoffs between data requirements and model utility.Methods and resultsWe examined the performance of machine learning models trained to detect prediagnostic heart failure in primary care patients using longitudinal electronic health records data. Model performance was assessed in relation to data requirements defined by the prediction window length (time before clinical diagnosis), the observation window length (duration of observation before prediction window), the number of different data domains (data diversity), the number of patient records in the training data set (data quantity), and the density of patient encounters (data density). A total of 1684 incident heart failure cases and 13 525 sex, age-category, and clinic matched controls were used for modeling. Model performance improved as (1) the prediction window length decreases, especially when <2 years; (2) the observation window length increases but then levels off after 2 years; (3) the training data set size increases but then levels off after 4000 patients; (4) more diverse data types are used, but, in order, the combination of diagnosis, medication order, and hospitalization data was most important; and (5) data were confined to patients who had ≥10 phone or face-to-face encounters in 2 years.ConclusionsThese empirical findings suggest possible guidelines for the minimum amount and type of data needed to train effective disease onset predictive models using longitudinal electronic health records data.
| S-EPMC5341145 | biostudies-literature
Project description:The last series of Raven's standard progressive matrices (SPM-LS) test was studied with respect to its psychometric properties in a series of recent papers. In this paper, the SPM-LS dataset is analyzed with regularized latent class models (RLCMs). For dichotomous item response data, an alternative estimation approach based on fused regularization for RLCMs is proposed. For polytomous item responses, different alternative fused regularization penalties are presented. The usefulness of the proposed methods is demonstrated in a simulated data illustration and for the SPM-LS dataset. For the SPM-LS dataset, it turned out the regularized latent class model resulted in five partially ordered latent classes. In total, three out of five latent classes are ordered for all items. For the remaining two classes, violations for two and three items were found, respectively, which can be interpreted as a kind of latent differential item functioning.
| S-EPMC7555561 | biostudies-literature
Project description:Response to survey questionnaires is vital for social and behavioural research, and most analyses assume full and accurate response by participants. However, nonresponse is common and impedes proper interpretation and generalizability of results. We examined item nonresponse behaviour across 109 questionnaire items in the UK Biobank (N = 360,628). Phenotypic factor scores for two participant-selected nonresponse answers, 'Prefer not to answer' (PNA) and 'I don't know' (IDK), each predicted participant nonresponse in follow-up surveys (incremental pseudo-R2 = 0.056), even when controlling for education and self-reported health (incremental pseudo-R2 = 0.046). After performing genome-wide association studies of our factors, PNA and IDK were highly genetically correlated with one another (rg = 0.73 (s.e. = 0.03)) and with education (rg,PNA = -0.51 (s.e. = 0.03); rg,IDK = -0.38 (s.e. = 0.02)), health (rg,PNA = 0.51 (s.e. = 0.03); rg,IDK = 0.49 (s.e. = 0.02)) and income (rg,PNA = -0.57 (s.e. = 0.04); rg,IDK = -0.46 (s.e. = 0.02)), with additional unique genetic associations observed for both PNA and IDK (P < 5 × 10-8). We discuss how these associations may bias studies of traits correlated with item nonresponse and demonstrate how this bias may substantially affect genome-wide association studies. While the UK Biobank data are deidentified, we further protected participant privacy by avoiding exploring non-response behaviour to single questions, assuring that no information can be used to associate results with any particular respondents.
| S-EPMC10444625 | biostudies-literature