Simulating Study Design Choice Effects on Observed Performance of Predictive Patient Monitoring Alarm Algorithms.
Ontology highlight
ABSTRACT:
SUBMITTER: Nahmias DO
PROVIDER: S-EPMC8392319 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:In [Scully, C.G., and Daluwatte, C., Evaluating performance of early warning indices to predict physiological instabilities. J Biomed Inform. 75 (2017) 14-21], a framework was presented to characterize the performance of warning indices to provide information on the 1) probability a critical health event will occur when a warning is given (analogous to positive predictive value) and 2) proportion of warned events to all events (analogous to sensitivity). This framework also provides information about the timeliness of the warnings with respect to event occurrence and the warning burden of the system. •In the current work, we provide information on how this framework can be used when cases without events are present in a dataset to examine the proportion of warned non-events to all non-events (analogous to false positive rate).•Information on steps to apply the method, software, data and results for the case study are also provided to enable implementation of the framework.•Application and extension of the framework is demonstrated and discussed by adding non-event records to our previous case study comparing two warning strategies to predict physiologic instabilities.
| S-EPMC6660561 | biostudies-literature
Project description:The laboratory study of how humans and other animals trade-off value and time has a long and storied history, and is the subject of a vast literature. However, despite a long history of study, there is no agreed upon mechanistic explanation of how intertemporal choice preferences arise. Several theorists have recently proposed model-based reinforcement learning as a candidate framework. This framework describes a suite of algorithms by which a model of the environment, in the form of a state transition function and reward function, can be converted on-line into a decision. The state transition function allows the model-based system to make decisions based on projected future states, while the reward function assigns value to each state, together capturing the necessary components for successful intertemporal choice. Empirical work has also pointed to a possible relationship between increased prospection and reduced discounting. In the current paper, we look for direct evidence of a relationship between temporal discounting and model-based control in a large new data set (n?=?168). However, testing the relationship under several different modeling formulations revealed no indication that the two quantities are related.
| S-EPMC5320483 | biostudies-literature
Project description:We elicited patient experiences from clinical trial simulations to aid in future trial development and to improve patient recruitment and retention.Two simulations of draft Phase II and Phase III anifrolumab studies for systemic lupus erythematosus (SLE)/lupus nephritis (LN) were performed involving African-American patients from Grady Hospital, an indigent care hospital in Atlanta, GA, USA, and white patients from Altoona Arthritis and Osteoporosis Center in Altoona, PA, USA. The clinical trial simulation included an informed consent procedure, a mock screening visit, a mock dosing visit, and a debriefing period for patients and staff. Patients and staff were interviewed to obtain sentiments and perceptions related to the simulated visits.The Atlanta study involved 6 African-American patients (5 female) aged 27-60 years with moderate to severe SLE/LN. The Altoona study involved 12 white females aged 32-75 years with mild to moderate SLE/LN. Patient experiences had an impact on four patient-centric care domains: 1) information, communication, and education; 2) responsiveness to needs; 3) access to care; and 4) coordination of care; and continuity and transition. Patients in both studies desired background material, knowledgeable staff, family and friend support, personal results, comfortable settings, shorter wait times, and greater scheduling flexibility. Compared with the Altoona study patients, Atlanta study patients reported greater preferences for information from the Internet, need for strong community and online support, difficulties in discussing SLE, emphasis on transportation and child care help during the visits, and concerns related to financial matters; and they placed greater importance on time commitment, understanding of potential personal benefit, trust, and confidentiality of patient data as factors for participation. Using these results, we present recommendations to improve study procedures to increase retention, recruitment, and compliance for clinical trials.Insights from these two studies can be applied to the development and implementation of future clinical trials to improve patient recruitment, retention, compliance, and advocacy.
| S-EPMC5545635 | biostudies-other
Project description:BackgroundThe necessity to translate eligibility criteria from free text into decision rules that are compatible with data from the electronic health record (EHR) constitutes the main challenge when developing and deploying clinical trial recruitment support systems. Recruitment decisions based on case-based reasoning, i.e. using past cases rather than explicit rules, could dispense with the need for translating eligibility criteria and could also be implemented largely independently from the terminology of the EHR's database. We evaluated the feasibility of predictive modeling to assess the eligibility of patients for clinical trials and report on a prototype's performance for different system configurations.MethodsThe prototype worked by using existing basic patient data of manually assessed eligible and ineligible patients to induce prediction models. Performance was measured retrospectively for three clinical trials by plotting receiver operating characteristic curves and comparing the area under the curve (ROC-AUC) for different prediction algorithms, different sizes of the learning set and different numbers and aggregation levels of the patient attributes.ResultsRandom forests were generally among the best performing models with a maximum ROC-AUC of 0.81 (CI: 0.72-0.88) for trial A, 0.96 (CI: 0.95-0.97) for trial B and 0.99 (CI: 0.98-0.99) for trial C. The full potential of this algorithm was reached after learning from approximately 200 manually screened patients (eligible and ineligible). Neither block- nor category-level aggregation of diagnosis and procedure codes influenced the algorithms' performance substantially.ConclusionsOur results indicate that predictive modeling is a feasible approach to support patient recruitment into clinical trials. Its major advantages over the commonly applied rule-based systems are its independency from the concrete representation of eligibility criteria and EHR data and its potential for automation.
| S-EPMC4029400 | biostudies-literature
Project description:ObjectivesTo determine whether an innovative graphical tool for accurate measurement of individual surgeon performance metrics, adjusted for both surgeon-specific and patient-specific factors, significantly alters interpretation of performance data.DesignRetrospective analysis of all total knee replacements (TKRs) conducted at the host institution between 1996 and 2009. The database was randomly divided into training and testing datasets. Using multivariate generalised estimating equation regression models, the training dataset enabled generation of patient-risk and surgeon-experience adjustment factors. To simulate prospective monitoring of individual surgeon outcomes, the testing dataset was mapped on control charts. Weighted ? statistics were calculated to measure the agreement between patient-risk adjusted and fully adjusted control charts.SettingTertiary care academic hospital.ParticipantsAll patients undergoing TKR at the host institution 1996-2009.Main outcome measureOperative efficiency.Results5313 procedures were analysed. Adjusted control charts were generated using a training dataset comprising 3756 procedures performed by 13 surgeons. The operative time gradually declined by 121 min with 25 years of experience (p<0.0001). Charts were tested by monitoring four other surgeons, performing an average of 389 procedures each. Adjustment for surgeon experience significantly altered the interpretation of operative efficiency (?=0.29 (95% CI 0.11 to 0.47)), and enhanced assessment of a surgeon's improvement or diminishment in efficiency over time. Specifically, experience adjustment inverted the interpretation of surgeon efficiency from above average to below average, or from improving to declining performance.ConclusionsAdjustment for surgeon experience is necessary for accurate interpretation of metrics over the course of a surgeon's career. Patient-adjusted and surgeon-adjusted control charts provide an accurate method of monitoring individual operative efficiency.
| S-EPMC3902330 | biostudies-literature
Project description:Spontaneous reports of suspected adverse drug reactions (ADRs) can be analyzed to yield additional drug safety evidence for the pediatric population. Signal detection algorithms (SDAs) are required for these analyses; however, the performance of SDAs in the pediatric population specifically is unknown. We tested the performance of two SDAs on pediatric data from the US FDA Adverse Event Reporting System (FAERS) and investigated the impact of age stratification and age adjustment on the performance of SDAs.We tested the performance of two established SDAs: the proportional reporting ratio (PRR) and the empirical Bayes geometric mean (EBGM) on a pediatric dataset from FAERS (2004-2012). We compared the performance of the SDAs with a published pediatric-specific reference set by calculating diagnostic test-related statistics, including the area under the curve (AUC) of receiver operating characteristics. Impact of age stratification and age-adjustment on the performance of the SDAs was assessed. Age adjustment was performed by pooling (Mantel-Hanszel) stratum-specific estimates.A total of 115,674 pediatric reports (patients aged 0-18 years) comprising 893,587 drug-event combinations (DECs) were analysed. Crude values of the AUC were similar for both SDAs: 0.731 (PRR) and 0.745 (EBGM). Stratification unmasked four DECs, e.g., 'ibuprofen and thrombocytopenia'. Age adjustment did not improve performance.The performance of the two tested SDAs was similar in the pediatric population. Age adjustment does not improve performance and is therefore not recommended to be performed routinely. Stratification can reveal new associations, and therefore is recommended when either drug use is age-specific or when an age-specific risk is suspected.
| S-EPMC4982893 | biostudies-other
Project description:BackgroundIntravascular catheter infections are associated with adverse clinical outcomes. However, a significant proportion of these infections are preventable. Evaluations of the performance of automated surveillance systems for adequate monitoring of central-line associated bloodstream infection (CLABSI) or catheter-related bloodstream infection (CRBSI) are limited.ObjectivesWe evaluated the predictive performance of automated algorithms for CLABSI/CRBSI detection, and investigated which parameters included in automated algorithms provide the greatest accuracy for CLABSI/CRBSI detection.MethodsWe performed a meta-analysis based on a systematic search of published studies in PubMed and EMBASE from 1 January 2000 to 31 December 2021. We included studies that evaluated predictive performance of automated surveillance algorithms for CLABSI/CRBSI detection and used manually collected surveillance data as reference. We estimated the pooled sensitivity and specificity of algorithms for accuracy and performed a univariable meta-regression of the different parameters used across algorithms.ResultsThe search identified five full text studies and 32 different algorithms or study populations were included in the meta-analysis. All studies analysed central venous catheters and identified CLABSI or CRBSI as an outcome. Pooled sensitivity and specificity of automated surveillance algorithm were 0.88 [95%CI 0.84-0.91] and 0.86 [95%CI 0.79-0.92] with significant heterogeneity (I2 = 91.9, p < 0.001 and I2 = 99.2, p < 0.001, respectively). In meta-regression, algorithms that include results of microbiological cultures from specific specimens (respiratory, urine and wound) to exclude non-CRBSI had higher specificity estimates (0.92, 95%CI 0.88-0.96) than algorithms that include results of microbiological cultures from any other body sites (0.88, 95% CI 0.81-0.95). The addition of clinical signs as a predictor did not improve performance of these algorithms with similar specificity estimates (0.92, 95%CI 0.88-0.96).ConclusionsPerformance of automated algorithms for detection of intravascular catheter infections in comparison to manual surveillance seems encouraging. The development of automated algorithms should consider the inclusion of results of microbiological cultures from specific specimens to exclude non-CRBSI, while the inclusion of clinical data may not have an added-value. Trail Registration Prospectively registered with International prospective register of systematic reviews (PROSPERO ID CRD42022299641; January 21, 2022). https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022299641.
| S-EPMC10468855 | biostudies-literature
Project description:To engineer synthetic gene circuits, molecular building blocks are developed which can modulate gene expression without interference, mutually or with the host’s cell machinery. Promoter libraries of E. coli sigma factor 70 and B. subtilis B-, F- and W-dependent promoters are exploited to construct prediction models, capable of both predicting promoter TIF and orthogonality of the specific promoters. This is achieved by the creation of high-throughput DNA sequencing data from fluorescence-activated cell sorted promoter libraries.
2020-08-01 | E-MTAB-8734 | biostudies-arrayexpress
Project description: Not available
| S-EPMC7871213 | biostudies-literature
Project description:A number of researchers have attempted to estimate salmonid smolt survival during outmigration through an estuary. However, it is currently unclear how the design of such studies influences the accuracy and precision of survival estimates. In this simulation study we consider four patterns of smolt survival probability in the estuary, and test the performance of several different sampling strategies for estimating estuarine survival assuming perfect detection. The four survival probability patterns each incorporate a systematic component (constant, linearly increasing, increasing and then decreasing, and two pulses) and a random component to reflect daily fluctuations in survival probability. Generally, spreading sampling effort (tagging) across the season resulted in more accurate estimates of survival. All sampling designs in this simulation tended to under-estimate the variation in the survival estimates because seasonal and daily variation in survival probability are not incorporated in the estimation procedure. This under-estimation results in poorer performance of estimates from larger samples. Thus, tagging more fish may not result in better estimates of survival if important components of variation are not accounted for. The results of our simulation incorporate survival probabilities and run distribution data from previous studies to help illustrate the tradeoffs among sampling strategies in terms of the number of tags needed and distribution of tagging effort. This information will assist researchers in developing improved monitoring programs and encourage discussion regarding issues that should be addressed prior to implementation of any telemetry-based monitoring plan. We believe implementation of an effective estuary survival monitoring program will strengthen the robustness of life cycle models used in recovery plans by providing missing data on where and how much mortality occurs in the riverine and estuarine portions of smolt migration. These data could result in better informed management decisions and assist in guidance for more effective estuarine restoration projects.
| S-EPMC4510331 | biostudies-literature