Dataset Information

Identifying science in the news: An assessment of the precision and recall of Altmetric.com news mention data.

ABSTRACT: The company Altmetric is often used to collect mentions of research in online news stories, yet there have been concerns about the quality of this data. This study investigates these concerns. Using a manual content analysis of 400 news stories as a comparison method, we analyzed the precision and recall with which Altmetric identified mentions of research in 8 news outlets. We also used logistic regression to identify the characteristics of research mentions that influence their likelihood of being successfully identified. We find that, for a predefined set of outlets, Altmetric's news mention data were relatively accurate (F-score = 0.80), with very high precision (0.95) and acceptable recall (0.70), although recall is below 0.50 for some news outlets. Altmetric is more likely to successfully identify mentions of research that include a hyperlink to the research item, an author name, and/or the title of a publication venue. This data source appears to be less reliable for mentions of research that provide little or no bibliometric information, as well as for identifying mentions of scholarly monographs, conference presentations, dissertations, and non-English research articles. Our findings suggest that, with caveats, scholars can use Altmetric news mention data as a relatively reliable source to identify research mentions across a range of outlets with high precision and acceptable recall, offering scholars the potential to conserve resources during data collection. Our study does not, however, offer an assessment of completeness or accuracy of Altmetric news data overall.

Supplementary information

The online version contains supplementary material available at 10.1007/s11192-022-04510-7.

SUBMITTER: Fleerackers A

PROVIDER: S-EPMC9526208 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Identifying science in the news: An assessment of the precision and recall of Altmetric.com news mention data.

Fleerackers Alice A Nehring Lise L Maggio Lauren A LA Enkhbayar Asura A Moorhead Laura L Alperin Juan Pablo JP

Scientometrics 20221001 11

The company Altmetric is often used to collect mentions of research in online news stories, yet there have been concerns about the quality of this data. This study investigates these concerns. Using a manual content analysis of 400 news stories as a comparison method, we analyzed the precision and recall with which Altmetric identified mentions of research in 8 news outlets. We also used logistic regression to identify the characteristics of research mentions that influence their likelihood of b ...[more]

PMID: 36212767

Similar Datasets

Project description:Journalists' health and science reporting aid the public's direct access to research through the inclusion of hyperlinks leading to original studies in peer-reviewed journals. While this effort supports the US-government mandate that research be made widely available, little is known about what research journalists share with the public. This cross-sectional exploratory study characterises US-government-funded research on cancer that appeared most frequently in news coverage and how that coverage varied by cancer type, disease incidence and mortality rates. The subject of analysis was 11436 research articles (published in 2016) on cancer funded by the US government and 642 news stories mentioning at least one of these articles. Based on Altmetric data, researchers identified articles via PubMed and characterised each based on the news media attention received online. Only 1.88% (n = 213) of research articles mentioning US government-funded cancer research included at least one mention in an online news publication. This is in contrast to previous research that found 16.8% (n = 1925) of articles received mention by online mass media publications. Of the 13 most common cancers in the US, 12 were the subject of at least one news mention; only urinary and bladder cancer received no mention. Traditional news sources included significantly more mentions of research on common cancers than digital native news sources. However, a general discrepancy exists between cancers prominent in news sources and those with the highest mortality rate. For instance, lung cancer accounted for the most deaths annually, while melanoma led to 56% less annual deaths; however, journalists cited research regarding these cancers nearly equally. Additionally, breast cancer received the greatest coverage per estimated annual death, while pancreatic cancer received the least coverage per death. Findings demonstrated a continued misalignment between prevalent cancers and cancers mentioned in online news media. Additionally, cancer control and prevention received less coverage from journalists than other cancer continuum stages, highlighting a continued underrepresentation of prevention-focused research. Results revealed a need for further scholarship regarding the role of journalists in research dissemination.

Project description:ObjectiveDuring cardiovascular disease progression, molecular systems of myocardium (e.g., a proteome) undergo diverse and distinct changes. Dynamic, temporally-regulated alterations of individual molecules underlie the collective response of the heart to pathological drivers and the ultimate development of pathogenesis. Advances in high-throughput omics technologies have enabled cost-effective, temporal profiling of targeted systems in animal models of human diseases. However, computational analysis of temporal patterns from omics data remains challenging. In particular, bioinformatic pipelines involving unsupervised statistical approaches to support cardiovascular investigations are lacking, which hinders one's ability to extract biomedical insights from these complex datasets.Approach and resultsWe developed a non-parametric data analysis platform to resolve computational challenges unique to temporal omics datasets. Our platform consists of three modules. Module I preprocesses the temporal data using either cubic splines or principal component analysis (PCA), and it simultaneously accomplishes the tasks on missing data imputation and denoising. Module II performs an unsupervised classification by K-means or hierarchical clustering. Module III evaluates and identifies biological entities (e.g., molecular events) that exhibit strong associations to specific temporal patterns. The jackstraw method for cluster membership has been applied to estimate p-values and posterior inclusion probabilities (PIPs), both of which guided feature selection. To demonstrate the utility of the analysis platform, we employed a temporal proteomics dataset that captured the proteome-wide dynamics of oxidative stress induced post-translational modifications (O-PTMs) in mouse hearts undergoing isoproterenol (ISO)-induced hypertrophy.ConclusionWe have created a platform, CV.Signature.TCP, to identify distinct temporal clusters in omics datasets. We presented a cardiovascular use case to demonstrate its utility in unveiling biological insights underlying O-PTM regulations in cardiac remodeling. This platform is implemented in an open source R package (https://github.com/UCLA-BD2K/CV.Signature.TCP).

Project description:BackgroundExaggerated or simplistic news is often blamed for adversely influencing public health. However, recent findings suggested many exaggerations were already present in university press releases, which scientists approve. Surprisingly, these exaggerations were not associated with more news coverage. Here we test whether these two controversial results also arise in press releases from prominent science and medical journals. We then investigate the influence of mitigating caveats in press releases, to test assumptions that caveats harm news interest or are ignored.Methods and findingsUsing quantitative content analysis, we analyzed press releases (N = 534) on biomedical and health-related science issued by leading peer-reviewed journals. We similarly analysed the associated peer-reviewed papers (N = 534) and news stories (N = 582). Main outcome measures were advice to readers and causal statements drawn from correlational research. Exaggerations in press releases predicted exaggerations in news (odds ratios 2.4 and 10.9, 95% CIs 1.3 to 4.5 and 3.9 to 30.1) but were not associated with increased news coverage, consistent with previous findings. Combining datasets from universities and journals (996 press releases, 1250 news), we found that when caveats appeared in press releases there was no reduction in journalistic uptake, but there was a clear increase in caveats in news (odds ratios 9.6 and 9.5 for caveats for advice and causal claims, CIs 4.1 to 24.3 and 6.0 to 15.2). The main study limitation is its retrospective correlational nature.ConclusionsFor health and science news directly inspired by press releases, the main source of both exaggerations and caveats appears to be the press release itself. However we find no evidence that exaggerations increase, or caveats decrease, the likelihood of news coverage. These findings should be encouraging for press officers and scientists who wish to minimise exaggeration and include caveats in their press releases.

Project description:Many publications make use of opportunistic data, such as citizen science observation data, to infer large-scale properties of species' distributions. However, the few publications that use opportunistic citizen science data to study animal ecology at a habitat level do so without accounting for spatial biases in opportunistic records or using methods that are difficult to generalize. In this study, we explore the biases that exist in opportunistic observations and suggest an approach to correct for them. We first examined the extent of the biases in opportunistic citizen science observations of three wild ungulate species in Norway by comparing them to data from GPS telemetry. We then quantified the extent of the biases by specifying a model of the biases. From the bias model, we sampled available locations within the species' home range. Along with opportunistic observations, we used the corrected availability locations to estimate a resource selection function (RSF). We tested this method with simulations and empirical datasets for the three species. We compared the results of our correction method to RSFs obtained using opportunistic observations without correction and to RSFs using GPS-telemetry data. Finally, we compared habitat suitability maps obtained using each of these models. Opportunistic observations are more affected by human access and visibility than locations derived from GPS telemetry. This has consequences for drawing inferences about species' ecology. Models naïvely using opportunistic observations in habitat-use studies can result in spurious inferences. However, sampling availability locations based on the spatial biases in opportunistic data improves the estimation of the species' RSFs and predicted habitat suitability maps in some cases. This study highlights the challenges and opportunities of using opportunistic observations in habitat-use studies. While our method is not foolproof it is a first step toward unlocking the potential of opportunistic citizen science data for habitat-use studies.

Dataset Information

Identifying science in the news: An assessment of the precision and recall of Altmetric.com news mention data.

Supplementary information

Publications

Identifying science in the news: An assessment of the precision and recall of Altmetric.com news mention data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets