Dataset Information

Guideline for improving the reliability of Google Ngram studies: Evidence from religious terms.

ABSTRACT: The Google Books Ngram Viewer (Google Ngram) is a search engine that charts word frequencies from a large corpus of books and thereby allows for the examination of cultural change as it is reflected in books. While the tool's massive corpus of data (about 8 million books or 6% of all books ever published) has been used in various scientific studies, concerns about the accuracy of results have simultaneously emerged. This paper reviews the literature and serves as a guideline for improving Google Ngram studies by suggesting five methodological procedures suited to increase the reliability of results. In particular, we recommend the use of (I) different language corpora, (II) cross-checks on different corpora from the same language, (III) word inflections, (IV) synonyms, and (V) a standardization procedure that accounts for both the influx of data and unequal weights of word frequencies. Further, we outline how to combine these procedures and address the risk of potential biases arising from censorship and propaganda. As an example of the proposed procedures, we examine the cross-cultural expression of religion via religious terms for the years 1900 to 2000. Special emphasis is placed on the situation during World War II. In line with the strand of literature that emphasizes the decline of collectivistic values, our results suggest an overall decrease of religion's importance. However, religion re-gains importance during times of crisis such as World War II. By comparing the results obtained through the different methods, we illustrate that applying and particularly combining our suggested procedures increase the reliability of results and prevents authors from deriving wrong assumptions.

SUBMITTER: Younes N

PROVIDER: S-EPMC6430395 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Guideline for improving the reliability of Google Ngram studies: Evidence from religious terms.

Younes Nadja N Reips Ulf-Dietrich UD

PloS one 20190322 3

The Google Books Ngram Viewer (Google Ngram) is a search engine that charts word frequencies from a large corpus of books and thereby allows for the examination of cultural change as it is reflected in books. While the tool's massive corpus of data (about 8 million books or 6% of all books ever published) has been used in various scientific studies, concerns about the accuracy of results have simultaneously emerged. This paper reviews the literature and serves as a guideline for improving Google ...[more]

PMID: 30901329

Similar Datasets

Project description:BackgroundThe World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 11th of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this.ObjectivesWe reviewed the literature on incorporating considerations of cost-effectiveness, affordability and resource implications in guidelines and recommendations.MethodsWe searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments.Key questions and answersWhen is it important to incorporate cost-effectiveness, resource implications and affordability considerations in WHO guidelines (which topics)? For cost-effectiveness:The need for cost/effectiveness information should be dictated by the specific question, of which several may be addressed in a single guideline. It is proposed that the indications for undertaking a cost-effectiveness analysis (CEA) could be a starting point for determining which recommendation(s) in the guideline would benefit from such analysis. For resource implications/affordability:The resource implications of each individual recommendation need to be considered when implementation issues are being discussed. How can cost-effectiveness, resource implications and affordability be explicitly taken into account in WHO guidelines? For cost-effectiveness: composite function If data are available, the ideal time to consider cost-effectiveness is during the evidence gathering and synthesizing stage. However, because of the inconsistent availability of CEAs and the procedural difficulty associated with adjusting results from different CEAs to make them comparable, it is also possible for cost-effectiveness to be considered during the stage of developing recommendations. composite function Depending on the quantity and quality and relevance of the data available, such data can be considered in a qualitative way or in a quantitative way, ranging from a listing of the costs to a modelling exercise. At the very least, a qualitative approach like a commentary outlining the economic issues that need to be considered is necessary. If a quantitative approach is to be used, the full model should be transparent and comprehensive. For resource implications/affordability: composite function Resource implications, including health system changes, for each recommendation in a WHO guideline should be explored. At the minimum, a qualitative description that can serve as a gross indicator of the amount of resources needed, relative to current practice, should be provided. How does one provide guidance in contextualizing guideline recommendations at the country level based on considerations of cost-effectiveness, resource implications and affordability? All models should be made available and ideally are designed to allow for analysts to make changes in key parameters and reapply results in their own country. In the global guidelines, scenarios and extensive sensitivity/uncertainty analysis can be applied.Resource implications for WHO. From the above, it is clear that guidelines development groups will need a health economist. There is need to ensure that this is included in the budget for guidelines and that there is in-house support for this as well.

Project description:In this contribution, we present an innovative data-driven model to reconstruct a reliable temporal pattern for time-lagged statistical monetary figures. Our research cuts across several domains regarding the production of robust economic inferences and the bridging of top-down aggregated information from central databases with disaggregated information obtained from local sources or national statistical offices. Our test bed case study is the European Regional Development Fund (ERDF). The application we discuss deals with the reported time lag between the local expenditures of ERDF by beneficiaries in Italian regions and the corresponding payments reported in the European Commission database. Our model reconstructs the timing of these local expenditures by back-dating the observed European Commission reimbursements. The inferred estimates are then validated against the expenditures reported from the Italian National Managing Authorities (NMAs) in terms of cumulative monetary difference. The lower cumulative yearly distance of our modelled expenditures compared to the official European Commission payments confirms the robustness of our model. Using sensitivity analysis, we also analyse the relative importance of the modelling parameters on the cumulative distance between the modelled and reported expenditures. The parameters with the greatest influence on the uncertainty of this distance are the following: first, how the non-clearly regionalised expenditures are attributed to individual regions; and second, the number of backward years that the residuals of the yearly payments are spread onto. In general, the distance between the modelled and reported expenditures can be further reduced by fixing these parameters. However, the gain is only marginal for some regions. The present study paves the way for modelling exercises that are aimed at more reliable estimates of the expenditures on the ground by the ultimate beneficiaries of European funds. Additionally, the output databases can contribute to enhancing the reliability of econometric studies on the effectiveness of European Union (EU) funds.

Dataset Information

Guideline for improving the reliability of Google Ngram studies: Evidence from religious terms.

Publications

Guideline for improving the reliability of Google Ngram studies: Evidence from religious terms.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets