Dataset Information

Difficulties in benchmarking ecological null models: an assessment of current methods.

ABSTRACT: Identifying species interactions and detecting when ecological communities are structured by them is an important problem in ecology and biogeography. Ecologists have developed specialized statistical hypothesis tests to detect patterns indicative of community-wide processes in their field data. In this respect, null model approaches have proved particularly popular. The freedom allowed in choosing the null model and statistic to construct a hypothesis test leads to a proliferation of possible hypothesis tests from which ecologists can choose to detect these processes. Here, we point out some serious shortcomings of a popular approach to choosing the best hypothesis for the ecological problem at hand that involves benchmarking different hypothesis tests by assessing their performance on artificially constructed data sets. Terminological errors concerning the use of Type I and Type II errors that underlie these approaches are discussed. We argue that the key benchmarking methods proposed in the literature are not a sound guide for selecting null hypothesis tests, and further, that there is no simple way to benchmark null hypothesis tests. Surprisingly, the basic problems identified here do not appear to have been addressed previously, and these methods are still being used to develop and test new null models and summary statistics, from quantifying community structure (e.g., nestedness and modularity) to analyzing ecological networks.

SUBMITTER: Molina C

PROVIDER: S-EPMC7078898 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Difficulties in benchmarking ecological null models: an assessment of current methods.

Molina Chai C Stone Lewi L

Ecology 20200203 3

Identifying species interactions and detecting when ecological communities are structured by them is an important problem in ecology and biogeography. Ecologists have developed specialized statistical hypothesis tests to detect patterns indicative of community-wide processes in their field data. In this respect, null model approaches have proved particularly popular. The freedom allowed in choosing the null model and statistic to construct a hypothesis test leads to a proliferation of possible h ...[more]

PMID: 31834622

Similar Datasets

Project description:Measurement of outcomes has become increasingly important to assess the benefit of audiologic rehabilitation, including hearing aids, in adults. Data from questionnaires, however, are based on retrospective recall of events and experiences, and often can be inaccurate. Questionnaires also do not capture the daily variation that typically occurs in relevant events and experiences. Clinical researchers in a variety of fields have turned to a methodology known as ecological momentary assessment (EMA) to assess quotidian experiences associated with health problems. The objective of this study was to determine the feasibility of using EMA to obtain real-time responses from hearing aid users describing their experiences with challenging hearing situations.This study required three phases: (1) develop EMA methodology to assess hearing difficulties experienced by hearing aid users; (2) make use of focus groups to refine the methodology; and (3) test the methodology with 24 hearing aid users. Phase 3 participants carried a personal digital assistant 12 hr per day for 2 weeks. The personal digital assistant alerted participants to respond to questions four times a day. Each assessment started with a question to determine whether a hearing problem was experienced since the last alert. If "yes," then up to 23 questions (depending on contingent response branching) obtained details about the situation. If "no," then up to 11 questions obtained information that would help to explain why hearing was not a problem. Each participant completed the Hearing Handicap Inventory for the Elderly (HHIE) both before and after the 2-week EMA testing period to evaluate for "reactivity" (exacerbation of self-perceived hearing problems that could result from the repeated assessments).Participants responded to the alerts with a 77% compliance rate, providing a total of 991 completed momentary assessments (mean = 43.1 per participant). A substantial amount of data were obtained with the methodology. It is important to note that participants reported a "hearing problem situation since the last alert" 37.6% of the time (372 responses). The most common problem situation involved "face-to-face conversation" (53.8% of the time). The next most common problem situation was "telephone conversation" (17.2%) followed by "TV, radio, iPod, etc." (15.3%), "environmental sounds" (9.7%), and "movies, lecture, etc." (4.0%). Comparison of pre- and post-EMA mean HHIE scores revealed no significant difference (p > 0.05), indicating that reactivity did not occur for this group. It should be noted, however, that 37.5% of participants reported a greater sense of awareness regarding their hearing loss and use of hearing aids.Results showed participants were compliant, gave positive feedback, and did not demonstrate reactivity based on pre- and post-HHIE scores. We conclude that EMA methodology is feasible with patients who use hearing aids and could potentially inform hearing healthcare (HHC) services. The next step is to develop and evaluate EMA protocols that provide detailed daily patient information to audiologists at each stage of HHC. The advantages of such an approach would be to obtain real-life outcome measures, and to determine within- and between-day variability in outcomes and associated factors. Such information at present is not available from patients who seek and use HHC services.

Project description:BackgroundNew methods for assessing diet in research are being developed to address the limitations of traditional dietary assessment methods. Mobile device-assisted ecological momentary diet assessment (mEMDA) is a new dietary assessment method that has not yet been optimized and has the potential to minimize recall biases and participant burden while maximizing ecological validity. There have been limited efforts to characterize the use of mEMDA in behavioral research settings.ObjectiveThe aims of this study were to summarize mEMDA protocols used in research to date, to characterize key aspects of these assessment approaches, and to discuss the advantages and disadvantages of mEMDA compared with the traditional dietary assessment methods as well as implications for future mEMDA research.MethodsStudies that used mobile devices and described mEMDA protocols to assess dietary intake were included. Data were extracted according to Preferred Reporting of Systematic Reviews and Meta-Analyses and Cochrane guidelines and then synthesized narratively.ResultsThe review included 20 studies with unique mEMDA protocols. Of these, 50% (10/20) used participant-initiated reports of intake at eating events (event-contingent mEMDA), and 50% (10/20) used researcher-initiated prompts requesting that participants report recent dietary intake (signal-contingent mEMDA). A majority of the study protocols (60%, 12/20) enabled participants to use mobile phones to report dietary data. Event-contingent mEMDA protocols most commonly assessed diet in real time, used dietary records for data collection (60%, 6/10), and provided estimates of energy and nutrient intake (60%, 6/10). All signal-contingent mEMDA protocols used a near real-time recall approach with unannounced (ie, random) abbreviated diet surveys. Most signal-contingent protocols (70%, 7/10) assessed the frequency with which (targeted) foods or food groups were consumed. Relatively few (30%, 6/20) studies compared mEMDA with the traditional dietary assessment methods.ConclusionsThis review demonstrates that mEMDA has the potential to reduce participant burden and recall bias, thus advancing the field beyond current dietary assessment methods while maximizing ecological validity.

Project description:The disturbance, damage and destruction of roosts are key drivers of bat population declines worldwide. In countries where bats are protected by law, bat roost surveys are often required to inform ecological impact assessments. Yet, evidence-based information on survey methodology to detect bat roosts is crucially lacking, and failing to detect a roost can lead to serious errors during decision-making processes. Here, we assess the efficacy of bat roost surveys in buildings as implemented in the UK. These consist of a daytime inspection of buildings, followed by a series of acoustic surveys at dusk/dawn if during the daytime inspection evidence of bats is found, or if the absence of bats cannot be verified. We reviewed 155 ecological consultants' reports to (1) compare survey outcome between daytime inspection and acoustic surveys and (2) determine the minimum sampling effort required during acoustic surveys to be confident that no bats are roosting within a building. We focused on two genera of bats most frequently found in buildings in Europe - Pipistrellus (crevice roosting species with high-intensity echolocation calls that can be easily detected by ultrasound detectors) and Plecotus (species that roost in open spaces and which emit faint echolocation calls that are difficult to detect). Daytime inspections were efficient in detecting open-roosting species such as Plecotus species but were likely to miss the presence of crevice-dwelling ones (here Pipistrellus species) which may lead to erroneous conclusions if no acoustic surveys are subsequently prescribed to confirm their absence. A minimum of three and four acoustic surveys are required to be 95% confident that a building does not host a roost of Pipistrellus species and Plecotus species, respectively, thus exceeding current recommendations. Overall, we demonstrated that reports submitted as part of an ecological impact assessment provide suitable data to test and improve survey methods.

Dataset Information

Difficulties in benchmarking ecological null models: an assessment of current methods.

Publications

Difficulties in benchmarking ecological null models: an assessment of current methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets