Dataset Information

A data science challenge for converting airborne remote sensing data into ecological information.

ABSTRACT: Ecology has reached the point where data science competitions, in which multiple groups solve the same problem using the same data by different methods, will be productive for advancing quantitative methods for tasks such as species identification from remote sensing images. We ran a competition to help improve three tasks that are central to converting images into information on individual trees: (1) crown segmentation, for identifying the location and size of individual trees; (2) alignment, to match ground truthed trees with remote sensing; and (3) species classification of individual trees. Six teams (composed of 16 individual participants) submitted predictions for one or more tasks. The crown segmentation task proved to be the most challenging, with the highest-performing algorithm yielding only 34% overlap between remotely sensed crowns and the ground truthed trees. However, most algorithms performed better on large trees. For the alignment task, an algorithm based on minimizing the difference, in terms of both position and tree size, between ground truthed and remotely sensed crowns yielded a perfect alignment. In hindsight, this task was over simplified by only including targeted trees instead of all possible remotely sensed crowns. Several algorithms performed well for species classification, with the highest-performing algorithm correctly classifying 92% of individuals and performing well on both common and rare species. Comparisons of results across algorithms provided a number of insights for improving the overall accuracy in extracting ecological information from remote sensing. Our experience suggests that this kind of competition can benefit methods development in ecology and biology more broadly.

SUBMITTER: Marconi S

PROVIDER: S-EPMC6397763 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A data science challenge for converting airborne remote sensing data into ecological information.

Marconi Sergio S Graves Sarah J SJ Gong Dihong D Nia Morteza Shahriari MS Le Bras Marion M Dorr Bonnie J BJ Fontana Peter P Gearhart Justin J Greenberg Craig C Harris Dave J DJ Kumar Sugumar Arvind SA Nishant Agarwal A Prarabdh Joshi J Rege Sundeep U SU Bohlman Stephanie Ann SA White Ethan P EP Wang Daisy Zhe DZ

PeerJ 20190228

Ecology has reached the point where data science competitions, in which multiple groups solve the same problem using the same data by different methods, will be productive for advancing quantitative methods for tasks such as species identification from remote sensing images. We ran a competition to help improve three tasks that are central to converting images into information on individual trees: (1) crown segmentation, for identifying the location and size of individual trees; (2) alignment, t ...[more]

PMID: 30842892

Similar Datasets

Project description:Data on individual tree crowns from remote sensing have the potential to advance forest ecology by providing information about forest composition and structure with a continuous spatial coverage over large spatial extents. Classifying individual trees to their taxonomic species over large regions from remote sensing data is challenging. Methods to classify individual species are often accurate for common species, but perform poorly for less common species and when applied to new sites. We ran a data science competition to help identify effective methods for the task of classification of individual crowns to species identity. The competition included data from three sites to assess each methods' ability to generalize patterns across two sites simultaneously and apply methods to an untrained site. Three different metrics were used to assess and compare model performance. Six teams participated, representing four countries and nine individuals. The highest performing method from a previous competition in 2017 was applied and used as a baseline to understand advancements and changes in successful methods. The best species classification method was based on a two-stage fully connected neural network that significantly outperformed the baseline random forest and gradient boosting ensemble methods. All methods generalized well by showing relatively strong performance on the trained sites (accuracy = 0.46-0.55, macro F1 = 0.09-0.32, cross entropy loss = 2.4-9.2), but generally failed to transfer effectively to the untrained site (accuracy = 0.07-0.32, macro F1 = 0.02-0.18, cross entropy loss = 2.8-16.3). Classification performance was influenced by the number of samples with species labels available for training, with most methods predicting common species at the training sites well (maximum F1 score of 0.86) relative to the uncommon species where none were predicted. Classification errors were most common between species in the same genus and different species that occur in the same habitat. Most methods performed better than the baseline in detecting if a species was not in the training data by predicting an untrained mixed-species class, especially in the untrained site. This work has highlighted that data science competitions can encourage advancement of methods, particularly by bringing in new people from outside the focal discipline, and by providing an open dataset and evaluation criteria from which participants can learn.

Project description:Ecological site classification has emerged as a highly effective land management framework, but its utility at a regional scale has been limited due to the spatial ambiguity of ecological site locations in the U.S. or the absence of ecological site maps in other regions of the world. In response to these shortcomings, this study evaluated the use of hyper-temporal remote sensing (i.e., hundreds of images) for high spatial resolution mapping of ecological sites. We posit that hyper-temporal remote sensing can provide novel insights into the spatial variability of ecological sites by quantifying the temporal response of land surface spectral properties. This temporal response provides a spectral 'fingerprint' of the soil-vegetation-climate relationship which is central to the concept of ecological sites. Consequently, the main objective of this study was to predict the spatial distribution of ecological sites in a semi-arid rangeland using a 28-year time series of normalized difference vegetation index from Landsat TM 5 data and modeled using support vector machine classification. Results from this study show that support vector machine classification using hyper-temporal remote sensing imagery was effective in modeling ecological site classes, with a 62% correct classification. These results were compared to Gridded Soil Survey Geographic database and expert delineated maps of ecological sites which had a 51 and 89% correct classification, respectively. An analysis of the effects of ecological state on ecological site misclassifications revealed that sites in degraded states (e.g., shrub-dominated/shrubland and bare/annuals) had a higher rate of misclassification due to their close spectral similarity with other ecological sites. This study identified three important factors that need to be addressed to improve future model predictions: 1) sampling designs need to fully represent the range of both within class (i.e., states) and between class (i.e., ecological sites) spectral variability through time, 2) field sampling protocols that accurately characterize key soil properties (e.g., texture, depth) need to be adopted, and 3) additional environmental covariates (e.g. terrain attributes) need to be evaluated that may help further differentiate sites with similar spectral signals. Finally, the proposed hyper-temporal remote sensing framework may provide a standardized approach to evaluate and test our ecological site concepts through examining differences in vegetation dynamics in response to climatic variability and other drivers of land-use change. Results from this study demonstrate the efficacy of the hyper-temporal remote sensing approach for high resolution mapping of ecological sites, and highlights its utility in terms of reduced cost and time investment relative to traditional manual mapping approaches.

Dataset Information

A data science challenge for converting airborne remote sensing data into ecological information.

Publications

A data science challenge for converting airborne remote sensing data into ecological information.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets