Dataset Information

Land-Cover Classification Using MaxEnt: Can We Trust in Model Quality Metrics for Estimating Classification Accuracy?

ABSTRACT: MaxEnt is a popular maximum entropy-based algorithm originally developed for modelling species distribution, but increasingly used for land-cover classification. In this article, we used MaxEnt as a single-class land-cover classification and explored if recommended procedures for generating high-quality species distribution models also apply for generating high-accuracy land-cover classification. We used remote sensing imagery and randomly selected ground-true points for four types of land covers (built, grass, deciduous, evergreen) to generate 1980 classification maps using MaxEnt. We calculated different accuracy discrimination and quality model metrics to determine if these metrics were suitable proxies for estimating the accuracy of land-cover classification outcomes. Correlation analysis between model quality metrics showed consistent patterns for the relationships between metrics, but not for all land-covers. Relationship between model quality metrics and land-cover classification accuracy were land-cover-dependent. While for built cover there was no consistent patterns of correlations for any quality metrics; for grass, evergreen and deciduous, there was a consistent association between quality metrics and classification accuracy. We recommend evaluating the accuracy of land-cover classification results by using proper discrimination accuracy coefficients (e.g., Kappa, Overall Accuracy), and not placing all the confidence in model's quality metrics as a reliable indicator of land-cover classification results.

SUBMITTER: Morales NS

PROVIDER: S-EPMC7516803 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Land-Cover Classification Using MaxEnt: Can We Trust in Model Quality Metrics for Estimating Classification Accuracy?

Morales Narkis S NS Fernández Ignacio C IC

Entropy (Basel, Switzerland) 20200317 3

MaxEnt is a popular maximum entropy-based algorithm originally developed for modelling species distribution, but increasingly used for land-cover classification. In this article, we used MaxEnt as a single-class land-cover classification and explored if recommended procedures for generating high-quality species distribution models also apply for generating high-accuracy land-cover classification. We used remote sensing imagery and randomly selected ground-true points for four types of land cover ...[more]

PMID: 33286116

Similar Datasets

Project description:Multiple-class land-cover classification approaches can be inefficient when the main goal is to classify only one or a few classes. Under this scenario one-class classification algorithms could be a more efficient alternative. Currently there are several algorithms that can fulfil this task, with MaxEnt being one of the most promising. However, there is scarce information regarding parametrization for performing land-cover classification using MaxEnt. In this study we aimed to understand how MaxEnt parameterization affects the classification accuracy of four different land-covers (i.e., built-up, irrigated grass, evergreen trees and deciduous trees) in the city of Santiago de Chile. We also evaluated if MaxEnt manual parameterization outperforms classification results obtained when using MaxEnt default parameters setting. To accomplish our objectives, we generated a set of 25,344 classification maps (i.e., 6,336 for each assessed land-cover), which are based on all the potential combination of 12 different classes of features restrictions, four regularization multipliers, four different sample sizes, three training/testing proportions, and 11 thresholds for generating the binary maps. Our results showed that with a good parameterization, MaxEnt can effectively classify different land covers with kappa values ranging from 0.68 for deciduous trees to 0.89 for irrigated grass. However, the accuracy of classification results is highly influenced by the type of land-cover being classified. Simpler models produced good classification outcomes for homogenous land-covers, but not for heterogeneous covers, where complex models provided better outcomes. In general, manual parameterization improves the accuracy of classification results, but this improvement will depend on the threshold used to generate the binary map. In fact, threshold selection showed to be the most relevant factor impacting the accuracy of the four land-cover classification. The number of sampling points for training the model also has a positive effect on classification results. However, this effect followed a logarithmic distribution, showing an improvement of kappa values when increasing the sampling from 40 to 60 points, but showing only a marginal effect if more than 60 sampling points are used. In light of these results, we suggest testing different parametrization and thresholds until satisfactory kappa or other accuracy metrics values are achieved. Our results highlight the huge potential that MaxEnt has a as a tool for one-class classification, but a good understanding of the software settings and model parameterization is needed to obtain reliable results.

Project description:Research on spatial non-stationarity of land-cover classification accuracy has been ongoing for over two decades with most of the work focusing on single date maps. We extend the understanding of thematic map accuracy spatial patterns by: (1) quantifying spatial patterns of map-reference agreement for class-specific land-cover change rather than class-specific land cover for both omission and commission expressions of map error; (2) reporting goodness-of-fit estimates for the empirical models, which have been lacking in previous assessments, and; (3) using the empirical model results to map the locations of the relative likelihoods of map-reference agreement for specific land-cover change classes. We evaluated 10 map-based explanatory variables in single and multivariable logistic regression models to predict the likelihood of agreement between map and reference land-cover change (2001-2011) labels using the National Land Cover Database (NLCD) 2011 land cover and accuracy data. Logistic models for omission error had better goodness-of-fit estimates than models for commission error. For the omission error models, the explanatory variable, density of the mapped class-specific change in the immediate neighbourhood surrounding the sample pixel, produced the best model fit results (Tjur coefficient of discrimination, D, ranged from 0.59 to 0.98) compared to multivariable models and all other single explanatory variable models. Maps of the predicted likelihood of map-reference agreement produced from the best fitting omission error models provide a spatially explicit description of spatial variation of classification uncertainty at both local and regional scales. Application of the models indicated higher likelihoods of agreement (>50%) comprised a greater proportion of the land-cover change class area than the proportion of the land-cover change class with lower likelihoods of agreement. NLCD users can apply reported equations to map land-cover change uncertainty.

Project description:Model-based global projections of future land-use and land-cover (LULC) change are frequently used in environmental assessments to study the impact of LULC change on environmental services and to provide decision support for policy. These projections are characterized by a high uncertainty in terms of quantity and allocation of projected changes, which can severely impact the results of environmental assessments. In this study, we identify hotspots of uncertainty, based on 43 simulations from 11 global-scale LULC change models representing a wide range of assumptions of future biophysical and socioeconomic conditions. We attribute components of uncertainty to input data, model structure, scenario storyline and a residual term, based on a regression analysis and analysis of variance. From this diverse set of models and scenarios, we find that the uncertainty varies, depending on the region and the LULC type under consideration. Hotspots of uncertainty appear mainly at the edges of globally important biomes (e.g., boreal and tropical forests). Our results indicate that an important source of uncertainty in forest and pasture areas originates from different input data applied in the models. Cropland, in contrast, is more consistent among the starting conditions, while variation in the projections gradually increases over time due to diverse scenario assumptions and different modeling approaches. Comparisons at the grid cell level indicate that disagreement is mainly related to LULC type definitions and the individual model allocation schemes. We conclude that improving the quality and consistency of observational data utilized in the modeling process and improving the allocation mechanisms of LULC change models remain important challenges. Current LULC representation in environmental assessments might miss the uncertainty arising from the diversity of LULC change modeling approaches, and many studies ignore the uncertainty in LULC projections in assessments of LULC change impacts on climate, water resources or biodiversity.

Dataset Information

Land-Cover Classification Using MaxEnt: Can We Trust in Model Quality Metrics for Estimating Classification Accuracy?

Publications

Land-Cover Classification Using MaxEnt: Can We Trust in Model Quality Metrics for Estimating Classification Accuracy?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets