11:15 AM - 11:30 AM
[MAG32-03] Sample size requirements and target accuracy for the validation of landcover change maps
Keywords:validation, landcover change, accuracy assessment, satellite thematic products
The use of satellite-derived landcover and landcover change thematic maps has exponentially increased in the past two decades, and has transitioned from the research to the operational domain. For example satellite derived datasets are now widely accepted as the main data source for the support of national-level reporting of carbon inventories from agriculture, forestry and other land use (AFOLU), and for evaluating the effectiveness of projects funded under the REDD+ (Reducing emissions from deforestation and forest degradation in developing countries) framework.
The potential research, policy and management applications of these datasets place a high priority on the rigorous, quantitative assessment of their accuracy and precision. It is widely recognized that the validation of landcover and landcover change products requires the adoption of design-based validation methods, where the reference data is selected via a probability sampling, allowing for unbiased estimation of global accuracy metrics together with their confidence intervals. The validation of national to global scale products that are highly variable in time and space (i.e. the validation of the majority of the vegetation disturbance maps, such as deforestation, forest degradation, burned areas, or snow cover among a few examples) poses unique challenges because the class of interest represents only a small fraction of the landscape (i.e. is a rare class) and therefore a number of common simplifying assumptions cannot be justified In particular, the current state of the art techniques do not adequately address the issue of the minimum amount of independent reference data (i.e. the minimum sample size) required to meet a given level of accuracy expressed in terms of area of the rare class. In this work we present an innovative framework for the a-priori planning of the validation sample size, and for the iterative estimation of the confidence intervals of the accuracy metrics. We illustrate the methods by presenting two applications to forest cover loss and burned area maps.
The potential research, policy and management applications of these datasets place a high priority on the rigorous, quantitative assessment of their accuracy and precision. It is widely recognized that the validation of landcover and landcover change products requires the adoption of design-based validation methods, where the reference data is selected via a probability sampling, allowing for unbiased estimation of global accuracy metrics together with their confidence intervals. The validation of national to global scale products that are highly variable in time and space (i.e. the validation of the majority of the vegetation disturbance maps, such as deforestation, forest degradation, burned areas, or snow cover among a few examples) poses unique challenges because the class of interest represents only a small fraction of the landscape (i.e. is a rare class) and therefore a number of common simplifying assumptions cannot be justified In particular, the current state of the art techniques do not adequately address the issue of the minimum amount of independent reference data (i.e. the minimum sample size) required to meet a given level of accuracy expressed in terms of area of the rare class. In this work we present an innovative framework for the a-priori planning of the validation sample size, and for the iterative estimation of the confidence intervals of the accuracy metrics. We illustrate the methods by presenting two applications to forest cover loss and burned area maps.