Dataset Information

Developing reliable hourly electricity demand data through screening and imputation.

ABSTRACT: Electricity usage (demand) data are used by utilities, governments, and academics to model electric grids for a variety of planning (e.g., capacity expansion and system operation) purposes. The U.S. Energy Information Administration collects hourly demand data from all balancing authorities (BAs) in the contiguous United States. As of September 2019, we find 2.2% of the demand data in their database are missing. Additionally, 0.5% of reported quantities are either negative values or are otherwise identified as outliers. With the goal of attaining non-missing, continuous, and physically plausible demand data to facilitate analysis, we developed a screening process to identify anomalous values. We then applied a Multiple Imputation by Chained Equations (MICE) technique to impute replacements for missing and anomalous values. We conduct cross-validation on the MICE technique by marking subsets of plausible data as missing, and using the remaining data to predict this "missing" data. The mean absolute percentage error of imputed values is 3.5% across all BAs. The cleaned data are published and available open access: https://doi.org/10.5281/zenodo.3690240.

SUBMITTER: Ruggles TH

PROVIDER: S-EPMC7250876 | biostudies-literature | 2020 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Developing reliable hourly electricity demand data through screening and imputation.

Ruggles Tyler H TH Farnham David J DJ Tong Dan D Caldeira Ken K

Scientific data 20200526 1

Electricity usage (demand) data are used by utilities, governments, and academics to model electric grids for a variety of planning (e.g., capacity expansion and system operation) purposes. The U.S. Energy Information Administration collects hourly demand data from all balancing authorities (BAs) in the contiguous United States. As of September 2019, we find 2.2% of the demand data in their database are missing. Additionally, 0.5% of reported quantities are either negative values or are otherwis ...[more]

PMID: 32457368

Similar Datasets

Project description:After the severe nuclear disaster in Fukushima, which was triggered by the Great East Japan earthquake in March 2011, nuclear power plants in Japan were temporarily shut down for mandatory inspections. To prevent large-scale blackouts, the Japanese government requested companies and households to reduce electricity consumption in summer and winter. It is reported that the domestic electricity demand had a structural decrease because of the electricity conservation effect (ECE). However, quantitative analysis of the ECE is not sufficient, and especially time variation of the ECE remains unclear. Understanding the ECE is important because Japan's NDC (nationally determined contribution) assumes the reduction of CO2 emissions through aggressive energy conservation. In this study, we develop a time series model of monthly electricity demand in Japan and estimate time variation of the ECE. Moreover, we evaluate the impact of electricity conservation on CO2 emissions from power plants. The dynamic linear model is used to separate the ECE from the effects of other irrelevant factors (e.g. air temperature, economic production, and electricity price). Our result clearly shows that consumers' electricity conservation behavior after the earthquake was not temporary but became established as a habit. Between March 2011 and March 2016, the ECE on industrial electricity demand ranged from 3.9% to 5.4%, and the ECE on residential electricity demand ranged from 1.6% to 7.6%. The ECE on the total electricity demand was estimated at 3.2%-6.0%. We found a seasonal pattern that the residential ECE in summer is higher than that in winter. The emissions increase from the shutdown of nuclear power plants was mitigated by electricity conservation. The emissions reduction effect was estimated at 0.82 MtCO2-2.26 MtCO2 (-4.5% on average compared to the zero-ECE case). The time-varying ECE is necessary for predicting Japan's electricity demand and CO2 emissions after the earthquake.

Dataset Information

Developing reliable hourly electricity demand data through screening and imputation.

Publications

Developing reliable hourly electricity demand data through screening and imputation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets