hide
Free keywords:
energy consumption, intervention, Systematic Review, Network Meta-Analysis, policy evaluation
Abstract:
Policymakers have little time left to prevent the worst impacts of climate change and limit global warming to well below two degrees. However, a systematic assessment of the available scientific evidence—that is up to date—is not always available to understand what climate policies work, to what extent, in what context, why, and for whom. This is also true for demand-side policies, including those that use behavioral change to reduce energy demand and the related carbon emissions. There is an ever-burgeoning literature on policy interventions that target behavioral change among households, with new insights and evidence of their efficacy in different contexts. This living systematic review (LSR) and network meta-analysis (NMA) synthesizes this evidence to provide timely, rigorous and up-to-date insights on this topic. Our LSR and NMA integrate the evidence available from multiple disciplines to answer the following questions: (1) to what extent can information, behavioral (including feedback, social comparison and motivation), and monetary based interventions reduce energy consumption of households; (2) what the relative effectiveness of interventions is; and (3) how effective are the combinations of different interventions. In doing so, we also pilot an LSR for climate policy solutions and share learnings with the community. To fulfill these objectives, we searched the academic and gray literature for experimental and quasi-experimental studies that quantitatively assessed the impact of either behavioral, monetary, or information interventions (or a combination of these) on energy consumption (including electricity and heat) of the households in residential buildings. We searched the relevant databases: Web of Science Core Collections Citation Indexes, Scopus, JSTOR, RePec, Google Scholar, and gray literature repository Policy Commons to retrieve over 109,000 potentially relevant article abstracts and apply machine learning algorithms to identify the most likely relevant papers. Note that with this update, that includes the relevant literature published till end of December 2024, we added roughly 53,000 potentially relevant documents to the previously existing pool of potentially relevant literature from Khanna et al. (2021). A team of four reviewers screened the titles and abstracts of studies identified as being potentially relevant by the machine learning algorithm, with full-text assessments and double-coded data collection following for a set of included studies. The effect sizes reported by different studies were harmonized to Cohen's d for synthesis. We used a multilevel random effects model and NMA for calculating the average intervention effect. We adjust our estimates for possible small-study effects (publication bias). The NMA allows us to visualize the relative efficacy of the interventions through rankograms and cumulative ranking probability plots. Unlike previous meta-analyses in this field of research, this study also implements a comprehensive risk of bias criteria for assessing the quality of each study using a modified version of the framework recommended by the Center for Environmental Evidence. We identified 213 relevant studies and conducted meta-analyses on 192 studies that provide quantitative estimates of the relationship between behavioral, monetary, and information incentives and reduction in energy consumption of households. The studies together represent evidence from 40 countries and 6,528,923 households (average total sample size of 33,216). The studies were of varying quality, with the presence of methodological weaknesses across the included studies. We find an overall average effect size of Cohen's d = 0.22 or 0.13 after adjusting for potential small-study bias across. Such an effect corresponds to approximately a 4%–6% reduction in energy consumption. Monetary incentives have the largest average effect, followed by some behavioral (motivation) and information interventions. Combining interventions can also increase effectiveness; for example, combining information, social, and behavioral (motivation) interventions has high average effects. Our analysis finds that behavioral, monetary and information interventions taken together on average have a small-moderate effect on energy consumption of households. Some intervention combinations yield substantially larger impacts—especially when considered at scale. However, the practical consequences of the average effect sizes reported in this review depend on at least three factors: how often a person makes decisions that could be influenced by the interventions under investigation, the scalability and cost of interventions, and the welfare consequences of the interventions. The fast-growing literature on behavioral, information, and monetary interventions in household energy consumption makes this field a fitting case study for a “living” review assessment. Of the 663 effect sizes used for synthesis, about half come from studies produced after 2020 that were not included in previous reviews on the topic. However, there are significant challenges with consistently updating a review, most importantly, in terms of maintaining consistency in the identification and coding of studies, given resource constraints and changing personnel. Applying machine learning algorithms during abstract-level document screening helped us significantly reduce the manual effort involved in identifying the relevant literature.
