Advanced Measurement & Verification (M&V) Research
Summary of Work
The growing availability of smart meter energy data and energy analytics software presents tremendous promise to accurately measure project energy savings in near real-time using advanced M&V methods. The last decade has seen significant progress to develop advanced M&V tool testing protocols, set guidance around accuracy and documentation, and to demonstrate the practical application of advanced M&V through pilots. Recent research has explored new opportunities and remaining technical challenges, including:
- How effectively can advanced M&V methods capture the timing and location of energy impacts for emerging demand-side management strategies such as load shed (demand response) and load shift, at the building level and at grid level?
- How can advanced M&V tools and guidelines account for energy impacts in buildings that are unrelated to the interventions targeted for M&V (so-called “non-routine events”)?
- With what levels of uncertainty and confidence can today's advanced M&V tools quantify savings, and what will industry require as indication of a good result?
- How can practitioners leverage automation to streamline the M&V process while using their expertise to attain a quality result using accurate M&V tools?
Below find summaries of Berkeley Lab's past and ongoing research to address these questions:
1. Grid services support: Berkeley Lab collaborated with Sacramento Municipal Utility District (SMUD) to characterize the impact of SMUD’s energy efficiency program portfolio at grid level. The results of the research demonstrated that advanced M&V methods can quantify hourly savings shape at substation/feeder level and seasonally (documented in a 2020 ACEEE Summer Study paper and a journal article). In 2022 Berkeley Lab launched the Grid Services M&V Working Group to define short and medium term industry needs for enabling resources such as tools, guidance, standards, etc.
2. Non-routine events: In collaboration with industry stakeholders, Berkeley Lab has been working on research, guidance, and tools to handle non-routine events (NREs). A 2020 ACEEE Summer Study paper addressed fundamental questions around frequency, magnitude, and directionality of NREs (paper and presentation). We also supported the Efficiency Valuation Organization (EVO) on development an application guide for identifying and accounting for NREs (link). We also developed and tested algorithms for detecting NREs and semi-automated non-routine adjustments (journal article).
3. M&V tool testing: Berkeley Lab developed a test procedure to objectively compare and contrast the accuracy of advanced M&V tools. The test procedure establishes metrics and benchmarks to assess the general robustness of both proprietary and open M&V algorithms. The test procedure is described and applied to monthly and interval open source models in (Granderson et al. 2015 and ACEEE Summer Study Presentation 2014); it is replicated with a more diverse test data set, consensus-based metrics, and both open and proprietary tools in (Granderson 2016a and 2015 webinar presentation). In 2019 we collaborated with the Efficiency Valuation Organization (EVO) to launch an online portal incorporating the test method. In 2020 we published similar tool testing research with a focus on peak demand prediction accuracy (journal article and webinar deck), and our ongoing work is extending this research to consider M&V for load shift applications (webinar deck and recording).
4. Accuracy, documentation, and reporting requirements: In collaboration with industry stakeholders in diverse regions across the US, Berkeley Lab worked to identify documentation and reporting requirements for the use of these analytic-based approaches to M&V (guidance document and summary presentation). More recently, we collaborated with EVO to develop a 2020 white paper that provides a snapshot of the current state of advanced M&V practice (white paper), which preceded the co-development of an EVO advanced M&V application guide (publication date TBD). We have also studied methods for quantifying uncertainty of advanced M&V savings estimates (journal paper).
5. M&V 2.0 pilots: Following successful demonstrations on historic program data, Berkeley Lab provided technical assistance to design and execute more structured pilots:
- Commercial and residential advanced M&V pilots were conducted in Connecticut, in collaboration with CT DEEP, Eversource, and United Illuminating (Commercial Factsheet, Residential Factsheet).
- A pilot in Seattle conducted in collaboration with Seattle City Light and Bonneville Power Administration (Project Final Report here).
- A pilot in California conducted in collaboration with the California Public Utilities Commission (Factsheet).
- A pilot in Sacramento conducted in collaboration with Sacramento Public Utilities District (Factsheet). A February 2020 webinar deck on grid-level analysis results from this pilot can be downloaded here.
The pilots were designed to more formally test the value proposition associated with these tools. Additional case studies documenting advanced M&V application collaborations with Efficiency Vermont and BC Hydro are provided in the table below.
6. RM&V Reference Tool: In 2017, Berkeley Lab published the open source M&V tool "RM&V," a reference implementation of advanced M&V analytical best practices. RM&V allows for multi-site analysis, model goodness of fit screening, and support to visualize savings as they accrue and identify potential non-routine events. The tool offers two modeling options: a piecewise linear regression that uses time of week and temperature ('TOWT') as driving variables, and a machine learning method. A 2021 presentation and demonstration on RM&V can be accessed here. More detail on the TOWT modeling method can be found in this 2011 report, and a link to a paper on the gradient boosting machine learning method is included in the table below.
Additional Resources
| Title and Description | Resource Type | Date |
|---|---|---|
|
Accuracy of hourly energy predictions for demand flexibility applications In this paper, we first characterize the hourly predictive accuracy of six commonly used baseline models in an application context of quantifying building-level load shift. Our analysis revealed insights such as hours of the day (afternoons), periods of the week (weekends), and seasons (summer) that were predicted with more accuracy than other time periods. Secondly, we provide the first published investigation of the potential for baseline erosion from repeated dispatch of building load shifting. |
Journal Article | 2023 |
|
Detecting the undetected: Dealing with non-routine events using advanced M&V meter-based savings approaches This paper documents the state of the art in non-routine event (NRE) quantification and analysis. The results |
Conference Paper | 2022 |
|
Assessment of Model-Based Peak Electric Consumption Prediction for Commercial Buildings This project explored whether advanced M&V regression approaches offer improvements over simpler averaging approaches for peak load prediction in commercial buildings. Predictive accuracy of hourly loads was tested using a large dataset of commercial buildings' meter data, applied to eight baseline algorithms falling into three model types: time-of-week-and-temperature (TOWT), day-matching, and weather-matching. |
Journal Article | 2021 |
|
Spatio-temporal impacts of a utility’s efficiency portfolio on the distribution grid This work demonstrates that advanced measurement and verification modeling techniques can be used to estimate the spatio-temporal grid impact of a portfolio of energy efficiency programs. It extends measurement-based methods to an entire Demand Side Management portfolio and uses a single model to predict annual as well as seasonal building energy use with near-zero bias. |
Journal Article | 2020 |
|
Detecting the undetected: Handling non-routine events for meter-based savings approaches This paper documents the state of the art in NRE quantification and analysis. The results of research to quantify the frequency, nature and direction of NREs, and methods and metrics for determining a trigger threshold for taking action on NREs are presented. The paper also documents the latest technical guidance on application of NRE detection and adjustment methods. |
Conference Paper | 2020 |
|
Meter-Based Assessment of the Time and Locational Benefits of a Large Utility’s DSM Portfolio This paper presents an analysis of interval data across more than 25,000 customers and twelve substations, from the Sacramento Municipal Utility District. The results show for different locations on the grid: achieved savings and the impact on grid consumption; hourly savings shapes for DSM program participants and non-participants, and how those shapes vary with season; and the impact of the programs on peak demand |
Conference Paper | 2020 |
|
Realizing high-accuracy low-cost measurement and verification for deep cost savings This report summarizes the key outcomes of a multi-year project, including a literature review of M&V tools, development of data-driven techniques for identifying and adjusting for non-routine events, pilot deployment of advanced M&V methods in the Pacific Northwest, and engagement with regulatory bodies on the topic of advanced M&V. |
Project Report | 2019 |
|
Evaluation of Methods to Assess the Uncertainty in Estimated Energy Savings This paper evaluates the accuracy of methods used to quantify the uncertainty of measurement & verification (M&V) savings estimates. The findings of this work indicate that standard methods for estimating savings uncertainty over the post-installation period tend to underestimate the uncertainty, and that this tendency is stronger for hourly models than for daily models. |
Journal Article | 2019 |
|
Statistical Change Detection of Building Energy Consumption: Applications to Savings Estimation A key challenge when applying advanced M&V is to identify and adjust for non-routine events (NREs), i.e., changes in building energy consumption unrelated to the project for which savings are being quantified. In this work, a data driven methodology was developed to (partially) automate, and therefore streamline the process of detecting NREs in the post-retrofit period and making associated savings adjustments. |
Journal Article | 2019 |
|
Gradient Boosting Machine for Modeling the Energy Consumption of Commercial Buildings The gradient boosting machine is a machine learning algorithm that is gaining considerable traction in a wide range of data driven applications. In this journal article an energy consumption baseline modeling method based on a gradient boosting machine was proposed and assessed. |
2018 | |
|
The State of Advanced Measurement and Verification Technology and Industry Application Article characterizes the features of sixteen automated M&V software tools, and reports on status of utility programs implementing whole building savings approaches. |
Journal Article | 2017 |
|
DOE M&V 2.0 Project Overview Powerpoint presentation to industry stakeholders, providing an overview and recent results of Berkeley Lab's research on advanced M&V. |
Presentation | 2018 |
|
The Status and Promise of Advanced M&V The paper identifies the benefits, methods, and requirements of advanced M&V and outlines key technical issues for applying these methods. It presents an overview of the distinguishing elements of M&V 2.0 tools and of how the industry is addressing needs for tool testing, consistency, and standardization, and it identifies opportunities for collaboration. |
Paper | 2017 |
|
BC Hydro to Deliver M&V 2.0 to Customers and Program Administrators (Case Study) In working to develop a customer-facing M&V 2.0 tool, BC Hydro partnered with researchers at Berkeley Lab to analyze a set of historic program data. |
Case Study | 2017 |
|
Efficiency Vermont Takes Action to Apply M&V 2.0 (Case Study) With a desire to develop an M&V 2.0 platform for program delivery, Efficiency Vermont worked with researchers at Berkeley Lab to test the performance of M&V 2.0 open-source algorithms against a small set of historic program data. Parallel analyses were conducted by VEIC and the Berkeley Lab team to ensure repeatability of results and correctness of code implementation |
Case Study | 2017 |
|
Will the Measurement Robots Take Our Jobs? An Update on the State of Automated M&V for Energy Efficiency Programs This ACEEE Summer Study paper details metrics to assess the performance of advanced M&V approaches, a framework to compute the metrics, and recent test results. We also discuss the accuracy, cost, and time trade-offs between advanced and traditional M&V. The paper also includes early results of advanced M&V pilot. |
Report | 2016 |
|
Presentation on testing procedure and accuracy of 10 public and proprietary baseline models Presentation gives an overview of the test method developed by Berkeley Lab to assess baseline modeling capability for M&V software tools. The test method was applied to ten modeling methods/tools with results presented here. Additional links are provided in item 1 above. |
Presentation | 2015 |
Contact
