At E Source, we talk to demand-side management (DSM) industry professionals every day about the work they’re doing and the challenges they face on the job. Two common sentiments that we often hear from DSM program managers and execs about performance are:

This year’s budget was the same as last year’s, and we expect that next year’s budget may be smaller. And yet our savings goals continue to increase, despite “moving target” baseline assumptions. Making ends meet is a serious challenge.

We set our program and portfolio goals based on internal drivers and local conditions. We interpret program cycle performance relative to our unique operating environment. It’s impossible for us to make direct comparisons to other administrators’ programs.

We regularly profile individual DSM programs as case studies in our reports, presentations, and industry events. These program profiles are invaluable for cross-pollinating ideas and sharing best practices across the industry, but it isn’t always easy to tell how transferrable each case study is from one administrator to another. And perhaps even more importantly, case-by-case program reporting doesn’t tell the larger story of both where we are as an industry and where we’re headed.

These are some of the reasons behind recent efforts to expand our industry performance benchmarking capabilities. In 2014, E Source partnered with QuadROI on an ambitious project to gather detailed information about utility DSM portfolio and program performance and make that information readily available to our members. At the time of this writing, we have gathered performance data on thousands of discrete programs sponsored by 125 administrators across 41 states and provinces in North America. Although nearly all of those data are available within the public domain via utility regulatory filings, E Source DSM Insights is the only tool in our industry that hosts all of this information in one easily accessible interactive platform. In DSM Insights, we’ve pulled information from source documents, structured it, served it up through a business reporting interface, and linked back to the original documents to enable quick cross-referencing.

Now that we’ve established a critical mass of information about DSM program performance, we can begin to ask more powerful questions about the industry as a whole. By taking a data-driven approach to industry analysis and benchmarking, we can address big-picture questions about the DSM industry, including:

• How have DSM spending and goals changed over time?
• How does spending vary by region, program category, or market sector?
• Which program categories are most cost-effective and is this expected to change over time?
• How is portfolio composition expected to change, and which programs are expected to deliver the most savings in the future?

We recognize that making comparisons about relative program performance is fraught with risks of unfair comparisons. With that sense of caution, we can look at the data in aggregate to see what can be learned, recognizing that we may be confusing multiple factors. Identifying areas of confusion is the first step in moving toward better and more-standardized industry performance tracking and reporting.

Trends in DSM Spending

The US Energy Information Administration (EIA) has been gathering data on electric DSM program spending and activity for more than two decades through its annual Electric Power Industry Report Survey (Form EIA-861)—the survey results, including detailed electricity data files, are available for download on the EIA website. The most recent survey data for 2012 includes reporting on DSM activities from 789 electric utilities operating in the US, with total spending across energy-efficiency and load-management programs totaling just over $6 billion (Figure 1).

FIGURE 1: DSM spending is on the rise

Reported spending over the last two decades shows an industry that has experienced steady growth over the last ten years after a significant decline in the mid-1990s, due in large part to utility deregulation. Over the last five years, spending has nearly doubled from just over $3 billion in 2008, which equates to a compound annual growth rate (CAGR) of more than 17 percent.

Though the administrator-reported DSM portfolio totals bring with them the inherent risk of misinterpreting trends based on additions and omissions (for example, a program administrator not reporting survey results in a particular year), where program results are consistently reported by their administrators year over year, insights can be gleaned from the trends in the growth of existing portfolios—those with some reported DSM spending in both the current and prior year (Figure 2). The data show that existing portfolios, in aggregate, saw double-digit annual growth starting in 2007, before slowing down to the 7 percent increase observed in 2012.

FIGURE 2: Growth in annual spending shows periodic cycling

Of the $6 billion of electric DSM spending reported in 2012 by more than 750 respondents, nearly two-thirds ($3.7 billion, or 66 percent) was accounted for by 30 program administrators with budgets over $50 million. Segmenting the population to look at the trend in these largest, established portfolios yields additional insights (Figure 3).

FIGURE 3: Portfolio growth over time

Total spending among these large portfolios actually decreased from 2008 to 2009 during the height of the global financial crisis and related economic slowdown. Growth rates have remained in the single digits since, at 4 percent year-over-year growth in 2011 and 2012.

A very different picture is seen for the aggregate spending level among smaller portfolios, including many in states that have seen new and expanding energy-efficiency resource standards. Total growth in spending for portfolios with prior year spending of under $30 million exceeded 25 percent every year from 2007 to 2011, then slowed down significantly in 2012 with total spending growing by only 13 percent.

Introducing DSM Insights

Although surveys such as those conducted by the EIA are useful in that they provide a rear-view mirror look into historical spending trends and program performance, industry decision-makers will be best able to make prudent resource decisions and add value for program administrators if they have a better sense of where the industry is headed.

With the E Source DSM Insights tool, we seek to liberate data that are available to the public but buried in difficult-to-access utility commission case proceedings, program evaluations, and other sources where portfolio budgets, planned impacts, and annual reports on actual spending and energy impacts are disclosed. By aggregating these source documents, making them searchable, and bringing program-level spending and impact details into a single structured repository with powerful tools for data visualization and discovery, DSM Insights can help program decision-makers benchmark performance, learn from program designs that are over- or under-performing, and spend less time and effort simply trying to access information.

Confirmation of Slowing Growth

Tracking multiple historical program year cycles and available approved and submitted program portfolio plans yields a clearer picture for the industry spending growth outlook. Portfolio plans confirm the slowdown in spending and the likelihood that it will continue at least into 2015 (Figure 4). Although the median year-over-year growth for tracked portfolios (currently all portfolios with annual spending over more than $10 million) was 20 percent in 2010, the observed rate has been half that in 2011 and 2012. Preliminary reporting and committed budgets for 2013 and beyond suggest that single-digit growth will be the norm, with a significant share of portfolios having experienced contractions in 2012 or earlier.

FIGURE 4: DSM budget growth is slowing across the board

The trajectory of DSM program spending is most directly a matter of state-level policy and mandates, so it is to be expected that growth is highly variable across different geographic regions. Figure 5 shows that the Midwest region of the US continues to experience substantial growth, with a median growth rate of 17 percent and the top quartile seeing rates in excess of 30 percent year-over-year growth in spending.

FIGURE 5: The Midwest and Northeast US lead in annual budget growth for 2014

Some regional trends seem to make intuitive sense, but others don’t. Western states, with some of the longest-tenured major portfolios and strong legislative and policy supports, demonstrate the most stability in spending growth—between 2 percent and 14 percent for the majority of tracked portfolios. And despite having a fair number of program administrators seeing portfolios ramp up, a significant proportion of program administrators in the South US region are seeing portfolio spending contract in 2014.

Pressure on Portfolio Performance

We were able to chart the achievement results for several dozen electric program portfolios over the last three years of reported results (Figure 6).

FIGURE 6: Program managers are having a harder time meeting goals on-budget

The challenge of slowing budget growth is not the only macro-industry dynamic that can be observed. Although 51 percent of tracked portfolios exceeded their goals while remaining under budget in 2010, that number dropped to only 27 percent by 2012.

Further, although only 13 percent of tracked program administrators exceeded budgeted spending in 2010, that proportion had more than doubled within two years, with 31 percent overspending in 2012. Nearly half (48 percent, up from 38 percent in 2010) of tracked program administrators did not reach their overall kilowatt-hour savings goal in 2012.

Though the distribution has moved down (lower impacts relative to goal) and to the right (spending over budget), the proportion of portfolios falling short of cost-effectiveness (dollar per kilowatt-hour) targets—29 percent—has remained constant. This combination suggests that budget levels have not kept pace with goals, and program administrators have had to do more with less, a directive we hear from our members on a regular basis.

Evidence to support that conclusion also comes from examining the trend in the share of spending by targeted market sectors (an attribute assigned in data production for DSM Insights based on program eligibility definitions and descriptions of targeted customer populations). In fact, from 2010 to 2014, the overall share of spending on residential-focused programs is projected to decline from a 32.4 percent share in 2010 to a 27.0 percent share in 2014 budget plans—with spending moving largely to those programs that serve commercial and industrial accounts (Figure 7).

FIGURE 7: Commercial and industrial programs fill the gap

To be fair, programs that specifically target low-income populations have grown as a share of spending, which accounts for some of the decrease in general residential program spending. However, other key segments including programs that target small businesses and the broader multifamily housing market are receiving a shrinking share of the overall budgetary pie, despite the fact that about half the administrators we’ve talked to are required to spend DSM dollars on programs targeted to the same sector from which they were collected.

Looking more closely at the cost-effectiveness performance of residential programs, there is clear evidence to suggest what might be forcing these shifts, particularly in the face of slowing budget growth. In 2010, on a net (not gross) savings basis, the overall first-year savings from general residential programs was $0.18 per kilowatt-hour (kWh), based on administrator reporting, which falls comfortably under the average $0.20/kWh targeted by portfolio plans. Despite the targeted rate ratcheting up to an average of $0.23/kWh in 2012, overall savings came in at a significantly higher cost in aggregate: $0.26/kWh (Figure 8). Further escalation is anticipated based on 2014 budgets as the industry continues to adjust to a number of contextual factors, including changes in lighting efficiency standards mandated by the Energy Independence and Security Act of 2007 (EISA).

FIGURE 8: Residential programs are expensive

Residential programs have seen a jump in achieved dollars per kilowatt-hour (Figure 9)—programs have been delivering significantly more cost-effective savings from hard-to-reach segments, including low-income and multifamily sectors (both of which saw a more than 30 percent drop from 2010 to 2012), as well as small and midsize business. Despite these advances, budget plans for 2014 show a shrinking share going to small business programs and those targeting broader multifamily housing populations beyond low income. Although low-income populations continue to see an increased share of spending, the relatively higher cost per unit of savings for these programs puts added pressure on portfolio budgets.

FIGURE 9: All sectors show improvement except residential

Comparing Apples to Apples

When we aggregate DSM performance data into a single repository and make it easy to access, industry trends and insights become apparent. With the information we now have readily available, we can quickly find out how administrators are performing at both the program and portfolio levels, which programs they’re running, how those programs are performing, and how it’s all changing over time. Though we’ve seen that budgets aren’t growing like they once were, we also know that industry spending is cyclical and that overall DSM portfolio spending could easily bounce back again.

For more details on our DSM program and portfolio benchmarking tool, see our DSM Insights marketing page.