Skip to main content

Regulators and Pilots: Now for Our First Act

Circus admission

This essay is the fifth of a six-part collaboration between Benjamin Bovarnick (FES ’18) and Sara Harari (FES ’19, SOM ’19). The collection closely examines the barriers facing adoption of new, advanced energy technologies that can revolutionize electric grid operations and utility business models and spark potential solutions to elicit faster transformation in this expansive industry.


Imagine you’re a TV producer with a great idea for a new show. How do you take this from a brilliant idea to a seven-season smash hit? The first step is to film a pilot episode to shop around. Using the pilot, you can cultivate a market for the show both with audiences and with networks.

Much like your brilliant new show, new energy technologies must also run the gauntlet of the pilot phase, soliciting interest from utilities and developers. In the electric industry, piloting new equipment can be particularly difficult because new, advanced energy-technology pilots must demonstrate that deployment won’t compromise the stability of the electric grid.

However, the stakes for a successful pilot season are much lower for TV than with electric grids. If viewers don’t like a TV show pilot, the show’s developers will lose their investment and maybe some credibility, but the rest of the fall lineup will be unaffected. A failed energy-technology pilot, on the other hand, may disrupt the grid by upsetting the delicate balance of system power flow, leaving a combination of ratepayers and shareholders bearing the financial burden of failure. Due to failed pilots, technology developers may also suffer reputational risk, particularly if they are new, smaller players.

A successful TV pilot yields a contract for a season of follow-on episodes. For the electric grid, a successful pilot should produce increased investment and deployment of the piloted technology, but this doesn’t necessarily happen.

In this explainer, we will focus on how to develop a successful technology pilot. We’ll also discuss why successful pilots don’t always result in wide-scale technology adoption.

Creating a Path to a Pilot

As we discussed in ”Electricity Evolution: Meet the Ringmasters,” state utility regulators provide oversight of the planning and investment decisions made and executed by utilities in their states. Regulators must approve utility capital investments through the ratemaking process described in “A High-Wire Act: Balancing a Modern Grid with Regulated Assets.”

If a utility wants to develop a new project and include the costs in future electric rates, the state regulator must approve the inclusion of this new expenditure in the rate base. However, technology pilots, with more targeted applications, may be financed directly by the utility or through special mechanisms. These mechanisms are established by state elected officials or regulatory commissions.

Testing a Technology or Solving a Problem

Even when a technology has demonstrated successes in the field, a positive result from preliminary testing may still not be enough to convince regulators or utilities to invest in a new pilot project. Pilots are typically developed from a research mindset in which vendors test technology to learn how it can be improved and refined. Meanwhile, utilities may want to see how a new technology would work with the grid.

Erik Desrosiers, formerly a technology-to-market advisor with Advanced Research Projects Agency – Energy (ARPA-E), spoke with Clean Energy Finance Forum about this process. He said this split perspective may have either a “research” mindset or a “disruption” mindset. “Pilots should be organized differently. If it goes well, adopters should be saying ‘We’re going to put this everywhere.’”

Pilot Projects Electricity Evolution table

Evaluating the Success of Pilots with a New Framework

Part of the problem in transitioning from research pilots to disruption pilots is that companies developing pilots may lack a clear goal to broadly replicate successful pilots.

To better understand how pilots can move to a disruption mindset, we need to specify a new vocabulary that differentiates the scale of disruptive technology pilots through three stages of adoption:

Stage 1 - Offline applications: In Stage 1, new technology is deployed and tested while the grid is offline, such as during planned maintenance of systems.

Stage 2 - Limited application: In Stage 2, new technology has some limited and controlled interaction with operations.

Stage 3 - Market transformation: In Stage 3, the technology is pervasive throughout the electric grid and completely changes the economics of the system, fundamentally altering grid geography.

Moving from Stage 1 to Stage 2 requires technical competence. If a technology can demonstrate both affordability and reliability, then there is more likely to be uptake by utilities.

However, moving from Stage 2 to Stage 3 is a much more difficult process than the previous step. This often requires direct stakeholder engagement that involves deciding how to calculate the risks and benefits of each technology and how and why it may improve the grid. This can be an extensive process, requiring time, patience and resources for both technology developers and utilities. 

Taking a Pilot to Grid Scale

Why don’t more successful technologies reach Stage 3? According to Desrosiers, many face “death by pilot” as they languish waiting for support.

In truth, very few developers can hope to deploy potentially disruptive technology without support from regulators. State regulators interviewed by Clean Energy Finance Forum said how important it is to view pilots as opportunities to test solutions to existing problems. They also said how important it is to have a well-defined problem before testing new solutions. They requested that their comments be anonymous.

To adequately assess how a proposed pilot will provide solutions to the electric grid, regulators look for answers to questions like these:

  • Why is this project necessary to meet the needs of the grid at this time?
  • Who will be sharing the risk of the project?
  • Who will benefit most?
  • What technical precedent exists for the technology being proposed?
  • How is this pilot tailored to the local grid?

Utilities and technology vendors must be prepared to clearly demonstrate how new pilots will remedy specific problems facing the grid now or in the future while also clearly justifying the value of upfront capital investments.  For example, if a utility develops an energy storage pilot in a part of the grid that is operating well and is not scheduled for upgrades, regulators may wonder why such a pilot is necessary and why ratepayers should carry the financial burden.

Macky McCleary, administrator of the Rhode Island Division of Public Utilities and Carriers, said regulators must balance risk appropriately between the utility and the ratepayers. They shouldn’t remove all liability. It is unwise to encourage utilities to take on projects that haven’t undergone rigorous assessments.

At the same time, regulators don’t want to reward utilities only if the pilots are successful, because this would discourage pilots for all but the most mature technologies, which likely don’t need pilots to begin with. This balance will vary from region to region and from one utility to another. It thus leads to highly negotiated treatment of new pilots.

Furthermore, public utility commissions operate in different ways depending on the characteristics of their states and the physical conditions of their electric grids. In Rhode Island, for example, the state employs an escalating staircase of performance in which shareholders are increasingly rewarded for improvements.

The specialized nature of financing and implementing pilots that different states employ means that regulators might appreciate the technical lessons learned from pilots taking place in other states but will be hesitant to replicate programs outright.

Just as regulators seek to tailor compensation for pilots around the variable risk tolerance of specific utilities, the regulators Clean Energy Finance Forum interviewed also highlighted the customized way pilot technologies are often integrated into different electric grids.

The characteristics of electric grids vary considerably, so pilots often have to be customized to the utility service territories in order to maximize their impacts.

As a result, utilities may be hesitant to give significant weight to the results of pilots from other service territories. This increases the likelihood that a utility will prioritize conducting pilots of technologies already piloted elsewhere. This can further impede the ability of new technologies to reach Stage 3 of deployment.

Transforming the Market

Despite these challenges, developing technology pilots in ways that make them relevant to new regions is critical to grid modernization and responsible energy development. More advanced energy technology is being developed than ever before. It’s essential that regulators have tools to evaluate new technologies in a cost-effective and time-efficient manner.

Providing regulators with the necessary technical details can be critical to ensuring a pilot project is implemented. One regulator who spoke with Clean Energy Finance Forum said it can be helpful to think of regulatory commissions as judicial panels rather than as active participants when they consider proposals for new pilots.

Utilities and stakeholders must present evidence on why a new technology will work, where it has worked, and why organizations should support it. Regulators are unlikely to seek out this information themselves and are likely to evaluate pilot proposals based on the information in front of them.

With 50 different state-level regulatory commissions in the United States alone, this process may feel like an insurmountable burden for small technology vendors or busy utilities.

That’s why technology vendors and utilities should take a page out of a TV producer’s agenda: pick your market well. Go to the regulators with a clear story of how your technology can solve a problem or bolster grid services. In the same way a TV pilot should improve a station’s prime time lineup, new grid technology pilots should be designed to address grid deficiencies. That’s how the case for the pilot projects will be made clear and win support.

Please contact us directly with any comments or questions or click here to be put on a notification list for future publications in this series.