Explainer: When Agencies Rate Companies' ESG Scores, What Are They Actually Rating?

Whatever ESG is, it's becoming more so. In 2021, $649 billion flowed into ESG-focused funds worldwide, up from $542 billion and $285 billion in 2020 and 2019. There is no agreed-upon definition of what the investment strategy looks like, besides coherence on what the acronym stands for—environmental, social, and governance factors that funds take into consideration while analyzing the sustainability of a company. Still, investors are increasingly demanding that their funds go towards ventures that will both survive the impacts of and not further contribute to climate change.

As their clients make these demands, asset management firms are required to come up with a methodology to measure the sustainability of their portfolio companies. ESG is particularly important for the clean energy transition, since one of the most significant ways investors judge the “E” component of ESG is through gauging the carbon reduction goals of particular companies. The Securities and Exchange Commission (SEC) has taken on this issue with urgency, recently proposing new climate change disclosure rules which included guidelines on how registrants would be required to measure and disclose greenhouse gas emissions. For more information on the history of ESG and its broader role within clean energy finance, see this article.  

The broadest ESG challenge facing financial services firms is how to establish and use objective metrics to assess ESG factors. The SEC’s new climate-related risk disclosure guidelines are only one component of greater standardization needs. Asset management firms often rely on rating agencies—independent, profit-seeking research firms—to provide ESG-related information, which frequently differs significantly from agency to agency. How can the agencies on whom fund managers rely on use terms that mean essentially the same things across companies, using objective criteria? How should managers interpret this data against the probably-impossible goal of measuring a company’s ESG virtues? 

The Inconsistency of ESG Scoring 

In order to find a set of evaluative terms to mitigate these issues, look at the current methods that ratings agencies use to create agency-specific ESG scores. According to a July 2021 article from Harvard Business School, among the dozens of ratings agencies in existence, the ones that dominate the market are Sustainalytics, ISS, and MSCI. (ISS provides other services, while a popular information provider called RepRisk does not rate companies). However, the information that ratings agencies provide does not always seem to correspond. Some have cited State Street research to suggest a less than 55% correlation between two major agencies' ratings. Perhaps relevantly, a closer look reveals ambiguities in their methodologies.  

Instead of focusing on companies’ virtues, Sustainalytics and MSCI instead take interest in materiality. Sustainalytics states in a 2021 abstract that it creates ESG risk ratings on exposure – the “extent to which a company is exposed to material ESG issues” and “management”—how well they manage the risk. This results in a company’s quantitative ESG score and its placement into a risk category of either negligible, low, medium, high, or severe. The risk categories are absolute, which means that a company can be measured across companies of various industries directly with each other. The final ESG score is “calculated as the sum of individual material ESG issues unmanaged risk scores”, which makes the ESG score the “overall unmanaged risk of the company.” (Sustainalytics defines unmanaged risk as the sum of unmanageable risk plus the management gap—risk that could be managed but is not).  

Despite the high level of detail provided in their ESG abstract, which resembles an investor relations report, Sustainalytics downplays the quantitative method by which risk factors are arrived at. How does it measure whether or not a company is prone to environmental risk? How many oil spills over the course of a decade merit one score over another? By what percentage does a company within a subindustry need to reduce its GHG emissions in order to be considered viably forward-looking? 

MSCI packages its ESG scores in a different manner. Unlike Sustainalytics’ “negligible, low, medium, high, and severe” risk categories, MSCI decides whether a company is “laggard, average, or leader” and subcategories further with letter-based rankings that span from CCC to AAA. When describing what goes into an ESG rating, MSCI writes that they combine “macro risk data, regulatory agency data, events data, and product risk data.” To arrive at a company’s final ESG rating, MSCI takes the weighted average of “individual Key Issue Scores normalized to ESG Rating Industry peers. Again, how does MSCI determine the exact weight of each Key Issue? The answer provided in their abstract is ambiguous, in that the weightings take into account “the contribution of the industry relative to others, the negative or positive impact on the environment or society, and the timeline within which we expect that risk to materialize.” While MSCI does define its timelines (short term as less than two years, long term as 5+ years), it doesn’t elaborate on how it assesses a company’s environmental impact. Similar to Sustainalytics, therefore, the method MSCI puts into the world comes from subjective decisions.     

"An asset management firm could arbitrarily choose to utilize the scores of one agency over another and arrive at significantly different investment decisions."

Consider how this confusion feeds the current ESG scores for Chevron Corporation. On Sustainalytics, Chevron has a ranking of 43.0 and is placed in the severe category (the highest risk category). But on MSCI, Chevron has a ranking between 4.2 and 5.7 and is placed in the risk category of “average” as opposed to the worst risk category of “laggard.” Two credible ratings agencies who conduct extensive research on the companies they assess should not come up with such different scores. An asset management firm could arbitrarily choose to utilize the scores of one agency over another and arrive at significantly different investment decisions. The subjectivity of the methodologies that rating agencies use calls for much greater standardization and more objective methodologies. How can a ratings agency develop objective criteria with which to create their own ESG score for a company? Where would these objective criteria come from? 

The Cort and Esty Framework 

While it’s difficult to determine a concrete answer, some experts question whether aiming for total objectivity is a worthwhile goal. Two experts affiliated with the Yale Center for Business and the Environment – Todd Cort and Dan Esty – study the issue in a 2020 article in the journal Organization & Environment. They argue that due to the various uses of ESG, “it is likely impossible, and probably counterproductive to the ESG investing industry, to attempt to force homogeneity across investment approaches.” In other words, it is unreasonable to standardize the data for ESG-oriented investors due to ESG’s different functions; some investors use ESG metrics to maximize financial returns, others to reduce volatility, and others still to achieve an environmental or social goal. Instead of attempting to completely standardize data, Cort and Esty believe that data collection ought to demonstrate greater transparency throughout the collection and validation processes while reflecting the varying needs of different investors. To this point, Cort and Esty distinguish among methodological, materiality-based, and impact standards.  

Methodological data standards are standards for collecting data regardless of the type of ESG investor using the data. The framework would include “validation procedures for all reported metrics, timeliness of data, consistent treatment of intangible value, control processes for qualitative data and descriptive information to project forward-looking data.” The methodological ESG data would harmonize with “government regulatory standards and review, third party auditing of the company producing the data, or the data vendors such as MSCI, Sustainalytics, Refinitiv, Bloomberg, etc.” Materiality-based standards drive financial risk and opportunity. There are two kinds of materiality-based standards. First, not-for-profit entities working for the interest of all investors, like the Sustainability Accounting Standards Board (SASB), will assign a company to an industry or sector, making it easier to align disclosure rules for companies placed into specific categories, such as energy or oil and gas. Second, individual companies can determine which information is material through their own stakeholder-informed assessment of impacts and opportunities. An entity like the Task Force on Climate-Related Financial Disclosure (TCFD) will then guide disclosure with a high degree of flexibility. Cort and Esty believe that a combination of the two methods is best.  

Finally, impact data standards are those that demonstrate a carefully structured environmental and social benefit. This is the most difficult information to standardize, given the high levels of subjectivity involved. How does one measure the extent to which a particular venture benefits the environment, or the extent to which a company is at risk of suffering from climate-related disaster? According to Cort and Esty, many companies are aligning with benchmarks like the United Nations Sustainable Development Goals. A 2017 report by the World Business Council for Sustainable Development, an organization committed to corporate sustainability, found that “79% of member companies acknowledged that SDGs in some way and 45% aligned their sustainability strategy with at least some goal-level criteria.” The language even within this one statistic is vague, reflecting but not necessarily organizing the diversity of how companies apply SDGs. Furthermore, while the SDGs are about high-level outcomes, the metrics for impact investing organizations like the Global Impact Investing Network are about quantifying specific outputs, such as the number of lives saved from a particular venture. The discrepancy between “outcomes” and “outputs” complicates these comparing the impact of varying investment decisions. 

Investors have called for environmental metrics, and ESG has imperfectly evolved to gain power. But ESG standardization still requires work. As Cort and Esty argue, "no narrow framework for ESG data will adequately address all investor needs." Instead, development of data standards must reflect varying investor purposes for ESG. While their framework will move ESG disclosures in a more coherent direction, it is probably not as specific as future standards will be. With more clarification and reform underway, future ESG data will underscore the original principle of the investment strategy: allowing clients the opportunity to facilitate sustainability.