As the financial community enthusiastically embraces environmental, social and governance investments, it needs to know two things:

• ESG scores are measured differently from one agency to another and can therefore be a measure of inconsistent performance.

• Some experts are increasingly skeptical that there is a direct correlation between ESG scores and more sustainable activities.

In a recent Q&A, Lenora Suki, Product Manager for Sustainable Finance Solutions at Bloomberg LP, summed it up: “Today, sustainability information is often non-financial and so diffuse and diverse that it is is difficult for decision makers in strategic and operational decisions.

That doesn’t mean it’s time to cancel ESG. All signs point to consumers wanting to buy products from companies that match their values, whether it’s protecting the environment, empowering women, or tackling societal inequalities.

And investors are increasingly enthusiastic about ESG investments.

According to the Global Sustainable Investment Alliance, global sustainable investment reached $ 35.3 trillion at the start of 2020, an increase of 15% over the past two years. In addition, the same report found that sustainable investment assets under management accounted for over a third (35.9%) of total assets under management in 2020, up from 33.4% in 2018.

A majority of business leaders fully support sustainable development measures, while also being aware that their efforts may not have a significant impact. In 2016, a joint United Nations / Accenture survey found that more than three-quarters (78%) of CEOs surveyed believed that companies’ efforts would contribute to the global goals, adopted by the United Nations as a universal call to action to put end poverty and protect the planet.

In the most recent survey (2019), only a fifth of CEOs surveyed felt that companies are actually making a difference on the global sustainability agenda. Nonetheless, almost half (48%) remain committed to implementing sustainability as part of their overall operations.

The disconnect between accelerating ESG activity and confidence in results should serve as a wake-up call for companies and investors, according to Kenneth Pucker, SM ’90, the former COO of Timberland.

The challenges include discrepancies in ESG metrics, persistent data quality issues and the pervasive greenwashing that can occur when companies attempt to present their data in a positive light, said Pucker, who published in June “Overselling Sustainability Reporting ”in Harvard Business Review.

“ESG rating agencies take self-reported data from companies on their corporate social responsibility [CSR] activities, add their own information and weights, and mix them in a cauldron to get a rating for a company, ”said Pucker, now a lecturer at the Fletcher School at Tufts University.

“One problem is garbage in, garbage out,” Pucker said. “The report is not complete, the results are mostly unaudited and they are not comparable, so ESG ratings often use bad data that is not audited, extrapolated and interpolated.”

Why ESG ratings diverge – and what to do about it

The MIT Sloan Sustainability Initiative has joined the cause with the launch of the Aggregate Confusion project, aimed at improving the quality of ESG measurements.

ESG ratings are used by individual investors and fund managers to guide investment strategies by understanding the potential sustainability risks to company performance. Companies are also leveraging ESG ratings as an internal benchmarking tool to help improve sustainability performance and for possible public relations opportunities.

The problem, as defined by the MIT project, is that ESG data is noisy. This leads to a divergence in the ratings of independent agencies that assess and assign ESG ratings to companies, according to Florian Berg, a researcher working on the challenge. In turn, this leads to confusion and a much lower likelihood that ESG ratings have a direct correlation with financial performance, compromising their usefulness as an investment tool, Berg said.

Berg, with research affiliate Julian Koelbel and MIT professor SloanRoberto Rigobon, published three articles on the divergence of ESG ratings and its impact. Using a common taxonomy and various mathematical modeling techniques, the MIT team was able to integrate the different approaches of rating agencies into a cohesive framework to better understand rating differences.

In the initial document, the researchers identified the following:

There is an ambiguity around ESG ratings. The study found that the correlation between six leading rating agencies (KLD / MSCI Stats, Sustainalytics, Vigeo Eiris / Moody’s, RobecoSAM / S & P Global, Asset4 / Refinitiv and MSCI) was on average 0.61. In comparison, the traditional credit ratings of Moody’s and Standard & Poor’s correlate at 0.99.

The result suggests that ESG ratings do not accurately reflect ESG performance, making it difficult for policymakers to identify outperformers and laggards.

Diverging ratings are also hampering companies’ motivation to improve their ESG performance, the document says, as rating agencies are sending mixed signals about what to focus on and what is valued in the industry.

There are three factors causing the odds divergence. Scope divergence occurs when ratings are based on different attributes – for example, one rating service includes carbon emissions or labor practices while another does not – leading to rating inconsistencies. Measurement deviation occurs when agencies measure the same attributes, but do so using different raw data, resulting in different ratings. The last factor concerns weight discrepancy, which arises when ESG rating agencies take different views on the relative importance of attributes.

They also identify something they call the “rater effect”, which is that a company that scores high in one category is more likely to score high in other categories, especially from from the same reviewer.

As part of its ongoing research, the team is trying to understand the effects of ESG investment flows on stock prices and company behavior. They are also developing smarter ways to aggregate ESG factors into composite indices for use by portfolio managers and passively managed funds.

ESG ratings and stock market performance

A subsequent article describes in detail a noise correction procedure designed to combat bias caused by noisy ESG data and rating divergence. Using various modeling techniques, the research team was able to demonstrate that ESG ratings actually have a much larger impact on stock returns than expected.

Without the correction procedure, standard regression estimates of the impact of ESG ratings on stock returns were biased down by around 60%, Berg said.

“We found that the link between financial performance and ESG performance was stronger because it is not eclipsed, hidden by noise,” explained Berg.

The practical takeaway from the research: “Relying on scores from multiple complementary assessments works better,” Berg said.

Filling ESG gaps

The next steps for the Aggregate Confusion project are to work with industry partners to collect data, solicit feedback and create a set of best practices for use by ESG rating providers as well as companies reporting CSR data to for disclosure purposes.

Improving the measurement of raw data and formalizing best practices is all well and good, but Pucker believes that it will take global standards and formal audits to demonstrably strengthen the usefulness of ESG ratings.

Pucker explained that the environmental impacts of sustainability reporting and ESG investments have been oversold, hampered by a lack of monitoring of what goes into sustainability reporting as well as huge gaps in the data tracked to generate insight. full picture of the carbon footprint.

As an example, Pucker cited Scope 3 emissions, which include all upstream and downstream emissions generated by suppliers and customers. Figures from CDP, a global aggregator of data on corporate carbon emissions, show that only about half of companies disclosing CSR data report Scope 3 emissions, even though this constitutes the bulk of their impact on greenhouse-effect gas.

Specifically, Pucker argues that the problem is bigger than noisy data or valuation discrepancies.

Fundamentally, ESG investing can offer better financial returns to investors, but it is not designed to produce the desired impact in solving environmental problems.

“Asset managers have an incentive to generate returns on equities, not to solve the water problems in Egypt or deforestation in Malaysia,” Pucker said. “That’s not what they’re paid for.”

Pucker believes impact investing is in a better position to make a bigger difference. Estimated by the Global Impact Investing Network to have a market of $ 715 billion, impact investing targets companies or institutions that have specific intentions to create products and services that generate social or environmental impact – for example , one who actually develops a technology that solves water problems.

This type of investment in companies pursuing climate technology to solve transportation, agriculture, energy and land use issues is more likely to bring about change, Pucker said.

More work to do

Regardless of this debate, Pucker said the industry has a lot of work to do to improve on CSR reporting and make ESG ratings a more effective tool.

Making reporting mandatory, creating comprehensive standards on what to report, increasing the transparency of what is reported, and adopting some form of sectoral or regulatory governance is part of the equation. Efforts like the Aggregate Confusion project to improve data quality and ESG metrics are another critical step.

“All of these things will help the correlation of ratings converge between different reviewers, but for me it’s not a leveraged solution for impact,” Pucker said. “Ultimately, tackling environmental and social externalities requires an effective regulatory policy. “

Read more: 4 strategies for a sustainable business