One of the evident effects of today’s interconnectivity is the abundant sharing of (and access to) opinions on virtually every topic. Those opinions often are expressed in the form of rankings, which are readily available for restaurants, hotels, tour operators, doctors, college professors, movies, and almost anything that one can fathom. Mutual funds are all too familiar with this phenomenon, as their performance and fees have long been ranked and evaluated by the likes of Morningstar, Inc. In our industry, scrutiny is understood and expected. Nevertheless, when Morningstar first announced its Stewardship Grades in 2004, many received the news with skepticism and concern. While the organization plainly communicated that its goal in launching the initiative was to identify the extent to which fund firms “align their interests with those of fund shareholders,” less obvious were the evaluative methods and sources that Morningstar would use to reach its conclusions. Among the questions bandied at the time were: 

• How does a fund group earn an A (or a B through F, for that matter)?
• Will Morningstar access the necessary information to make qualitative judgments?
• Will grades be assigned consistently across fund complexes?
• Who will do the grading?
• How, if at all, will those grades (or the process used to derive them) be relevant in litigation or regulatory investigations?
• Will (can) these grades really achieve Morningstar’s stated goal?

Twelve years later, and in keeping with the spirit of today’s rank-obsessed world, it seems appropriate to evaluate Morningstar and consider the job that it has done with its Stewardship Grades. Essentially, it’s time to grade the grader. 

Summary of Methodology                       

Morningstar’s grades—ranging from A to F—are “primarily based on [its] qualitative analysis of a fund family’s stewardship of fundholders’ capital.” This analysis focuses on a fund firm’s (i) manager incentives (like, the structure of portfolio manager compensation and portfolio manager investments in the funds they manage), (ii) fees, (iii) regulatory history, (iv) corporate culture, and (v) fund board “quality.” Morningstar awards a maximum of 10 points to a fund group: two points each can be assigned for the first two factors, zero points are awarded for the third factor (though a fund group can lose up to two points for bad regulatory outcomes), the corporate culture accounts for a possible four points, and fund board quality counts for a maximum of two points. 

Manager Incentives and Fees

Despite Morningstar’s assertion that its Stewardship Grades are assigned largely on qualitative factors, its evaluation of these two factors is largely quantitative in that they require less discernment and interpretation. Yet, these two supposedly qualitative components can account for 40% or more of the final grade. To base such a significant portion of a qualitative grade on quantitative factors seems to reflect a disconnect between Morningstar’s intent of providing a largely qualitative evaluation and the quantitative methodology it follows to achieve its goal. 

Substantively, Morningstar has determined that an investment of less than a $1 million dollars by a portfolio manager in a fund that he or she runs, is worth only “partial credit.” Are all funds created equal, and do all portfolio managers earn the same? What if a portfolio manager is in charge of a fund that simply does not fit into his or her financial circumstances? Why should the same investment amount be applicable to a portfolio manager that may be earning four times as much as another? And how did Morningstar establish the $1 million threshold? Morningstar underscores that “if reasonable circumstances prevent a manager from investing in his or her fund,” it will “consider those situations on a case-by-case basis.” Will a portfolio manager be comfortable divulging what may be delicate financial circumstances so that Morningstar can consider clemency for the fund firm? These two factors deserve greater clarification, transparency and, possibly, thoughtfulness. Until we know more, Morningstar earns a B-. 

Regulatory History

Morningstar notes that a fund group cannot earn points for its regulatory history. Instead, it can only lose them. Since when is there only downside to good behavior? And, why should Morningstar assume that two fund groups that have not suffered public regulatory hiccups are created equal? Maybe one is sloppy and has simply been lucky, while the other actually enjoys the fruits of diligent compliance and regulatory oversight. To grade a fund group’s regulatory history without more than a superficial review of what is in the SEC’s public record is like deciding which of two strawberry ice cream brands is better based on the depth and quality of the strawberry color that each reflects. Looks can be deceiving. 

If this factor is to be a meaningful contributor to Morningstar’s methodology, then in must be evaluated through a more discerning lens than an internet search can provide. Morningstar earns a C.

Corporate Culture

Among the questions that inform the grade in this category are whether:

• The fund firm is  “focused on investing or gathering assets;”
• It fosters a “thoughtful repeatable investment process;”
• Its product line reflects its investment expertise;
• Its public disclosures evidence that it is “straight forward” with investors;
• It retains talented investment professionals; and
• Its funds are a “good value proposition overall.” 

The merit of asking these questions is difficult to debate. Investment firms probably ask themselves similar questions frequently and may even grapple with them often—without conceding, by the way, that gathering assets is an inherently bad result. The question, however, is whether Morningstar can truly penetrate a firm and find meaningful answers. 

By its own admission, its “analysts consider how fund firms stack up on various quantitative factors, such as fund manager tenure and turnover, fund performance, fund expenses, and fund launches, mergers, and closings.” A good portion of this qualitative exercise, therefore, is quantitative, and it underscores the tension previously highlighted between Morningstar’s stated qualitative objective and its largely quantitative process. 

To evaluate other aspects of corporate culture, Morningstar “interviews a variety of fund-company personnel, including analysts, portfolio managers, chief investment officers, distribution chiefs, compliance personnel, and top executives.” Does this process, in fact, yield meaningful results? This seems almost impossible to gauge. Interviewing senior executives of a firm (who surely want a good grade) to determine the merits of the firm is somewhat analogous to interviewing a person for a job or talking to the teenage boy who is taking your daughter to the prom to discern his moral character. Can Morningstar really know? Morningstar deserves credit for pursuing a process that could result in a meaningful, qualitative evaluation of corporate culture. Also, of the five factors that comprise the Stewardship Grades, this is the one that most involves  its historic bread and butter. Benefit of the doubt goes to Morningstar. It earns an A-.

Fund Board Quality

Morningstar evaluates  fund  boards by considering whether:

• The board “consistently acts in shareholders’ best interest” (worth one point);
• The directors have “meaningful” investments in the funds—meaning that at least 75% of the board invests more than $100,000 in the funds (worth half a point); and
• Whether the board is led by an independent chair and at least 75% of its directors are “independent” (worth half a point). 

As is true of the other factors, this one too involves significant quantitative data. Half the score is derived from objective criteria. Thus, the previously discussed disconnect or tension carries through to this factor too. 

Because many, if not most, of the industry’s fund groups meet the objective aspects of this factor, for them the board grade will rest entirely on what Morningstar believes it means to act in a shareholder’s best interest. This is the single, most subjective piece of the mosaic that results in a Morningstar Stewardship Grade. So, how exactly does Morningstar form its opinion? “[P]rimarily by analyzing publicly available information, but...also [by] interview[ing] members of the fund's board when possible.” How can a meaningful qualitative evaluation of a governance body be based on publicly available data? Is it the prospectus, SAI or some other document that provides Morningstar a glimpse into the difficult deliberations that boards undertake to reach their most sensitive conclusions? What disclosure tells the story of months of negotiations and of outcomes derived through persistence and tenacity to improve the shareholder experience? Where does a third party find evidence of the countless hours spent by a board or committee chair with senior management executives to ensure a result that would not otherwise have been achieved? Definitely not in shareholder disclosures.

Morningstar’s capacity to formulate a fair opinion about a board rests on its ability to interview the directors. Ponder that. Morningstar is left at the mercy of the party under scrutiny to evaluate the party under scrutiny—an imperfect method to say the least. Further, directors are constrained in what they can and should reveal to Morningstar because providing detailed insights into boardroom deliberations can shatter the precarious attorney-client privilege. The facts that are most relevant to Morningstar are the ones that it is least likely to  know. And who at Morningstar is evaluating the directors? Is it former directors or people with intimate knowledge of the inner workings of a fund boardroom? No. How many board meetings have Morningstar analysts attended (not to present but to observe)? None. This is not to suggest that Morningstar be invited to board meetings. That result could have unintended and severe consequences in the face of litigation or regulatory scrutiny. The point is that Morningstar can’t access what it purports to need. 

Rather than trying to evaluate what it cannot, Morningstar could focus on tangibles that also can provide credible qualitative measures to evaluate a board. It might consider, for example, (i) the number and types of board committees, (ii) the amount of time spent in meetings in light of the size of the fund group, (iii) the expertise of the directors on the board and the likelihood, in light of that expertise, that a director is able to exercise prudent business judgment, (iv) the process followed by boards to identify and select new directors, and (v) the manner in which a board oversees various aspects of fund operations. The possibilities are endless. But they will not be explored until Morningstar recognizes that its methodology has structural and practical impediments that obfuscate rather than facilitate its evaluation of fund boards. Morningstar earns a D.

If intentions played a role in ratings, then Morningstar could set the curve. Focusing the investing public’s attention on aspects of a fund firm that are important reflections of tone but that normally go unperceived is an educational service to that public. But it must be done fairly and with the requisite expertise. It also must be done with greater transparency (a review of Stewardship Grades awarded to different fund groups fails to disclose how Morningstar evaluates the “softer” aspects of its methodology) and with due regard for liability concerns (it should come as no surprise that plaintiffs’ attorneys and the SEC subscribe to Morningstar’s data base). 

Morningstar also should recognize that while it may think that Stewardship Grades are “completely different from the Morningstar star ratings,” the two often are inextricably linked in the minds of those it evaluates. While many in the industry have echoed some of the sentiments reflected here, there are few who are willing to provide Morningstar with constructive criticism for fear that it can affect the relationship with analysts that star-rate their funds. It is against this backdrop that Morningstar should consider what little feedback it receives. Morningstar can salvage a worthwhile idea by implementing it in a more effective and credible manner. Until then, weighing each of its five factors equally, Morningstar’s final grade is a C+.

Paulita Pike is a partner in Ropes & Gray's Chicago office; the views expressed here are her own and not those of her law firm. Elizabeth Plaster, a summer associate at Ropes & Gray, assisted with this article.