Manager Consistency Analysis: A Unique Approach to Evaluating Investment Manager Performance

  print  

Institutional investors are continually looking for the most appropriate ways to measure the effectiveness and consistency of their investment portfolios. For evaluation and manager search purposes, it is crucial that institutional investors utilize both quantitative and qualitative methods of review. The qualitative component is critical to ensure that the investment management process and stability of the firm are established and validated, but the quantitative analysis is the more researched and debated topic. There are many quantitative approaches to evaluate investment portfolios, including absolute, relative and risk-adjusted returns. The challenge is to determine the measure or combination of measures that truly indicate consistently superior performance.

In this article, we evaluate some of the more traditional quantitative approaches to portfolio evaluation. We then present a unique approach that combines some of the more traditional quantitative measures in an attempt to identify active managers that are able to consistently generate risk-adjusted excess returns over time.

Traditional Measures for Evaluation
The most fundamental gauge of the performance of an investment portfolio is its absolute return. The return is what drives the dollar growth of the underlying portfolio, and is therefore the most easily understood approach to portfolio evaluation. However, although measuring portfolios in an absolute return context may be appropriate for some non-traditional asset classes, it is limited in its effectiveness for asset classes that are typically benchmarked against market indices. If not measured against a relevant benchmark or market index, absolute returns can be misleading because they may be comparing different strategies or asset classes.

A more complete approach to investment manager evaluation is excess return - the portfolio return less the stated benchmark return. Relative performance analysis as an evaluation tool is a widely accepted practice because active management is based on the premise of out-performing a passive alternative or benchmark. Using excess return, the institutional investor can easily see what value their manager has added to a passive strategy.

However, absolute and relative performance calculations are point-in-time measures. They are not the most complete indicators of effectiveness because they ignore the risk component. These measures do not capture the month-over-month volatility of returns on either an absolute or relative basis. It is critical to understand the potential volatility of returns relative to the market or to a risk-free investment option. This can be achieved by using the Sharpe Ratio, Jensen's measure or other risk-adjusted measures that show how much return a portfolio has earned per unit of risk taken.

Still another method of evaluating an investment portfolio is to see how it ranks among its asset class peers. When the institutional investor has a finite number of investment managers from which to select, it can be beneficial to perform a relative comparison of their performance by asset class, style, size and strategy. Peer comparison allows the institutional investor to rank their investment managers versus a universe of similar investment managers. This is a popular and valid method for evaluating managers, but once again, it is not complete in capturing the effectiveness and consistency of performance. For example, peer grouping does not account for constraints placed on a manager's asset weighting or security selection, which may be limited by investment guidelines.

Combination of Methods
A single quantitative approach in isolation may not be sufficient when evaluating investment portfolios. Focusing only on absolute or excess returns does not provide a clear picture of the relative volatility of the investment. While peer grouping can be effective, it has its own shortcomings, such as the categorization of portfolios or the inability to properly account for investment limits. In addition, such comparisons are affected by survivorship bias - the elimination of under-performing managers who do not stay in business and drop out of the universe while the investment managers that perform well continue to exist and remain in the universe. Survivorship bias can result in a positive but inaccurately skewed universe history.

At JPMorgan Investment Analytics and Consulting (IAC), we have developed an algorithm that attempts to identify active managers that are able to generate risk-adjusted excess returns consistently over time. Our methodology ranks managers based on a variety of quantitative metrics that are assigned weightings to yield a weighted-average score. The managers are then ranked from highest to lowest based on this score. Comparisons are made between independent time period analyses to produce JPMorgan IAC's Investment Managers of Interest (IMI) for manager searches.

Consistency of Results
IMI is an effort to capture a number of the key statistics that indicate positive risk-adjusted performance. However, a key question is how consistent the performance is over time. In order to measure the consistency of the performance, this analysis is performed over two non-overlapping five-year time periods, and the results from both periods are analyzed for crossover. The top performers are those that score in the top quartile of investment managers in an individual asset class for both time periods.

Sample of Manager Analysis: Small Cap Growth Managers (Click to enlarge)

Source: JPMorgan Investment Analytics & Consulting

This example shows the top-ranked Small Cap Growth manager universe using JPMorgan IAC's IMI approach. The ten mandates listed are the portfolios that ranked in the top quartile for both the 1997-2001 and 2002-2006 time periods. The Calamos Advisors Growth Strategy product ranks first under this methodology, and has provided the most consistent value-added performance in this asset class during this time frame. The IMI output can be used by institutional investors to quantitatively screen managers in a specific asset class. Once this initial screen has been completed, it is up to the institutional investor and their manager search consultant to complete a more thorough analysis of the portfolio as well as an extensive qualitative review.

Conclusion
Although vast amounts of research and analysis have been done on the quantitative evaluation of investment portfolios, there is no perfect methodology. Traditional methods used by institutional investors today have varying levels of effectiveness, but many fall short in measuring the continued consistency of value-added performance. JPMorgan IAC's IMI approach is intended to provide a quantitative ranking method with a weighted scoring approach and a time-period consistency gauge. The blending of these techniques provides a more concrete indicator of which investor mandates have truly added value in the past and which are most likely to continue to add value in the future.