This article is part of an ongoing series exploring flaws in popular investment risk and skill evaluation techniques. We focus on the most common pitfalls that have been particularly costly for asset managers and fund investors.
Investment risk and skill evaluation frequently relies on returns-based style analysis, and returns-based performance attribution. These techniques perform regressions to compute portfolio betas (exposures to systematic risk factors) and alphas (residual returns unexplained by systematic risk factors).
The simplicity of returns-based approach has made it popular. It is often the only practical method for evaluating multi-asset-class portfolios that span commodities, public securities, derivatives, and private investments. However, this simplicity comes at a heavy cost, which we explore in this and upcoming articles.
The Assumption of Stable Exposures
A key assumption of most returns-based analyses is the constancy of factor exposures. This assumption breaks down for active managers.
A good example of this is the Fairholme Fund, ticker FAIRX. The Fairholme Fund has dramatically varied its bets over the past ten years. A simple linear regression of historical fund returns against the US Market is below. It estimates beta at 1.14 and monthly alpha at 0.12%:
This and similar regression approaches form the basis of returns-based analysis. This convenient but overly simplistic analysis does not attempt to estimate market exposure at each point in time. Hence, the beta of 1.14 may not be representative of the current or historical US Market exposures of a fund.
The Variation of Exposure
To test this beta, we estimated monthly US Market exposures of the fund using the ABW Peer Analytics US Equity Risk Model. For each month, we estimated the betas (exposures) of individual positions to the US Market Factor and aggregated these into monthly estimates of aggregate portfolio beta. It turns out that over the past 10 years the Fairholme Fund has varied its US Market Exposure between 50% and 170%:
The US Market Exposure ranged from 50% to 170% and was infrequently anywhere near 110%. It turns out, risk estimated using returns-based analysis is inaccurate most of the time. It is also an inaccurate estimate of the true mean exposure:
Consequences of Varied Exposures
Returns-based analysis can produce deeply flawed estimates of the current risk for funds that vary their bets. Even estimates of average risk and style may be inaccurate. In the case of the Fairholme Fund, the returns-based estimate of US Market exposure—around 110%—is well off from the current portfolio exposure—around 140%. To make matters worse, there is a domino effect: Returns-based performance attribution builds upon any errors in returns-based style analysis, compounding them.
Anyone paying for an actively managed investment product must have confidence that expected future active returns exceed fees. Returns-based style analysis and performance attribution are frequently used for this purpose. Can this analysis be trusted?
We estimated cumulative alpha, or residual return, for the Fairholme Fund with a single risk factor for the US Market using the returns-based exposure of 110% calculated above:
Unsurprisingly, errors in the beta estimate lead to a flawed picture of a fund’s security-selection performance. The returns-based approach estimates cumulative alpha greater than 10%. Our approach, aggregating the betas of the individual portfolio positions throughout the period, produces a negative value:
An investor who used returns-based style analysis and attribution would have estimated significant positive security-selection performance. In reality, an investor would have outperformed by taking the same market risks passively. A capable risk model specifically tuned for skill evaluation and performance prediction, such as the ABW Peer Analytics Statistical Equity Risk Model, averts this and similar pitfalls.
Conclusions
- Returns-based analysis can be effective—but only when a passive manager does not significantly vary exposures to market, sector, and macroeconomic factors.
- When an active manager varies bets, a returns-based analysis typically yields flawed estimates of portfolio risk.
- When a manager varies bets, a returns-based analysis may not even accurately estimate average portfolio risk.
- A returns-based analysis will be the least predictive for active managers. In fact, errors will be most pronounced for the most active funds:
- Estimates of a managers’ historical and current systematic risks may be flawed.
- Skilled funds may be deemed unskilled.
- Unskilled funds may be deemed skilled.
- An analysis and aggregation of factor exposures of individual holdings throughout portfolio history using a capable multi-factor risk model addresses these shortcomings.
More subtle, but no less dangerous, issues with investment risk and skill evaluation using returns-based performance attribution are discussed in subsequent articles.