Those magic formula loose ends...

The spam filter installed on this site is currently unavailable. Per site policy, we are unable to accept new submissions until that problem is resolved. Please try resubmitting the form in a couple of minutes.
Share this

Too good to be true? The loose end that bothers me most about last week’s magic formula testing is the incredible success of the formula in the first three years of the test. It’s visible to the naked eye: The 12 portfolios formed between 31 March 2000 and 31 December 2002 beat the FTSE by an average of about 23%. It wasn’t just the margin by which the formula beat the market in those years that surprised me, it was the fact that the formula beat the market at all considering between 2000 and 2002 the market was crashing. In his book, The Little Book That Still Beats The Market, Joel Greenblatt says: turns out that much of the outperformance of our portfolios comes during the up months. On average during this 22-year period, the magic formula portfolios "captured" 95% of the S&P 500's performance during down months and 140% of its performance during up months. In other words, in his experience the formula performs badly in bear markets. In mine it did particularly well in one bear market. I emailed Steve Lewis, the architect of Sharelockholmes, to ask him whether he could think of any reason why the data might be less trustworthy in the earlier period. He replied with three possibilities, all stemming from the fact that the data in Sharelockholmes comes straight from company results and is not adjusted for:

  1. Changes to the accounting of pension deficits, which would affect the value used in the ROA calculation.
  2. The introduction of IFRS in 2005, which replaced the old GAAP accounting standards.
  3. Delisted companies before 2003. Steve set up Sharelockholmes in 2003, using data going back to 1999 for companies still active in 2003. So Sharelockholmes doesn’t contain records for companies that delisted between 2000 and 2003, although it does thereafter.

Don’t be critical Sharelockholmes, it’s the only affordable database I know that has the data I need, albeit unadjusted. Instinct tells me that none of these factors should radically influence the results. Pension accounting wouldn’t affect the ROC calculation yet it exhibits a very similar pattern to the ROA version of the magic formula, and delistings in the 2003-2010 data didn’t affect the average performance of the magic formula strategies. But instinct is basically worthless, and reconstructing the database to include companies that delisted is beyond me. Maybe we should ignore the data from 2000-2003. This would knock out most of the ‘dirty data’ at the expense of reducing the total number of portfolios from 40 to 28, reducing the time period of the test from eleven years to eight, and perhaps exclude a period of genuinely outstanding performance for the magic formula. In the most recent eight years of the test, the marginally superior ROA version of the magic formula beat the FTSE 100 by about 8% a year as opposed to 13% for the whole period of the test, and the ROC version 7% as opposed to 12%. Which is closer to the truth? I honestly don’t know, but either way, the magic formula has worked in the UK. That’s good news.


i have three observations:

i) The monster outperformance between 2000 and '03 could be due to a 'value effect' - Buffett as well as lots of other value investors did do well in this period also.

ii) There is a potential problem with data availablility. If you are basing your backtest on data or finacials which wasn't available at the time of historic portfolio construction, then this could be distorting results in your favour. For example, your backtest might have told you to buy company x in Jan '04 based on its ROC / ROA in the period to Dec '03. The trouble is that company x might not have reported their financials to Dec '03 until the end of Feb '04. From memory Greenblatt bought access to an expensive database which controlled this effect when he was doing his backtesting.

iii) Anecdotally, I do think that this approach adds a lot of value though. I have been running a UK oriented fund this way for the last few months (with a qualitative overlay to avoid HMV-type dogs) and it is early days but performance seems to be strong.

In any case - thanks very much for doing the analysis - it is interesting. I'd be grateful for a copy of the spreadsheet if you are making it available.


Thanks for your comment Will. I've sent you the spreadsheet.

I agree with your first point, but then why didn't Greenblatt find the same thing using his formula in the US?

Regarding your second point I think the data in Sharelockholmes is from the preliminary results and applies only after the date of the announcement, but I'll check on that. There is another way in which Greenblatt's database is more accurate though. I believe it's updated every quarter. Some of the earnings related data in Sharelockholmes could be up to a year old as no attempt is made to calculate trailing twelve month performance by combining the earnings figures from multiple financial statements. Whether this would boost results, or undermine them I cannot say, but it would result in different groups of companies being selected.

In the screens I use for the Thrifty 30 portfolio I tend to get around this problem by only screening for companies that have reported in the last four months.

I'd be interested in what your 'qualitative overlay' is. I run a portfolio for Money Observer magazine that uses the F_Score. Performance in its first nine months has not been great. That, of course, could just be bad luck. One of the holdings is Yell!

Interesting article. I tend to agree with Will - sometimes performance can be linked with style ... defensives tend to work better in down markets, and so on.

I think the last decade has been kind to value investors. It may well be relatively kind for the remainder of the decade.

Just thought I'd do a little followup on EAGA, the green energy company. It was a company that I noticed crop up quite high as a Magic Formula stock. I expressed grave reservations about it at the time, as I thought it to be a bit of a fad stock that was too dependent on government generosity. Recently, it was bought out by Carillion for 120p per share. If you had bought at the lowest price in the last yeat, 51p, you would of course be very happy with the result. If you had bought at the high end, 148p, you wouldn't be feeling so lucky. Was I wrong to shun the company? I don't think so (although feel free to argue). There was no guarantee that the company would be bought out, and I feel that the nature of the business made it an extremely risky share.

I've also highlighted such shares as YELL, HMV, HLO and RCG (if I remember rightly) that seemed to come out quite well in the SH (Sharelock Holmes) Greenblatt rankings, but would have been complete investing disasters. You'd need substantial performance of other shares in order make ammends for these horrendous dogs.

I'd like to mention a couple of other companies that have turned up on my Greenblatt screen, but which I think people should avoid. The first is BA. (Brit Aerospace), and the second is DEB (Debenhams). I used to hold shares in BA., but sold in disgust at the high debts and their continual expensing of exceptional charges. DEB, IMO, also has too much debt. It has net profits for 2010 of 97m, and net debt of over 1bn. I think that's quite a high debt load, making the shares much more risky.

Perhaps I should set up an Unmagic Formula board, in which I try to big dubious-looking MFI stocks, with the aim of underperforming the market.

Hi Blippy, thanks for your comment and for sharing your mf horror stocks! I'll add them to the ones mentioned in our earlier thread:

Taking up your strategic ignorance comment from before you really do have to concentrate on the system and not the companies or I think you'd talk youself out of many of them (of course some people try and pick and choose mf companies and that may be a viable strategy - it's definitely one I'm tempted to try).

Your idea of an unmagic formula is quite superb. If the market were perfectly efficient it ought to be impossible to deliberately underperform the index. I wonder if anyone's tried !?

Hi Richard,

Good work you have done here. It does concern me the impact of comnpanies no longer listed. where did you find the historic stock prices for such companies? I assume sharelockholmes will only show you the stock price at time of annual report publication - which doesn't help to tweak - i.e., I would like to look at Greenblatt with the intention of getting out of the investments when they are no longer considered value (as opposed to end of year).

Hi John, thanks for your comment. Prices on Sharelockholmes are updated daily so the current statistics are as of the previous close.

I'm not sure if you can get at historic prices in Sharelockholmes. I used its built-in back testing facility which calculates the % return relative to the FTSE 100 1 year, 2 years etc. after particular (quarterly) start dates. In practice that meant creating four portfolios a year (in March, June, September and December) and extracting the returns for each company in each portfolio. I outlined what I was planning to do (and subsequently did) in this post:

Good luck :-)

[...] believe it will weed out some of the weaker companies. After a fair amount of soul searching (here, here, and here) I’ve decided to replace Return on Capital with Return on Assets [...]

Dear Richard,
I also have been using Sharelockholmes for a number of years, trying to create my own magic formula. I am filled with admiration, because you have achieved more in 2 weeks than I have in 5 years.
The approach I have tried most recently is a weighting system (weighting seems less arbitrary than screening). I have taken the 500 largest companies on Sharelockholmes and ranked them on a series of criteria (dividend, earnings, free cash flow, earnings 5-year growth, tangible book value, tangible gearing). For each criterion I have given the top 10% a weighting of 1.0, the bottom 50% a weighting of 0, and the remainder are on a sliding scale from 1 to 0. The average of all these weightings is between about 0.75 and 0. As an exercise, I have looked at the data since 2000 for 12-month returns, March-March. I simply ignored the companies that disappeared (this is what has stopped me ever doing a rigorous analysis, it is too much work to chase them all up. However, when I have tried to look, nearly all are taken private or taken over at a premium. Very few go bust.)
I have looked at three fantasy portfolios: (a) unweighted, holding equal quantities of all 500 companies = picking shares with a pin. (b) value-weighted, using my home-brewed value weighting, and (c) size-weighted = conventional tracker.
Average starting values and average changes 2000-2011 per annum
Init yld In TBV Inc Mktcp Yld TBV
a) unwt 2.87 32.3 3.47% 4.69% 2.34%
b) value 3.62 43.1 4.50% 1.67% 0.88%
c) index 3.24 25.4 0.86% 4.25% 4.44%
The value-weighted did indeed outperform though, just as you found, the main outperformance was between 2000 and 2003. Interestingly, picking shares with a pin on average would also have outperformed a tracker by nearly 3%pa. Not surprising since this is effectively a small cap bias and small cap has outperformed massively over the decade. Again not surprising, a value portfolio has higher starting yield and tangible book value.
However, I believe that it is essential not just to look at increases in market capitalisation, but to look at other measures of size – dividends, earnings, book value etc. When I do this it confirms a paradox that I have found before. If you select for a measure of value, such as yield, you may well get a better market cap return than from a tracker, but the measure of value itself will do worse. In order to get the effect you want, you must select for something else.
This confirms what others have said, that the main benefit from value investing comes from mean reversion, rather than the identification of "good" companies. In the long run, the market price of a company can't increase faster than its "value". I still am no nearer the holy grail of identifying companies that will grow faster than a tracker, but are available at a reasonable price.

Hi Westwinds3, thanks for your comments, though all I've done is provided more evidence that *somebody eles's* formula works (i.e. Joel Greenblatt's magic formula)!

Regarding your results, concentrating on the largest 500 companies makes things more difficult I suppose as they are the best researched and most widely followed so least likely to be undervalued.

I think they're interesting though. The companies that grew their market capitalisations (i.e. share prices) the most were the companies that grew their accounting value the least, which seems to confirm what academics have found - that low asset growth companies do best, maybe because also because they don't raise lots of funds and blow it on unprofitable projects. As you say, it confirms reversion to the mean is a powerful factor in the returns of value investors.

I'm not sure what you mean by "companies that will grow faster than a tracker" - it seems in terms of share price the value portfolio does? What results would you get if you picked the twenty cheapest companies from your value weighted universe every year?

Post new comment

The content of this field is kept private and will not be shown publicly.