October 5, 2022 – After a bit of a hiatus from the blog – thanks to our ambitious summer travel schedule – it’s time for another post. Over the years, I’ve gotten a lot of questions about the Shiller CAPE Ratio and if it’s still relevant. If you’re a regular reader of my blog, you’ll likely be familiar with the CAPE concept, but just as a refresher, Prof. Robert Shiller, economist and Nobel Laureate, came up with the cool idea of calculating a Price-Earnings (PE) ratio based not just on 1-year trailing earnings, which can be very volatile, but on a longer-term average to iron out the corporate earnings fluctuations over the business cycle. Hence the name Cyclically-Adjusted Price Earnings (CAPE) Ratio. If we use a 10-year moving average of inflation-adjusted earnings as the denominator in the PE ratio, we get a measure of market valuations that’s more informative in many instances. For example, historically, the CAPE ratio has been significantly negatively correlated with subsequent equity returns. It’s not useful for the very short-term equity outlook, but over longer horizons, say 10+ years, the CAPE ratio has been highly informative. Especially retirees should take notice because your retirement success hinges a lot on those first 10 or so retirement years due to Sequence of Return Risk. In fact, all failures of the 4% Rule occurred when the CAPE was above 20! A high initial CAPE ratio signals that retirees should probably be more cautious with their withdrawal rate!
But the CAPE has been elevated for such a long time, people wonder if this measure is still relevant. In the comments section, people ask me all the time what kind of adjustments I would perform to “fix” the CAPE. Can we make the Shiller CAPE more comparable over time, to account for different corporate tax environments and stock buybacks and/or dividend payout ratios over the decades? Yes, I will present my ideas here today. And even better, I will post regular updates (potentially daily!) in my Google Drive for everyone to access for free.
So, what do I find? The adjustments certainly lower the CAPE, but don’t get your hopes too high. Even after the adjustments, the CAPE is still a bit elevated today! Let’s take a look at the details…
Before we even get started with the tax and dividend payout ratios, here are two additional and crucial adjustments I always like to perform when calculating the Shiller CAPE. I might have mentioned these adjustments in a previous post, but never delved much into the details. But today is a good excuse to do so…
Fixing Shiller’s data lags
First, Shiller operates with woefully outdated data, especially earnings data. That may not be too much of a problem in typical macroeconomic research projects studying many decades worth of data. But the average retiree likes to have a timely and accurate estimate of the CAPE Ratio. Below is a screenshot from the Shiller Excel Sheet posted on his website. I downloaded this file on October 3, 2022. The first thing we notice is that the data are crazy outdated. The rows only go to July 2022 and the index level not even the month-end but July 5, so almost 3 months outdated. Of course, we can still derive a pretty decent estimate of the October 3 Shiller CAPE. For example, if we use an SPX quote of 3678.43 (=index level as of 10/3/2022, market close) and we rescale the 28.90 CAPE to 28.90/3831.39*3678.43 we get a CAPE of 27.75. But that’s not really precise because the July 2022 CAPE ratio uses the average real earnings over the 120 months of July 2012 to June 2022. But for the October 2022 CAPE, we have to use October 2012 to September 2022 average real earnings. We are using 3 months of data we shouldn’t use and are missing 3 months worth of data that we should be using.
But the data lag problem is even worse. Shiller’s earnings data go only up to March 2022. So, even back in July, he was 3 months short of earnings data. So, we are inadvertently underestimating the “E” part of the CAPE calculation because we’re putting a weight of 1/117 instead of 1/120 on the (significantly lower) earnings numbers from ten years ago. And using Shiller’s data for an October 2022 CAPE ratio estimate we’re now missing 6 months’ worth of earnings data. Excuse me for being pedantic, but that’s not acceptable.
So, how do I deal with the missing data in the Shiller Sheet? The first problem is easy to solve: I simply download the additional SPX index data for the other months. But what about the earnings data? The problem is that even as of October 3, 2022, the earnings for Q2 are not 100% finalized. According to the index provider, DJ SP Global, that Q2 number is “only” 99.8% final, so there are still 0.2% of the index members that haven’t finalized their number (link to their Excel Sheet).
Well, 99.8% is obviously good enough. And remember, the 99.8% refers to the current quarter. Shiller uses the 4-quarter moving average, so the estimate for the annual trailing earnings per share (EPS) is really 99.95% finalized as of 10/3/2022. So, I can certainly use the $192.26 estimate for June 2022. (also notice that, just as a quick check, the 197.91 and 197.87 figures for 2022Q1 and 2021Q4 exactly match the Shiller numbers, so he’s getting his numbers from the same source.)
And while we’re at it, I am also using the subsequent earnings forecasts for Q3 as the September EPS number in my table. And just like Shiller, I linearly interpolate the EPS for the months in between. And I concede that these are only forecasts, and sure, the forecasts may be off. But not using the SP Global estimates and effectively using the average over the 114 past months as an estimate for the six missing months is also a forecast and likely one with an even greater error!
In any case, I wrote a little Python program to perform the Shiller CAPE calculations, but instead of using the outdated Shiller EPS data, I access the SP Global data and fill in the missing earnings data and use the earnings estimates of the index provider instead. See the output below:
Month-end vs. monthly average index data
One peculiar feature of the data displayed above is that I get different SPX index readings. That’s because Prof. Shiller uses monthly average index levels, while I am using the month-end figure (or the latest available in the current month). In fact, using the month-end index levels is the second change I like to make in the CAPE calculation. Just to be sure, there is no correct or incorrect way. For example, as a macroeconomist, you might indeed be more interested in the average valuation of the S&P 500 during the month of September 2022. However, if you’re a retiree and pondering about what’s the right withdrawal rate today, then you’d be better served using today’s S&P value and not a 31-day moving average. That’s because you will sell your equities at today’s price not an average price over the last month. So, with those two adjustments, the table below is a sample of the most recent CAPE calculations.
Notice how the estimates can be significantly different. For example, in April, my CAPE (31.94) is almost 6% below the Shiller CAPE (33.89). That’s because the index finished significantly below the monthly average. Differences like that will certainly have a meaningful impact on your CAPE-based withdrawal rate. You don’t want to be sloppy about that. Precision is important in retirement planning!
And I want to make one thing really clear: I’m not trying to ding Prof. Shiller’s methodology. It may be correct in another context. But for our specific application, using the most recent index value and padding the EPS data with the currently available estimates makes the most sense.
Sorry about the deep dive into the CAPE methodology. For the three to four readers that haven’t fallen asleep yet, let’s talk about the issues mentioned in the intro now. One criticism of the Shiller methodology is that the CAPE has been elevated for such a long time. It seems that a CAPE ratio in the high-20s in today’s market seems a lot more sustainable than in the 1920s or 30s. What could be responsible for this shift? I found an interesting post by a guy called Damien Klassen on a financial planner website that proposes adjustments for both corporate taxes and share buybacks. I liked the way he models the corporate tax adjustments and I will use his methodology one-for-one. But I will use a slightly different approach for the buybacks, more on that below.
Accounting for the lower corporate tax rate
First, why do we want to adjust for corporate taxes? Imagine there’s a corporation that generated (CPI-adjusted) pre-tax earnings of exactly $100 a year for the last 10 years. During the last five years, corporate taxes were 21%, and thus the corporation earned $79 after-tax. The five years before that, corporate taxes were 35% and the company earned $65. Your average 10-year rolling earnings were $72. But $72 is a really poor estimate for the earnings trend. Your tax rate today isn’t 28% but 21%. Thus, the Klassen adjustment takes all past earnings and divides them by 1.0 minus the then-prevailing tax rate to get a sense of the pre-tax earnings. And then you apply the current corporate tax rate to all past earnings. Klassen thus calculates the 10-year after-tax rolling earnings per share (EPS) as if today’s corporate tax rates had been in effect for the entire past 10 years. The reason for this adjustment is that it would take an entire 10 years for a tax cut like the 2018 corporate tax reduction (from 35% to 21%) to work itself through Shiller’s CAPE calculations. The Klassen adjustment applies the tax impact instantaneously, exactly when the tax cut happens.
Thus, whenever corporate tax rates have decreased, our adjustment will increase the rolling average EPS used in the denominator and thus lower the CAPE, increasing the valuation-based attractiveness of stocks. And by the way, if the corporate tax rate were to rise again, and there are some rumblings in Washington D.C. right now, then this effect reverses and the adjusted Shiller CAPE will look unattractive again.
Accounting for share buybacks
The second adjustment has to do with share buybacks. Mr. Klassen doesn’t spend a lot of time explaining the rationale behind his adjustment. So here’s my take. Imagine we have two very similar corporations: Corporation A and Corporation B. They each have $1,000 of capital that generates $100 in CPI-adjusted profits every year for each corporation. They are also both valued at $1000. Imagine they each have 100 shares in circulation, valued at $10 a share.
Corporation A pays out all of its earnings as dividends, i.e., $1.00 per share. Corporation B retains all its earnings and simply buys back shares in the equity market. Thus, the investors in corporation B, in lieu of receiving a dividend, will see their share price increase by 10% every year. And if they want to get the same cash flow as the Corporation A shareholders they simply sell 10% of their shares every year. On average, people would have to do so anyway because of the buyback demand.
In a perfect world then, investors should be completely indifferent between the two corporations. You make 10% p.a. and people shouldn’t care if they get their 10% return through dividends or capital gains. Of course, in a slightly imperfect world, retail investors with taxable accounts might prefer corporation B because you can defer your capital gains taxes. But let’s abstract from that issue. A lot of retail money is in tax-deferred accounts and a lot of institutional investors (pension funds, endowments, sovereign wealth funds, etc.) don’t pay income taxes directly either; only the retirees that will eventually get the benefits will pay income taxes. In any case, we should agree that the equity return from investing in the two corporations will be identical in a perfect world and close to identical in the real world.
What would be the CAPE ratio of the two corporations? Corporation A had $1.00 earnings per share in each of the last 10 years. With a share price of $10.00, we get a CAPE of exactly 10. People might be tempted to argue that Corporation B has the same CAPE. But that’s incorrect! Recall that Corporation B has 100 shares now, but the share number has been declining by 10% every year. So, one year ago, we had 100/0.9=111.11 shares. Two years ago there are 100/0.9^2 shares=123.46 shares, and so on. Thus, earnings were only $0.39 a share at the beginning of the 10-year window and then grew by 10% each year to reach $1.00 in the current year, see the table below. Thus, the rolling 10-year average EPS in Corporation B was only about $0.65 vs. the $1.00 in the dividend-paying company.
This implies a CAPE of 15.35, more than 50% above the CAPE of the dividend payer. Thus, share buybacks would consistently hurt CAPE valuations relative to dividend payers. To adjust for this disadvantage, Klassen proposes to make an adjustment to the EPS to account for the changing number of shares. Specifically, he scales up the past EPS numbers to exactly undo the effect of the shrinking share base due to the buybacks. Basically, scale up corporation B’s EPS sequence [0.39,0.43,…,0.90,1.00] back to [1.00,…,1.00] to make it comparable to Corporation A.
Accounting for all retained earnings
I agree with Klassen’s general idea, but there are two problems with his approach: First, I don’t trust the share buyback data, certainly not the historical data. And second, not only buybacks, but any kind of reinvestment of retained earnings should trigger an adjustment in the CAPE. To illustrate this point, let me introduce Corporation C. It pays no dividend and it does not buy back shares either. Instead, it uses its corporate earnings to grow its productive capital. We could imagine that this corporation purchases more machines that will yield that same 10% return on equity. Or alternatively, this corporation might even purchase shares of the other two companies. In either case, if I assume again that Corporation C also has 100 shares outstanding and $10 in earnings in 2022, then the stats for Corporation C look as follows, see the table below. We get the same messed-up CAPE ratio, even though this company generates the exact same annual returns per share as the other two corporations:
Summary so far: an adjustment has to be made not just for buybacks but for all retained earnings, provided they are directed at investments that are at least as profitable as the return on equity (ROE) of the overall corporation. That certainly doesn’t have to be the case for all corporations. Some companies have a terrible record of investing – looking at you General Electric under Jeffrey Immelt. But for the economy as a whole, that should be a good assumption.
And you know what’s the advantage of taking into account all retained earnings? Data availability! Screw the whacky buyback data because we do have reliable earnings and dividends data. We can just calculate the reinvestment percentage equal to the earnings yield minus the dividend yield, create an EPS scaling factor equal to 1.0 in the current year and then compound the growth in that scaling factor from reinvested earnings going backward. If you’re interested, Klassen posts his Excel Sheet and he shows how he calculates this with the buyback percentages. I use the same methodology but I simply use the earnings minus dividend yield numbers instead of the buyback percentages.
So, with all those adjustments, what CAPE ratio do we get today, as of 10/3/2022, when I’m writing this? That’s in the table below. We go from a 26.59 CAPE without adjustments to 21.40. That’s a 20% drop in the CAPE Ratio. Quite meaningful. But keep in mind that even at that CAPE we’re significantly above historical average CAPE ratios (about 15). So, we’d still expect below-average returns going forward.
Here’s the same chart as in the blog post header again, the CAPE time series since 1925. The adjustments didn’t make a large difference in the 1920s and 1930s. From the early 40s to the mid-50s, the adjusted CAPE was even higher than the unadjusted one (due to a rise in corporate tax rates). Only in the mid-1980s did we really see the adjusted CAPE come down, as a result of both corporate tax rates as well as dividend payout ratios moving down.
And here’s a time series chart since 1970, when the adjustments are really the most noticeable. Instead of plotting the CAPE, though, I transform this into an earnings yield (one divided by the CAPE), so this would be a series with a positive correlation with future earnings.
There are meaningful differences. In the late-1980s the adjusted CAPE yield was about 2 percentage points higher. It didn’t make too much of a difference during the dot-com bubble and more recently we see an impact of about one percentage point. But of course, the CAPE adjustment can’t so easily explain away the crazy CAPE valuations we’ve had over the last two decades. Even with the adjustment, earnings yields are still low compared to historical ones. Indeed, looking at longer-term averages in the table below, we see that the average CAPE yield has been about 6.78% (roughly a CAPE of 15, the widely-cited historical average) over the last 100 or so years. The standard CAPE yield since 2000 has been only 3.87 (a CAPE average of about 26). Even with the adjustment, we can lift that most recent CAPE yield to only 4.55%. Still much lower than long-term averages. But keep in mind that a 0.67 percentage point (absolute) increase, is still a relative increase by more than one-sixth. For example, if someone wanted to tie his or her withdrawal rate to the CAPE, then the withdrawal rate may only go up by 0.67 percentage points, but the annual withdrawal amounts will rise by 17.37%. That can be tens of thousands of dollars annually!
Oh, and before I forget, I post my CAPE numbers, specifically, the entire time series since 1871, here on my Google Drive:
https://drive.google.com/file/d/1ugtRN3TaAVwQi-20mjt4DctF-glppSMD/view?usp=sharing
Please let me know if you have trouble accessing the file. As usual, you can view the file, but before you do edits, you’ll have to download it to your own computer and/or Google Drive. I will run this (almost) every weekday, so you should be able to get regular updates on the most recent CAPE estimates, both the standard CAPE.ERN.1 and the adjusted CAPE.ERN.2. And for fun, you can also monitor how hopelessly outdated the Shiller numbers are. 🙂
Update 1 (9am 10/5/2022)
Someone in the comments section pointed out that Frank Vasquez from the “Risk Parity Radio Podcast” recently had an episode criticizing valuation in general and the CAPE ratio in particular. If you don’t want to listen to the whole thing, the relevant part starts at the 25:00 mark. The “definitive proof” he puts forward: The CAPE was high in 2011 and subsequent returns were high as well. One single counter-example!? Well, that’s not really proving anything. Even if he had found one single counter-example it would only weaken, not eliminate the case for the CAPE. But Frank’s argument is even worse! In fact, his precise example works in favor of the CAPE ratio. The 2011 CAPE earnings yield (using my new adjusted series) hovered around 4.6% and subsequent returns were almost 14% (Dec 2011 to Dec 2021). But guess what? Compared to the valuations prevailing during that time, say starting in 2000, we get a beautiful positive correlation between valuation and subsequent returns. 2011 had much more attractive valuations than 2000 or 2007, right before the respective bear markets starts. And 2011 had much better subsequent 10-year returns than 2000 or 2007. Thanks, Frank, for providing more proof that valuations matter!
Update 2 (10/11/2022)
Someone in the comments section asked if there’s any evidence that a lower dividend payout ratio coincides with higher real EPS growth. Let’s plot the two series, see the chart below. The blue line is the log of the real EPS (normalized to 0 in 1871). Because of all the business cycle fluctuations, it’s not that easy to see, but you will notice that between 1871 and 1945 you had a pretty slow average growth rate (0.0->0.75) and a lot of the growth in real EPS came after that (0.75->3.0). In fact, it’s quite clear that a linear trend line will not fit this blue line very well. The fitted curve (dotted blue line) is a quadratic line, which beautifully creates an accelerating growth rate (=slope of the dotted blue trend line). So, in other words, the earnings trend growth rate accelerated when firms retained more earnings.
Conclusion
Time to wrap up since we’re already pushing past 3,000 words. To sum up, we can easily fix Shiller’s data reporting lags and we can certainly apply some adjustments to the Shiller CAPE. But the measure remains solidly above its long-term average, even after the major drop this year. What does this all mean for retirees? Initially, I had planned to make this part 54 of my Safe Withdrawal Rate Series with additional calculations on how the different CAPE Ratio scenarios impact your retirement safe withdrawal rates, but I will defer that to another post, hopefully in the next one or two weeks. Stay tuned!
Hi Karsten,
Two questions:
a) How much of the difference between traditional and adjusted CAPE is due to accounting for tax changes vs buybacks?
b) Not sure I understand your point about the impact of retained earning on CAPE. I thought retained earning is a part (use) of earnings. If a company decides to invest more or less of its earnings should not immediately impact CAPE. Am I wrong?
a) You see the total effect. I’ve never drilled down to see how much each one contributes in isolation.
b) retained earnings are part of earnings, true. That doesn’t cause the difference. The numerical example for Corporation C in the section “Accounting for all retained earnings” showcases how retained earnings tilt the Shiller CAPE, when compared to an otherwise identical firm A or B.
Thank you for your reply. I do not understand the example for Corporation C. To me that refers to a company with lower earnings than A and B, as such it gets a higher CAPE. I am not clear what retained earnings have to do with it given that they are account below the earnings line. If the earnings are low it must be for another reason. Let me know where I am wrong please. Thanks.
Corporations A, B and C all have the same earnings in the current year. The same P/E Ratio = 10 over the years. The same return every year (+10%). They are effectively identical corporations. But the CAPE ratios differ because of the way past earnings grew and because of the 10-year rolling average process in the CAPE ratio penalizes corporations with fast growing EPS, either through share buybacks (Corporation B) or retained earnings (Corporation C), while Corporation A with flat earnings has a lower CAPE. Even though they all offer the same profits per $ invested in every year.
Hi ERN,
Thanks for this, very interesting. Any thoughts on how I could apply this same principle to a global CAPE measure of the MSCI ACWI index? I’m using an ETF as a proxy to get price & earnings for each month, dividends or distributions are 6 monthly…perhaps not going to yield accurate results, I’m working on getting earnings for all the stocks in the index but it’s not easy (for me at least!). I’ve checked klassens spreadsheet but it’s unclear to me how to replace buybacks with dividends. Any thoughts much appreciated!
Data availability is the issue. You need 10 years of earnings and dividend data, plus estimates of the corporate rate (weighted by market share). That’s not easy to get.
You might be able to approximate it if you can find appropriate weightings for each country by using data from this source: https://indices.cib.barclays/IM/21/en/indices/static/historic-cape.app
Cheers.
Right. The weightings are the easy part. All the underlying corporate tax rates would be hard.
That’s certainly the problem I’ve been having! ChatGPT (I have no shame!) has helped me write a few python API scripts for scraping oecd data for tax and inflation by country, I’ve found I can get country weightings from ETF websites, and with a little digging down the MSCI has country weightings too but it’s quite a bit of work gathering the market cap info. Earnings data is the most flakey, I’m having to extrapolate it from P/E ratios either from the ETF’s or the MSCI ACWI factsheets.
Thanks for the suggestion, Dru, I’ve done just that as well and just like you said. Every global CAPE online seems to be quite different! My concern is being half way through retirement and having the CAPE measure I was using change, so I’d like to gather the data myself…if I can figure out a method I hope to future-proof it for myself, and if I can make it more accurate by incorporating tax and dividend yield then all the better.
No shame in using the tools out there. Good luck.
But again: it’s not just scraping current PE, but also historical. I do that efficiently with the SP500 data (https://www.spglobal.com/spdji/en/documents/additional-material/sp-500-eps-est.xlsx), spliced with the Shiller historical data. It’s harder with international data.
Hello ERN, thank you for you work. I was wondering, why are there 3 trend lines on the CAPE yield vs. S&P 10 year returns? Or is it just a false pattern?
The first is a line going from the bottom-left to the top-right; with a break between 5% and 7% CAEY.
The second is an elevated, truncated line from 4% to 7% CAEY.
The third is the high globule between 5% and 6% CAEY.
Is there a reason for these? Or just seeing things that aren’t there?
Which chart are you referring to?
Equity valuation (x-axis) vs. subsequent 10-year annualized return
(Update 1)
All I’m saying with that chart is that there’s a positive relationship between the two.
I wouldn’t read too much into three apparent separate lines (all about the same slope but different intercepts). The intercept would not be different with any statistical significance.
Thanks for this resource, and especially the ongoing updates to your modified CAPE estimate. I notice the CSV file hasn’t been updated since 31 July. (I don’t necessarily updated every day, but it has been an exciting couple days for CAPE. 🙂 ) Do you think you’ll be able to keep maintaining it, or would you be open to sharing your code or pseudo-code so that those of us who use this can roll our own?
Something must have been wrong with the Google Drive connection. The Pyhton code ran on all days but didn’t upload the results file. I fixed the problem.
In the future, if you see an outdated file: simply scale the CAPE to the with the formula:
CAPE_new = CAPE_outdated x SPX_current / SPX_outdated
CAPE values are not updating in CSV file or on toolbox. Thanks for all you do!
Should work now.
The CSV file seems stuck at 11 December at the moment, and the unadjusted SPX value (which I think is what the SPX.Shiller reflects?) doesn’t seem to have been updated in the CSV file since September so I’m not sure that formula still works?
Shiller has some crazy long reporting lags occasionally. That’s why I don’t use his numbers and recreate everything in house here.
My numbers are sometimes “outdated” when I’m traveling. This should update again soon. Until then you can figure the current CAPE values with this formula:
CAPE(now) = CAPE(ERN)*SPX(now)/SPX(ERN)
Hi Karsten,
Coming back to this great article, I always learn something new.
It appears that Vanguard in their fair value CAPE adjustment (https://corporate.vanguard.com/content/corporatesite/us/en/corp/articles/reasons-caution-about-us-equity-valuations.html) also consider the prevailing interest rates. This makes sense to me. Do you think your adjustment could benefit from such an additional tweak?
Your current adjusted CAPE is about 5 points below Shiller’s. But interest rates (10 year treasuries for example) have gone up a lot, raising the cost of capital for the foreseeable future. That should reduce future earnings somewhat. Therefore Vanguard is reducing a fair value cape quite a bit.
Thank you again for all your great work in the Goldgrube.
The CAPE is what it’s defined to be. End of story. Future bond yields don’t impact past earnings (which is what enters the CAPE). Past bond yields might have indirectly impacted the CAPE through earnings. But I keep the CAPE as is.
What you make out of the CAPE is a different story: Of course you want to compare the CAPE yield to expected returns of other asset classes. But I would never implement the bond yields into the basic CAPE calculations.
Summary: I keep the construction of valuation measures separate from how I want to later compare different asset classes.
Hi Karsten
I’m new to your website and am absolutely thrilled to have found such a valuable resource. I learned that Prof Shiller has also expanded his CAPE methodology to include compare earnings yield with real bond yield. It is called Excess CAPE Yield (ECY). That number is very low right now (as you can imagine) compared with the historical range but I wonder how it will look if you had perform the same adjustments above to it.
Thanks! I’m aware of Shiller’s new series. A relative (i.e., stock vs. bond) attractiveness measure might be helpful for tactical asset allocation decisions, i.e., timing stocks vs. bonds. But, I’m more interested in the absolute (not relative) stock valuation for the SWR analysis. So, I prefer the ERN-adjusted CAPE.
Besides, I wouldn’t use Shiller’s bond-adjusted CAPE, even for S/B allocation, but that’s a story for another day. 🙂
Hi Big Ern, love the CAPE-based spreadsheet for evaluating a dynamic spend on an as going basis and a bit of front end planning for hypothetical market downturn. But wondering about how to consider the CAPE.ERN.2 value into the context of evaluating the next 10 year equity horizon projection similar to what was done for the inferior original CAPE value as show in the 1st table (3rd image) in this link from Morningstar….https://www.morningstar.com/markets/improving-cape-10
The reason for the ask is I am trying to consider possible equity returns for say the next 10 years in say a long-term retirement plan.
I met Victor Haghani at the 2024 Bogleheads conference. I conformed with him that he incorporates something similar to the payout ratio adjustment I propose, so my CAPE.ERN.2 will generate very similar return forecasts as the P-CAPE.
I prefer my measure because I also factor in the corporate tax regimes and because I generate my estimates in-house and daily. I don’t want to wait for other people to update their data.
Hi Big ERN. I must be missing something and I am hoping you can clear it up for me. And I apologize if this has been addressed in a blog post before or after this one. I haven’t had time to read everything you’ve written yet.
The adjustment for share buybacks does not make sense to me. The S&P 500 index is market cap weighted, so the price of the index should be unaffected by the number of shares any company has outstanding at any given time, the change in the number of shares over time or the decision to return capital to shareholders through buybacks as opposed to through dividends. I had assumed, possibly incorrectly, that we would market cap weight earnings when calculating the CAPE for the index as a whole, which would mean that earnings would also be unaffected by buybacks. After all, owning one share of an S&P 500 index fund represents (in a rough, imprecise sort of way) a share of the market cap weighted profits of the 500 companies in the index. A CAPE that divides market weighted prices by price weighted earnings would make little sense.
So if we are adjusting for market capitalization both the numerator of the CAPE (the price) and the denominator (the earnings) then why do we need to make a further adjustment for buybacks?
Compare corporations A and B. The total market cap is the same every year. Total earnings are the same every year. The PE ratio is the same every year. I never claimed otherwise.
I merely point out that Corporations A and B have different CAPEs in the current year. The difference arises because now you are using earnings and prices from different years in the CAPE fraction. And that CAPE calculation is messed up as I display in the table:
Hi Karsten,
In the last few years there’s talk about a shift in the equity markets that imply that CAPE will remain historically elevated for many years to come and that a CAPE >30 doesn’t have to indicate equities being overbought.
A few arguments:
1) Historically low interest rate environment over the past couple of decades reducing discount rate (making equities more attractive relative to bonds)
2) More of the economy shifting to intangible asset driven companies; higher valuation multiples are justified
3) Higher floor for baseline demand for equities – driven by institutional participation (401Ks, etc) and rise of en masse retail investing over the past 25 years
4) Central bank interventions over time reducing perceived downside risks, encouraging investors to pay a premium for equities
The “This time it’s different” fallacy comes to mind, but I wonder what your thoughts are?
All these are great points!
1: Well, interest rates have risen again. That will be a speedbump for the stock market
2: Agree: some high-flying stocks looks overvalued but only because earnings and book values are not measured properly. I am hopeful that the market isn’t quite as badly overvalued as some folks claim.
3: Agree! Good point. There will always be a lot of money moving into the market. We will likely never have a total capitulation sale again like 1982.
4: Yes, agree. The Fed will not let the market unravel like 1929 again.
So, does this mean CAPE-based SWR calculations may be overly conservative as a result of above? Why or why not? Thanks!
Great SWR series. I keep discovering more depth as I delve into different parts in the series.
Hard to tell. In 1929 someone said the stock market was on a new plateau and we don’t have to worry about market crashes anymore. And then the crash happened.
Sure. No way to ever know the future.
What prompted my question is the claim that CAPE doesn’t exhibit mean reversion. If so how does it impact the predictive value of CAPE for 10 year future performance?
Whose claim of “no mean reversion” is that?
It’s true that the original Shiller CAPE is hard to compare historically because of the issues I raised here. My new version makes mean reversion a bit easier. But even with my version, it’s hard to come back to an average of 15 again.
Thanks Karsten. I was referring to a combination of your reply to MISAP above and Goldman Sachs claim, amplified by Sam Ro, back in 2022. Because on your above comments, I was wondering what you thought of that claim (applied to your modified vs original CAPE), and if you thought it had merit, what implications it had on the 10 year predictive ability of CAPE, since you use it for SWR analysis. Thanks.
The S&P displays mean reversion and the CAPE does, too. Though, there is clearly a time-varying mean. And we don’t know that mean in real time. Not sure if that contradicts or confirms what GS says.
Thanks Karsten.
So I interpret you to be saying that with a rising mean reversion, CAPE (at least your modified CAPE) is still sufficiently predictive to be effective in your CAPE-based SWR model, right?
And maybe also that your conclusion is at least similar to, if not identical to, https://www.advisorpedia.com/strategists/goldman-sachs-destroys-an-investing-myth/, right?
What I have seen about GS claim was only that they said they didn’t see evidence of mean reversion to a *fixed* mean – at least that is the claim being widely quoted.
They say there is no fixed mean. I say there is a time-varying mean. Both statements are saying the same.
Thanks. I understand both can be / are true at once for CAPE (time-varying mean with no fixed mean).
Given that, I’m simply asking that with a time-varying (ERN-modified) CAPE mean, is the modified CAPE still sufficiently predictive of 10 year future stock returns to be effective in your CAPE-based SWR model? I believe your answer is yes, but just asking for confirmation.
I think your conclusion is that it is still sufficiently predictive / highly correlated based on the long backtest data analysis you have performed, even in light of the rising mean.
Just asking if I understand your conclusion correctly. Thanks! Your SWR series/toolkit (which I use) is impressive and I just want to make sure I understand what if any implications time-varying rather than fixed mean reversion has on it.
When you look at a scatter plot you assume a fixed mean to revert to. That simple scatter plot for 100+ years of data shows a strong relationship between the CAPE/CAEY and the subsequent return.
If you assume a time-varying mean you’d get an even better fit, albeit to a mean that’s harder to pin down in real time.
So whether fixed or time-varying mean: there is a correlation, it’s substantial and it’s significant.
So, nothing has changed: CAPE/CAEY are essential in financial planning.
Thanks! That was just the clarification / confirmation I was looking for.
Hey Karsten, given that you regularly update the google sheet with the adjusted P/E, it’d be a nice feature for the latest numbers to appear on top of the sheet (either the list fully in reverse chronological order, or just a box showing today’s CAPE).
Thanks for the great info and writeup!
I update the sheet regularly. If people need the data, they should be able to scroll down. I’m used to chronological order, not the reverse. 🙂
You can also look up the recent data in the SWR Google Sheet. Tab=”CAPE-based Rule” Cell D8 and D9.
Love your work, thank you so much for sharing your wealth of knowledge! I keep hearing that the CAPE ratio is elevated from a historical perspective. I would be curious for your view. I believe even your adjusted data suggested the average over the last several decades may even be in line with where we are currently (on an adjusted basis)?
According to my post here, even with the CPA adjustments, the new CAPE is certainly elevated. True, today’s CAPE is closer to the average over the last few decades, But that’s what Irving Fisher said in 1929, too. Right before the big crash.
Hey Big ERN, was wondering if you could help me understand how to recreate your better CAPE numbers in that CSV file you linked above. I made a copy of the CSV file and then joined in Klassen’s corporate tax data.
What I’m doing is something like this, using your ERN SPX, ERN CPI, and ERN EPS numbers. For each row from Jan 1871 to present:
– calculate nominal pre-tax earnings by dividing EPS by 1-corptax%
– calculate real pre-tax earnings by adjusting the above to today’s $ using CPI ratios
– calculate real price by adjusting the SPX to today’s $ using CPI ratios
– calculate earnings yield and dividend yield by dividing EPS by SPX and dividends by SPX (I used real numbers but nominal is the same, as these are ratios)
– subtract dividend yield from earnings yield to get retained earnings yield (REY)
– set a scaling factor equal to 1 for the present month, and for each row above it take the scaling factor below it and divide by 1-REY/12 for that month, which continuously scales the numbers upward to Jan 1871 (the numbers are big, like 10k+)
– find 10y average earnings by taking the SUMPRODUCT of the pre-tax EPS with the scaling factors for the previous 120 months, then dividing by 120, dividing by the scaling factor for that row’s month, and multiplying by 1-corptax% of that row — this matches Klassen AFAICT
– finally, divide real price by real 10y average earnings to get a CAPE ratio
However, my final results seem to consistently underestimate the CAPE compared to your generated numbers; there wasn’t a single month when it matched exactly. Is there something I’m missing? Any advice would be really appreciated! Thanks so much 😀
P.S. I’m also curious how you calculated the CAPE numbers for the data points within the first 10 years, since there isn’t anything to smooth over yet until Jan 1881(?)
I wrote my code in Python. I checked to make sure than my numbers coincide with Shiller’s numbers. I then replicated Klassen’s numbers. Can you match Shiller’s numbers? Can you match Klassen’s methodology?
Apart from that, I have no appetite to check other folks’ calculations for bugs. Sorry, buddy, you’re on your own on that one. 🙂
The first 10 years: use expanding averages until you reach 120 months of data.
Hi Big ERN,
No worries. Thanks for the ideas! I’ll try the replication strategy first to make sure I’m on the right path and then see if I can get your numbers as well. Appreciate it.
The idea for first 10 years makes sense too.
Best wishes,
THY
Hi Big ERN,
After a few days, I was able to reproduce Shiller’s data and Klassen’s data. I was curious if you just directly replaced the buyback percentage with the EY-DY percentage. I tried various approaches but I still can’t seem to reproduce your numbers.
The other thing I noticed is that Klassen actually defines a “forward index” — he sets the scaling factor equal to 1 (actually 100, but it doesn’t matter as it’s relative) and then scales forward to the current month, which has a much smaller factor. I see that you’re doing the opposite via the “compound backward” idea you talk about. The effect is similar, however: the older months need to get magnified more.
Can you shed some light on the differences or potentially explain what you did differently from Klassen? I basically reproduced his sheet exactly, then made a clone of it and changed out the buyback percentage for the EY-DY percentage as you mentioned in your post. But the numbers still seem to differ from yours. I also tried all sorts of crazy ideas; Klassen divides the annualized buyback percentages by 12 to get the monthly percentages, but I assume the EY-DY percentages are already monthly? Additionally, he multiplies the buyback percentage by DPS/price. It just seems strange to me, like from a dimensional analysis standpoint.
Thanks for any insights that you may be able to pass along!
The EY – DY is annual. It needs to be divided by 12 to get the monthly rate. Then, construct the share index with those monthly changes.
FYI, the number of shares moves from 100.0 in 1871 to approximately 37.9 in June 2025, to provide you a ballpark estimate for the time series of outstanding shares after accounting for the imputed buybacks.
It has struck me that neither the CAPE nor the 1-year trailing P/E ratio has been significantly below its long-term average for nearly 40 years as presented on the multpl.com website. And both ratios have, of course, been well above their long-term averages for a great deal of that period.
I can see how changes in corporate tax rates, increased indulgence in buybacks and retained earnings reinvestment have affected both measures, but can’t shake the feeling that things happened at around that point in time which shifted the whole valuation situation.
Could that have something to do with the increase in the availability of inexpensive index funds and increasing adoption of IRA’s and 401K plans, both of which in effect “broadened the market” for stocks in general? Current valuations are, of course, elevated even given this new apparent plateau.
True. And that may mean that the “target CAPE” is now higher because we’re a more mature economy in 2025 than in 1925. We also have lower risk-free interest rates now, which should take down the entire expected return world.
But careful what you wish for: the risk for an impending blowup may be lower. But the expected returns for equities may also be lower. So, instead of 6.7% real return over the last 150 years we may now only expect 5% real return in equities. That would lower your sustainable withdrawal rate.
Python be broke?
I’m on vacation. Until I get back to running my daily Python code routine again, please use the proportional adjustment:
CAPE(today) = CAPE(outdated)*SPX(today)/SPX(outdated)
Ah enjoy! I misinterpreted that the python code as being automated.
I access your CAPE file monthly. It is very important to me.
Today it said “CAPEoutput4blog.csv File is in the owner’s trash. No preview available. Download”.
I was able to download the file. This is just a heads up in case something is amiss. This is the link I follow: https://drive.google.com/file/d/1ugtRN3TaAVwQi-20mjt4DctF-glppSMD/view?usp=sharing
I ‘quit’ in 2018 and use your file to guide my SWR. Invaluable!
Just to add a +1. The link doesn’t work for me either, and I get the same error message that it is in the trash. Would love to have access to the file to see the latest Cape ratio, fixed for the various items noted. Thanks!
In the CAPE tab, Cell X1, please replace that formla with “https://drive.google.com/uc?export=download&id=13Rp-khxsUXyFbyJVii4IpSu1mQhrSVcE”
I updated the file. Please go to the most recent SWR file and check if you have the new version.
But I also restored the old file. But that one is no longer updated.
Whew!!!!! I’ve been wondering what happened, so glad I checked these comments! I check this spreadsheet every single day the market is open, it is absolutely invaluable for my safe withdrawal calculations (especially Column K, “CAPE.ERN.2”). Thank you very much Karsten.
Out of curiosity, what prompted the file update? Something in your python script?
It also looks like in your post above (“Oh, and before I forget, I post my CAPE numbers, specifically, the entire time series since 1871, here on my Google Drive:”) you still have the old link: https://drive.google.com/file/d/1ugtRN3TaAVwQi-20mjt4DctF-glppSMD/view.
Finally, in the category of “it can’t hurt to ask”, have you ever considered providing the python script to readers? I would love to see how you’re doing that calculation in python. I understand if you’re not comfortable doing that though.
Thanks man!
I used to add a last row into the CSV file with the link to the CAPE blog post. But that’s silly. So, I now use the file with the data only. I had two files parallel for a while. But now took down the old one.
Also important: Using the new file also requires a different formula in the Google Sheet (because the new CSV has one fewer line).
The Python script is proprietary right now.
Hi Karsten,
Happy New Year, and thank you for the work you so generously do for the community. While updating my 30-year budget spreadsheet using the “Big ERN’s Improved CAPE Ratio” sheet, I noticed that the link you shared above no longer works.
In the CAPE tab, Cell X1, please replace that formla with “https://drive.google.com/uc?export=download&id=13Rp-khxsUXyFbyJVii4IpSu1mQhrSVcE”