“Lies, damned lies and statistics” (Mark Twain)
“Do not trust any statistic you did not fake yourself” (Winston Churchill)
There is a classic book called “How to Lie with Statistics” that I read many, many years ago (actually decades ago!) as a college student. If you’re ever looking for an inexpensive but fun and impactful present for a young student/graduate with the hidden agenda of getting that person interested in math and statistics, this is the one! The book taught me to take with a grain of salt pretty much anything and everything number-related. Anywhere! Whether it’s in the news or in the Personal Finance blogging world and even (particularly?!) in academia. I’m not sure if I was already a severely suspicious (paranoid?) person before reading this or the book turned me into the person I’m today. So, inspired by that book, I thought it would be a nice idea to write a blog post about the different ways numbers are misrepresented in the FIRE/Personal Finance arena. And just to be sure, this post is not to be understood as a manual for fudging numbers, but – in the spirit of the “How to Lie With Statistics” classic – serves as a manual on how to spot the personal finance “lies” out there!
And there’s a lot of material! Probably enough for at least one more followup post, so for today’s post, I look at just four different way of how quantitative financial issues are frequently fudged in the personal finance world. And a side note about the slightly attention-grabbing title I used here: Well, I put the word “Lie” in quotation marks to show to the faint-hearted that this is a bit tongue-in-cheek. I could have written, “fudge the numbers” or “Enron-accounting” or “How we delude ourselves in personal finance,” or something like that. Also, Hanlon’s Razor (“don’t attribute to malice what can be explained by incompetence”) comes to mind here, but I’m not sure if those faint-hearted folks feel that incompetence is a significantly more benign explanation than malice.
So, let’s look at some of my favorite examples of how people lie to themselves (and others) in the realm of personal finance…
1: Using arithmetic, non-compounded average returns
If I wanted to lie to myself and others (or at least delude myself and others) and inflate my equity market return expectations, here’s a great way to fudge the numbers: Instead of using compounded average returns, simply use the arithmetic average returns over time. Huh? How can that make a difference? Aren’t the two the same? No, and here’s a simple example: If an asset returns -10%, +10%, +30%, respectively over three consecutive years then the portfolio evolves from $100 to $90, then $99, then $128.70. That’s a roughly +8.8% annualized return. The arithmetic average of the three return numbers is indeed (-10+10+30)/3=+10%. But where did we lose that one percent? Well, it’s apparent in the time series of portfolio values over time. After a +10% and -10% return we’re left with only $99! We never made it back to the initial $100 even though the +/-10% returns in the first two years average out to zero in the arithmetic sense! To make it back to $100 would have required a +11.11% return! That loss of one percent return after year 2 plus the fact that a 30% return in the third year is also slightly below 10% annualized, about 9.1% annualized to be precise, explains why the compounded returns lag behind the average returns.
How much of a difference does this make when we look at actual equity market returns? Quite a bit. Let’s take a look at the S&P 500 (U.S. large-cap stocks) since 1923 (apparently the start date used by Dave Ramsey). (side note for the history buffs: prior to 1957 this was the “Composite Index” and only in 1957 it became the S&P 500). In the table below are the results. Just for completeness, I do this in the following different ways:
- The CAGR (the correct way): 10.22%
- I first calculate the arithmetic average of the monthly returns (0.9596%) and annualize this by multiplying it by 12. 11.51%
- Take the monthly average but annualize it by compounding it to the annual number: 12.14%
- Take the arithmetic average of the calendar year returns: 12.11%
So, the arithmetic averages in 3 and 4 are roughly two percentage points above the CAGR!
And it’s quite common to mess up the average return calculations; Dave Ramsey is touting that 12% number very aggressively. And he even admits that his 12% figure refers to the arithmetic average. He doesn’t seem to comprehend the difference and he’s actually quite vocal and belligerent:
“…you can discuss all these freakin’ mathematical theories that you – some of you — financial nerds just sit around and crunch your numbers all the time and you do nothing to help people” Dave Ramsey, via Youtube (clip starts at the 2:44 mark)
Well, unfortunately, it’s not a mathematical theory. It’s just plain arithmetic. Dave Ramsey seems to even excuse fudging the numbers to “help people” – I guess – by getting them more excited about investing. I find this ethically troubling! I’ve also seen this 12% nonsense figure slowly making its way into the FIRE community. But a 12% return expectation is way above the historical norm. Be suspicious if anyone proposes a double-digit equity expected return!
Update 5/31/2019: Since this was requested by several people, below is the link to Ramsey’s page with the 1923 starting point. 12%+ returns over such a long period are only possible using arithmetic averages:
“The current average annual return from 1923 (the year of the S&P’s inception) through 2016 is 12.25%” Source: Dave Ramsey
And if you go to the footnote 2 provided on Dave Ramsey’s page you land on the MoneyChimp page, where the “Average” return was indeed 12.25%. But even on this page, they put the word “Average” in quotation marks. The “True CAGR” was indeed two percentage points lower. So, Dave Ramsey is using a monkey business number from a site named MoneyChimp. You can’t make that up! 🙂
How to spot this “lie” in the field: Always make sure that return numbers are CAGR. Sometimes that CAGR acronym is quoted explicitly. The phrases “compound,” “compounded” or “geometric average” would also be indicators that the calculation is done the right way. If no additional clarification is given, you might want to get your own calculator/spreadsheet out and confirm.
How I deal with this issue here on the ERN blog: I actually don’t even mention CAGR explicitly very often but rest assured that everything is A-OK here! 🙂
2: Ignoring inflation (i.e., confounding nominal and real returns)
OK, so let’s assume we’re smart and careful enough not to mess up the average return calculation. 10% equity returns, that’s still quite impressive! But unfortunately, that’s still a bit too high because that was the nominal return. If we’d had 10% inflation over that time span then a 10% nominal return doesn’t look so hot anymore. Of course, in most Western economies we’re talking about 1.5% or 2% CPI inflation per year today. What difference does that make if we ignore inflation? Admittedly, a negligible difference in any given year but it adds up over time. Even 2% inflation will erode your purchasing power over time. $100 today will become only $55 after 30 years and only a bit more than $30 after 60 years. Ignore that in your FIRE calculations and your fat-FIRE budget might slowly turn into lean-FIRE!
And this slow erosion of purchasing power is what makes inflation so dangerous. Not doing any inflation adjustment to your retirement budget in any given year seems innocuous. But over time you’d accumulate a dangerous cut in purchasing power. My blogging buddy Fritz called inflation the Silent Killer of Retirement, quite appropriately if you look at the chart of inflation erosion in the chart above! So, in any case, if we take out inflation from the S&P500/Composite return we’re now down to 7.15% for average returns 1923-2018. I still find this really impressive. Using the Rule of 72, that means your money doubles every ten years!
Of course, bungling the nominal vs. real returns isn’t always that obvious. Sometimes the nominal vs. real lie is very elegantly hidden. Here are some examples:
Example 1: “The 1980 and 1982 recessions weren’t all that bad for the stock market”
For someone like me, who has a morbid fascination with all the things that can go wrong in (personal) finance, the 1970s and early 80s are really intriguing. Funny thing is, if you look at the nominal equity performance between, say 1972 and 1982, the S&P 500 performance (total returns = dividends reinvested) doesn’t look so bad. Sure, there was a 43% drop in 1973/4, but a swift recovery followed and the two recessions in 1980 and 1981/2 saw only a very shallow drop in the stock market. Seems like a pretty benign period, right? Wrong! First, the drop in 1973/4 was over 50% when adjusted for inflation, roughly in line with the dot-com bust and the 2008/9 meltdown. Then the peak-to-bottom drop in the early 80s was 27%, which is a bit higher than the 17% drop in nominal terms. But even this 27% drop disguises the fact that the bottom in 1982 was still a whopping 40% below the CPI-adjusted 1972 peak.
By the way, adding to the pain during this period was the fact that bonds didn’t offer any diversification benefit because yields went up and caused both stocks and bonds to lose exactly at the same time. So, from a personal finance point of view, the 1970s/80s were a total unmitigated disaster, by some measures even worse than the Great Depression! But believe it or not, I’ve seen material from otherwise trustworthy bloggers who make the indefensible case that recessions aren’t really that bad for the stock market, pointing to the experience during the 70s and 80s. A little bit of soul dies every time I read that!
Example 2: “The year 2000 retirement cohort would have done just fine with a 4% withdrawal rate because they’d recovered their initial portfolio value by now”
This claim is often based on Michael Kitces’ case study of the 2000 retirement cohort. Kitces very accurately points out that an imaginary retirement portfolio would have recovered its $1,000,000 value by 2015 even with annual $40,000 withdrawals, adjusted for CPI inflation. But make sure you read the entire post, otherwise, you’ll miss this important piece of information:
“Of course, an important caveat to the chart above is that it’s based on ‘nominal’ dollars, not adjusted for inflation. Which is important, because it means that retirees who had similar portfolio balances after the first half of retirement were not necessarily going to have the same buying power with those dollars for the rest of retirement” Kitces.com, accessed May 18, 2019.
Adjusting for inflation, your portfolio would still be significantly under its starting value, just around $674,000 as of 4/30/2019. If you keep withdrawing $40,000 a year out of that portfolio you’re now looking at a 5.9% effective withdrawal rate. Probably fine if your horizon ends in the year 2030, but likely a reason for concern if you had been an early retiree in 2000 and you’re looking at another 30+ years going forward from here!
Well, at least you can’t blame Michael Kitces. He clearly states that the portfolio value was in nominal dollars, visible for anyone who wants to see it. The “lie” is mostly a result of confirmation bias running amok, i.e., the 4% Rule cheerleaders in the FIRE community ignoring the disclaimer about nominal returns and touting this as a great success of the 4% Rule even for early retirees.
But I’ve also seen some bloggers obfuscating this nominal vs. real issue a lot more blatantly. For example in this blog post on Ben Carlson’s blog “A Wealth of Coming Sense” (which I generally like, just not this one blog post), there are similar calculations as in the Kitces case study – notice the $1m+ portfolio value for the year 2000 retirement cohort – but now there is no mention at all of the fact that the final portfolio values being reported in nominal terms only! In fact, phrases like this one:
“but inflation did take a monster bite out of the ending balance in the 1973 start date”
…suggest to the unsuspecting reader, that all numbers were CPI-adjusted: withdrawals and portfolio values. But again, only the withdrawals, not the portfolio values are adjusted for inflation! How sneaky! And a side note: there are at least two additional data fudging problems in Ben’s calculations:
- He starts the 1929 retirement cohort in January rather than in September – the true market peak and over 30% above the beginning of the year level. This greatly obscures the true danger of “retiring at the market peak!”
- He displays only the 1973 cohort but ignores the 1966 market peak where retirees actually would have run out of money after 30 years due to the horrendous 1970s and early 80s recessions and bear markets!
So, to sum up, if you were to do this exercise the right way, then the portfolio values after 30 years look a lot less appealing. Instead of preserving or even growing your capital after 30 years it looks more likely you may run out of money (1966) or almost run out of money (1929, 1973) to the point where your portfolio won’t last for the fourth decade. Which would be a bit of a problem for early retirees!
Example 3: “[After 30 years,] the [4%] safe withdrawal rate actually has a 96% probability of leaving more than all of your original starting principle [sic].” Source: Mad Fientist podcast with Michael Kitces
Again, the final portfolio values are not adjusted for inflation. This gives the false impression to a gullible FIRE fan that since you preserve your capital for 30 years with such a large probability, you can easily tag on another 20 or even 30 years and you can thus “extrapolate” the 4% safe rate from 30 years to a FIRE-style retirement horizon. But that’s not true. Using the inflation adjustment both for withdrawals and for the portfolio value over time you run a much larger risk of shrinking or even depleting your capital over 30 years. In the table below, I display the failure probability calculations for a 60/40 portfolio, 30 years horizon, monthly withdrawals, 1/1871-4/2019, all calculated with my SWR toolbox. You would have failed to maintain your purchasing power in the portfolio over 30 years with a 42.8% probability. And an almost 60% probability conditional on an elevated CAPE ratio (above 20. Note: Today’s CAPE is 30!!!). In fact, in the historical simulations, you faced an 8.9% probability of totally depleting your money after 30 years, conditional on an elevated CAPE (>20). This shows you how totally senseless the alleged 4% failure probability for capital preservation is! If it sounds too good to be true it probably ain’t!
How to spot this “lie” in the field: Without any further clarification, one would almost always assume that return figures are nominal. Every time a study states that withdrawals are inflation adjusted but there is no further mention of what was done to the portfolio values over time, one could almost safely assume that the portfolio value over time is not inflation adjusted (and is thus a completely useless statistic, more on that below).
How I deal with this issue here on the ERN blog: In safe withdrawal simulations it is an absolute must to use the double inflation adjustment, i.e., inflation adjustments to both withdrawals and final portfolio values to use comparability of final portfolio values across time. I try to be really, really explicit about what I do: Unless explicitly stated, the default position is that all calculations here on the blog in general and the Safe Withdrawal Rate Series, in particular, are done using real, CPI-adjusted returns. It’s the only sensible way to do it.
3: Ignoring different inflation regimes
Notice that none of the issues in item #2 above would be that much of a concern if the U.S. economy simply had had a constant inflation rate over time. If we’d faced x% constant inflation every year then everyone can just easily transform some nominal median portfolio after y years back into real numbers. But inflation rates do differ wildly over time, see the chart below. Not adjusting for inflation would have melted a $100 nominal dollar amount to anywhere between $20 to $85 since the existence of the U.S. Federal Reserve:
To demonstrate why we should be cautious about summary stats that rely purely on the nominal final portfolio value, let’s look at the following numerical example. There are 5 (completely fictional) observations of retirement cohorts with their final portfolio value after 30 years. They are conveniently ordered from cohort 1 = highest to cohort 5 = lowest. The median among the nominal values is exactly $1m. If we now deflate the nominal values by their corresponding average inflation rate, i.e., Real = Nominal/(1+Inflation)^30, then we get the real portfolio values anywhere between $356k and $662k. But do you notice something peculiar? The ranking is completely different for the real portfolio values:
- The cohort that has the median nominal value now has the lowest real portfolio value.
- The median real portfolio value is $416,479, which is the portfolio of the cohort that has the highest nominal value
In other words, the ranking is completely reshuffled when adjusting for inflation. (For the math geeks, it’s because the median operator and the inflation adjustments are highly non-linear operations) Consequently, reporting any stats on the nominal portfolio value (median value, the probability that we exceed a certain threshold, etc.) is a completely moot exercise. The only figure that’s not impacted is the probability of hitting $0 because – you guessed it – a 0$ final portfolio value is the one single value that’s the same in nominal and real dollars! So, we can’t deduce anything about the real, inflation-adjusted numbers from the nominal values. And by the way, it’s also of no use to, for example, “guesstimate” the median real value through MedianReal = MedianNominal/(1+MedianInflation)^30, which would be $1,000,000/1.02^30 = $552,071 in this case. Very different from the actual median value.
So, not only do we overestimate the final portfolio value by looking at the nominal values (see item 2 above) but by looking only at the nominal final portfolio values we lose so much important information about the distribution. I have to roll my eyes every time I read how the Trinity Study elaborates on median nominal final portfolio values. Or how FIRE bloggers parrot senseless stats like “the median portfolio value using the 4% Rule would have been 2.9x the starting value” or “you’d have a 96% chance of preserving your initial capital” (see item 2 above). These are totally useless junk statistics when derived from nominal values alone! That’s why I’m such a stickler on doing my analysis with double-CPI-adjustment: withdrawals and portfolio values have to be inflation-adjusted over time to make them comparable across different inflation regimes! Everything else is junk science!
How to spot this “lie” in the field: Again, the gold standard in quantitative financial/economic research is to do the inflation adjustment period by period. It’s also quite common in Finance to adjust returns by calculating the excess return over a risk-free benchmark (i.e., 3-month T-bill, etc.) instead of CPI inflation. If nothing is mentioned, one can assume almost for sure that these are again nominal returns.
How I deal with this issue here on the ERN blog: Again, I always do the inflation adjustment period by period with the exact inflation number in that month. I wrote a nice little summary about how exactly I do the inflation-adjustments in my SWR Series, see this post, especially the appropriately named section “How I account for inflation in my Safe Withdrawal Rate Series!”
4: Pick the “right” Start/End Points
Notice that in my own calculations I always work with return data going back to 1871. The average real stock return since 1871 has been about 6.6%, see the chart below. Notice that due to the y-axis log-scale, the red CAGR return line is one straight line!
Looking at the picture above I can come up with a nice new opportunity of number fudging numbers and data malfeasance. If we start the equity return interval at the trough of the market in 2009, we’d have observed a 14%+ average equity return, see the chart below. And that’s above inflation! Quite impressive but likely not repeatable indefinitely. But starting at some of the other market troughs, like 1975 or 1982 we can still boost the average return by 1 to 2 percentage points. Notice a problem here? We are now taking the average over x bear markets and x+1 bull markets. And that can move up your average return to values that are unlikely to be repeated going forward from today’s perspective. Especially after we’ve now experienced 10+ years of a very impressive bull market!
How to spot this “lie” in the field:
- I always ask myself, is the return series long enough to include at least several up and down markets? If there’s never been any downmarket, be careful about extrapolating returns. We’ve had 14% annualized returns since March 2009 but nobody in their right mind would believe that this trend will continue long-term. That said, some folks who entrust their money with some of the new lending platforms (e.g., debt, mezzanine debt or equity investments, personal, business and real estate loans) should ask themselves how those investments will perform if we go through another recession again.
- Even if the return series includes multiple up/down cycles, check if the starting point was “conveniently” picked right at the bottom of a down market while the endpoint is many years into the bull market. If an average return is calculated with a starting point later than the earliest available date, always ask yourself if there’s a sensible reason to do so.
How I deal with this issue here on the ERN blog: Some folks use data starting in 1926. I use data going back to 1871. For all intents and purposes, the results of safe withdrawal rate studies will be very similar. That’s because of the really scary Sequence Risk episodes (Great Depression, 1970s/80s) all happening after the 1926 cutoff. Also, because 1926 was more or less in the middle of the 1920s bull market the average real return 1926-today is strikingly similar to the one using data starting in 1871.
Conclusion
Whoah! That’s a lot of material for one post already. And we’re just getting started! I got another set of “lies” for a future post.
In any case, we started with 12% average equity returns (actually 12-18% according to the great finance genius Dave Ramsey) and now knocked it down all the way to 6.6% if we look at the very long time trend of real total equity returns. That’s still extremely impressive! Multiple percentage points above the average real GDP growth! It’s also significantly higher than many other major asset classes: Bonds, gold, silver, etc. so, please don’t misinterpret this post as trashing stock investments. I hold way too much (and I mean waaayyyy too much!!!) equity percentage in our portfolio to be a stock market trash talker. Quite the contrary, I’ve gone on the record recently that the whole scare about the yield curve inversion was exaggerated. I’ve also gone on the record in early January, both here on the blog and on the ChooseFI podcast, episode 109R, saying that the equity drop in Q4 of 2018 was likely overdone. Looks like good advice so far! I’m neither a cheerleader nor a doom-and-gloom guy. Just a realist.
And then, finally, be cautious about some of the cheerleading about the 4% out there. I’m a skeptic, as you all know, see the Ten things the “Makers” of the 4% Rule don’t want you to know (SWR Series Part 26). If people using the exact same return data as I come to very different conclusions I’d get very suspicious!