Why we don’t use Robo-advisers

1. Most investors will get much smaller excess returns from the tax savings than what the Robo-advisers claim.
2. Robo-advisers pick an asset allocation that may have tax inefficiencies built in for some investors, worth at least several basis points of annualized returns.
3. Smart investors should still perform Tax Loss Harvesting, but it’s best to DIY because the benefits may not outweigh the Robo-adviser fees, especially if taking into account some of the potential inefficiencies introduced in the Robo-adviser target portfolios.

Intro

In our earlier post we showed how to be your own DIY Robo adviser. That post got quite long and even then it didn’t deal with all the issues of Robo advisers and especially tax loss harvesting (TLH). Here are some additional thoughts and caveats about TLH, Robo-advisers and their – in our opinion – slightly “creative” marketing practices.

The alpha (=excess return) that’s quoted by Robo-advisers applies to a tiny fraction of the population; if you don’t satisfy all of their assumptions your excess return might be smaller, significantly smaller or even non-existent. Specifically, to get the full benefit of the quoted tax alpha you have to satisfy the following assumptions:

1. You are in a very high federal tax bracket and live in one of the high tax states, e.g. CA, DC, HI, MN, NJ, NY, OR, VT
2. Your marginal taxes on capital gains are hit with the extra 3.8% Obamacare tax
3. In years with very high TLH short-term losses, you are not constrained by the maximum of $3,000 you can write off from your ordinary income in one year. Rather, you have boatloads of capital gains from other unrelated assets that you can offset with the TLH
4. In addition to your initial investment, you invest another roughly 35-40% of the initial investment every year, to add fresh money for better TLH opportunities

(Sources: Betterment, Wealthfront basic TLH, Wealthfront advanced TLH)

Uhhhm, how many people do we know that satisfy these conditions? We know some that might satisfy conditions 1, 2 and 4. We don’t know a single person that satisfies condition 3. Let’s look how much of the tax alpha is left over after we adjust some of these assumptions above to more realistic numbers, more representative for the typical extreme early retirement saver:

The alpha from TLH, expressed as a percent of excess return is calculated only for the taxable portion of the portfolio.

Say, if a Robo adviser quotes you 0.77% of annualized tax alpha but you have half your assets in a retirement account with them, then your tax alpha just shrunk to 0.385% of your portfolio value because TLH only works in the taxable account. Assuming you pay 0.15% in annual fees, your excess return after fees just shrunk from 0.62% to 0.235%. Of course, there is the option to keep only taxable accounts with the Robo-adviser and the tax-deferred stuff outside, but be aware, that if you trade assets in own your tax-deferred accounts (or other taxable accounts outside the Robo platform) you could potentially invalidate the TLH by creating wash sales. Good luck trying to sort that out with the IRS.

You could be in a much lower tax bracket

So, you don’t have $400,000+ in taxable income subjecting you to a 42.7% to 44.5% combined marginal tax rate? Say, you are in the 25% bracket plus 4% for the state, then your tax alpha just went down another 35%.

TLH is not arbitrarily scalable

Here’s a good analogy: Imagine you hire a tax accountant who charges $1,000 and he saves you $2,000 in taxes. Great job! Does that mean you should hire 10 tax accountants for a total of $10,000 and they save you $20,000? Probably not. In a similar way, TLH works potentially really well for small amounts, but scaling everything up by a factor of, say, 10 times or 100 times gives you less than 10 times or 100 times the dollar amount in tax savings. Here’s an example. Suppose you just started with a fresh shiny taxable account of $10,000 at a Robo-adviser. Suppose the market drops quite a bit and generates the motherlode of TLH, $3,000 in short-term losses. By the time you do your taxes next year the market might have already recovered but you can now write off $3,000 from your ordinary income. At a 44% marginal tax rate (roughly what Mr. and Mrs. ERN are paying for Federal and state) you just saved $1,320. Not bad. You realized a 13.2% tax alpha.

What if you had $100,000 in investments with the Robo-adviser? You now sit on $30,000 in short-term capital losses. Too bad you are only allowed to write off $3,000 this year. You can carry forward the remaining $27,000 into next year. So, over the next 10 years you slowly collect $1,320 per year. If you discount future alpha by 7% p.a., then the discounted tax reduction is only $9,920, rather than $13,200. That’s a 25% reduction in the alpha quoted by the Robo-adviser. If you had invested $1,000,000 with the adviser you would be dead before you collected the 100 years worth of tax write offs. Even if you lived that long, the tax benefit is reduced by 85% due to discounting future payments at 7%.

How do the Robo-advisers get around this obstacle with the $3,000 maximum tax write-off for short-term losses? It’s in the fine print!

Betterment at least acknowledges in its white paper that there exists a $3,000 annual max on netting ordinary income. They give this example: “[W]e assume that harvested losses are offset against ordinary income up to $3,000, and the excess, if any, against LTCG outside the portfolio. This mimics a typical scenario where the investor waits until the end of the year, when he knows exactly how much LTCG he can realize tax-free, and still leave enough losses to fully utilize the more valuable ordinary income offset (for example, if harvested losses add up to $8,000 by year-end, he would trigger $5,000 in LTCG).” This is very inappropriate! They assume tax gain harvesting during your high income, high marginal tax years. Why would anybody do that? We want to wait until retirement and try to pay zero or at most 15% capital gains tax. Betterment basically invents phantom income from offsetting phantom tax gains that no sane person would have voluntarily realized.

Wealthfront goes even further because they apparently assume that the entire short-term loss can be written off at the high marginal tax rate, at least that’s how I understand their statement: “[Y]our net realized short-term capital loss on this position for the year is $10,000, which generates a $4,000 tax savings when applied against your income and other realized gains [my emphasis] under the assumption of a combined federal and state tax rate of 40%.” How sneaky! They know that you can’t write off the entire $10,000 in short-term losses against your ordinary income, so they have to conjure up “other realized gains” of $7,000 out of nowhere. Ostensibly, these are short-term gains since they are taxed at the same rate as ordinary income. Where the average investor gets these short-term gains in the middle of a recession stays a mystery. If you thought that the Betterment long-term gain harvesting was iffy, the assumption that someone would have sizable short-term gains in the same years when the big stock market crashes occur is a real whopper. We don’t believe those tax alpha estimates one bit!

Back to our example, here’s how much of a haircut we might have to apply to the tax alpha if we had $30,000 in short-term losses in 2015. Assume a 44% marginal tax rate on ordinary income, about representative of Mr. and Mrs. ERN. At that tax rate you can save $1,320 on your taxes in the years 2015-2024, slowly walking down your stash of $30K in short-term losses. If you retire after two years, however, and your ordinary income falls into the 15% marginal bracket, all your future $3,000 write-offs are only worth $450. If you retire after two years and you live exclusively off passive income like dividends, you will get zero tax benefit because your marginal tax rate for ordinary income might fall to zero in the lowest two federal brackets:

Thus, the actual tax alpha can be significantly lower than what’s calculated by Wealthfront and Betterment. If you are really sure you can stay in the 44% tax bracket for the entire 10 years, tax savings are 25.7% under the Wealthfront assumption. If your marginal tax rate drops to 15% in the future, you get a whopping 62.1% reduction in savings. If your marginal rate drops to zero in retirement, more than 80% of the assumed Wealthfront tax alpha goes up in smoke:

The IRS wants you to net short-term losses with long-term gains

If you thought things couldn’t get any worse, you’re in for another bad surprise; one that Robo-advisers will probably not advertise very aggressively. If you enter retirement with a big stash of harvested tax losses, even if your marginal rate on ordinary income is 15% in retirement you may still lose all of your carry-over tax losses. Imagine in retirement you live off your passive income of dividends and long-term capital gains. Your carry-over short-term losses are netted against your long-term capital gains before you can apply them ordinary income. So, imagine in the year 2017 you generate passive income to fund your retirement: $24,000 in interest and rental income, $24,000 in dividends and $24,000 in long-term gains. The $24,000 in ordinary income goes above your standard deduction and is indeed taxed at 15% marginal. You would definitely enjoy chopping off $3,000 from your taxable income, right? But all your $24,000 in short-term losses will first be netted against the $24,000 in long-term gains. But those gains would have been taxed at 0% anyway because you stayed in the lowest two income brackets where dividends and capital gains are not taxed. Your $24,000 in short-term losses from 2015 are now gone forever. Even if in a future year you have no long-term capital gains, your $24,000 are kaput!

TLH only defers taxes. It doesn’t necessarily eliminate them

This is a concern, though not quite as bad as some of the other ones. Under certain conditions it is legitimate to assume that TLH actually saves you upfront without any long-term costs:

1. You do what GCC does so well: Pay zero marginal taxes on long-term capital gains (and all other income as well, for that matter)
2. You never realize the long-term gains and bequeath your portfolio to your children. Their cost basis will step up to the market value

We hope to accomplish both so this concern about TLH is not relevant for us, but not everybody has this same plan.

Inflexible asset allocation

It’s almost hard to believe, but even the high tech geeks at Betterment and Wealthfront, these geniuses of FinTech, actually commit some of the worst beginner mistakes of asset allocation and tax minimization. At least that’s the way we read their online material. Here we summarize the recommended weights of Betterment and Wealthfront, using the most risky allocation. We also include the current yield (per Yahoo Finance, ostensibly the 12M rolling dividend yield):

 

Looks pretty reasonable. Both suggest 90% equities and 10% bonds. They do the usual stuff, like Muni bonds in the taxable account, taxable bonds and REITs in the tax-deferred. Standard stuff. But for the average investor, shifting their portfolio over to the Robo-advisers, 50% in a taxable account and 50% in a tax-deferred account it would be sub-optimal to pick those weights:

1. For Betterment, you’d be better off having the high-yielding VTV only in the tax-deferred account (32.4%), and the lower yielding VTI only in the taxable account (32.4%). You’d have the exact same overall allocation, but you’d not have to pay taxes on the additional 2.73%-2.10%=0.63% in dividend yield. Assuming 18.8% federal taxes on dividends (15%+3.8% Obamacare tax) and 9% state = 27.8% combined, you would reduce the tax drag in the taxable account by 16.2%*0.63%*27.8%=0.0284%. Since the taxable account is half of the total portfolio, you throw away 0.0142% in annual returns. Doesn’t sound like much, but this is just the beginning.
2. The Muni Bond fund MUB is tax-free for federal taxes. But not necessarily for the state. If you live in a high tax state like California or New York, you should buy the Muni ETF that’s also exempt from your state taxes. MUB will generate about 80% of income that’s not state tax-free. So replacing the 5% share of the MUB with CMF or NYF, depending on you residency, and assuming a 9% marginal state tax, you can save another 5%*2.48%*80%*9%. Another 0.0089% free savings in the taxable account, 0.0045% for the overall portfolio. Small amounts, but they will add up!
3. Betterment still keeps about 4.5% of the taxable portfolio in taxable bonds. They have yields of up to 4.82% that are taxed at the high personal income tax rate. So, let’s move those over to the tax-deferred account and instead bring the 4.5% VBR with a low 2.09% dividend yield in the tax-deferred account over to the taxable account. Tax saving is (0.6%*3.45%+2.4%*1.56%+1.6%*4.82%)*44%-4.5%*2.09%*27.8%=0.0328% in the taxable account = 0.0164% in the overall portfolio
4. In the Betterment portfolio, there’s a 5% allocation to XLE (Natural Resources ETF) with a dividend yield of 3.61. It’s best to put that into your tax-deferred account and replace it with VTI (dividend yield 2.10%). Your tax savings are 5%*(3.61%-2.10%)*27.8%=0.0210% in the taxable portfolio, 0.0105% for the overall portfolio
5. Muni Bonds only make sense for people in the highest income brackets. Yields are lower than for bonds with comparable credit rating and duration statistics. There is no free lunch, after all and thus much of the tax advantage is already arbitraged away into getting lower interest rates. If you are in a very low tax bracket, you could be better off buying the taxable bond fund instead. Surely, the rocket scientists at Betterment and Wealthfront will take that into account, right? Wrong! The recommended allocation to MUB seems to be independent of your income. If you’re making $20,000 a year (likely paying zero marginal taxes, after standard deduction and personal exemptions) or $400,000 per year, the asset allocation seems to depend only on the 2-3 questions on risk-tolerance. For people in low (or zero) tax brackets, why not use the LQD bond fund with a yield of 3.45% (=2.9325% after tax in the 15% bracket) instead of the 2.44% after tax yield of the Muni fund. At a 5% portfolio weight, you are losing 0.0246% in your taxable account. If you are more risk averse (risk tolerance 4.0 and below in Wealthfront) and the Muni fund weight goes up to 35%, your drag from the inefficient allocation is now a whopping 0.1724% p.a., almost as high as the Wealthfront annual fee.

To sum up, you could easily lose several basis points of returns p.a. due to inefficient allocation of capital between the taxable and tax-deferred accounts. Smart DIY investors would take all of this into account in their DIY portfolio.

The one sole bright spot: Direct Indexing, as performed by Wealthfront

Wealthfront is the only Robo advisor that offers one feature that’s not so easily replicable even for finance geeks like Mr. ERN. To be sure, Mr. ERN has decades of trading experience. But he still wouldn’t attempt what Weathfront is currently doing: direct indexing. It, together with tax loss harvesting, is a potentially powerful tool and worth a careful look, but even then it may be useful for only a small number of extreme early retirement savers, and even then the benefits might be vastly exaggerated.

Imagine in an index with 1,000 stocks (assume equally weighted just for this example), 600 stocks go down by 1% each and 400 stocks go up by 1%. The index will be down by 0.2%. You could do the TLH, but it wouldn’t be very efficient because you are selling the entire index, including some stocks that are up. If, on the other hand, you owned each one of the 1,000 stocks you could harvest only the losers, worth 0.6% of losses, which is three times the harvestable loss of the index. Even more intriguingly, if 400 stocks go down by one percent and 600 go up by one percent, the index cannot harvest any loss, while direct indexing can harvest 0.4% worth of losses. You generated a tax loss, even though your investment is up overall, how amazing is that? Besides, owning all the stocks individually saves you the index ETF management fee. That may be only 0.03% or 0.05%, but every little basis point counts for us frugal folks!

So, kudos to the folks at Wealthfront. But there are still the concerns about generating “too much of a good thing” in the form of tax losses that are limited by the $3,000 limit the IRS puts using short-term losses as offsets for ordinary income. When you are already hitting that limit every year, generating more tax losses from direct indexing would be the last thing on my mind. Besides, the direct indexing is only available in the VTI portion (max 35% weight), and only in your taxable account.

Final Verdict

By no means do we want to imply that tax loss harvesting is not worth pursuing. It is! Mr. and Mrs. ERN have been doing it for many years. Once you account for the $3,000 maximum write-off per year against ordinary income, and the netting of short-term losses with long-term gains during retirement, the benefits shrink pretty substantially. Still worth it to do and our DIY guide explains how but the benefits are much smaller than purported by the aggressive marketing of the Robo-advisers and some finance bloggers.

The costs at 0.15% to 0.25% for the Robo-advisers is too high for us and likely many other investors planning for retirement, especially if you take into account the additional 0.03% to 0.10% in performance drag in the Robo portfolios. If you are already retired, and living mostly on passive income, like the typical turbo-retiree, Robo-advisers are essentially ineffective from a tax alpha perspective.

For us personally TLH has also become less useful over the last few years. We stopped contributing to equity funds in taxable accounts and instead pursued some other investments: Real Estate Private Equity (Multi-Family Housing) and an option trading strategy that Papa ERN developed and now runs live. Our last tax lots of our equity funds are now solidly in the green and even a pretty bad bear market will not change that. We see essentially zero benefit from TLH now and going forward.