The Worst Back-Test

Back-testing, BREIT & a new fund launch

I just launched a fund that has the worst back-test anyone has ever seen, and I couldn't be more excited. I'm going to tell you why.

The worst ten-year period for any backtest is the next ten years. -Michael Batnick

The first time I saw BREIT’s eye-popping, logic-defying performance I immediately took it to a quant shop famous for statistical replication. But I didn’t tell them what it was. Just “can you replicate this return stream?”.

I was expecting a long response with charts and graphs and implements of destruction. All I got back was this: "ha ha ha"

So I called. "I'm sorry, but I don't get it. What do you mean?"

"That was a joke, right, Phil? This is just leveraged treasuries"

"No, it's BREIT. Can you replicate it?"

"Sure. Leveraged treasuries"

Yah, well, risk is forward looking. That was not going to work.

So I tried a fundamental replication. Replicate the characteristics of their holdings. After all, the building doesn’t know or care if it’s a public or private REIT. It’s a building! I can compose a portfolio of the same class buildings, the same geographies, the same sub-sectors.

And we did. We painstakingly put together a ten year back-test.

And wouldn’t you know, it was the worst back-test anyone has ever seen.

---

Here is a good place to jump in with some context on back-tests. This next part, I wrote is six years ago, but nobody read it because nobody knew or cared who I was back then. I think it's worth reposting.

Anything I know about business I learned playing fantasy baseball.

When I was in high school my big brother was an actuarial student, and his group of obsessive math-junky friends spent no less time preparing for their fantasy baseball auction than they did on their actuarial exams.

One year they had a late cancellation and needed one more team owner. “Kid, pack your bags, you made it to the big leagues.”

This is long before Michael Lewis pulled open the curtains on statistical analysis in sports, long before Brad Pitt made trading for players who could stand there and take a walk look a lot cooler than Charlie Sheen had made a 110 mph beanball, and even before the internet made xFip more readily available than the September National League standings were a decade prior.

I’ve had a couple intense business negotiations during my career. I can assure you that none came close to the intensity of a hyper-competitive 14 year old who thinks he has an informational edge and has not yet become distracted by girls or responsibilities or life, trying to plead, nudge and cajole his rival 14 year old to trade him Scott Rolen for Troy Percival in a free fantasy baseball league.

It was pointless, and it was fun, and we honed our skills in a way that university students and eager interns could only hope to.

Our league was what would be called a “quant” league today, a “sabr” league by those in the know, or a “math” league in layman’s terms. If you didn’t know your babip from your ISO, you weren’t winning the league.

But it didn’t take long for everyone to catch on to the same metrics, and they became the new baseline. Not knowing the math could lose you the league, but knowing the math wasn’t enough to win the league.

If you wanted to win the league, what you had to understand was context.

All of us were able to look at a pitcher’s balls in play and isolate the signal from the noise, but if you didn’t understand that a middle reliever with 60 innings was a less reliable sample to project forward upon than a starter who posted 200 innings, it was going to cost you.

If you didn’t understand that a baseball player’s career arc might look like an upside-down U, but that doesn't mean that each season moves in a linear way, that too would cost you.

If you didn’t understand that young pitchers are fool's gold, that injured pitchers don't come back the same, that speed peaks on day one while power peaks a few years later, those mistakes will cost you.

You can calculate the value of the data set or tinker with the precision of projections, but at the end of the day the ability to put that data into a broader context is what set apart the teams that were always on top.

The first ETF back-test I ever saw blew my mind. It was 2006 or 2007, and I had a brand new job as an ETF product developer, and I’m staring at a spreadsheet that showed the five year returns of a gaming industry index.

It took me an embarrassingly long time to figure out the context, that the gambling industry is extremely sensitive to economic conditions and is basically a proxy for disposable income.

That index would crash during the global financial crisis, as you might expect.

Its become a meme to mock back-tests, and it’s a fair meme. No one has ever seen a bad back-test. But the memes miss the point. If you read a back-test without considering the context, you deserve the investment you were sold. What you can learn from a back-test, what could make your portfolio better, how you model risk and cyclicalities, its all context.

A back-test can show you where the holes are, what the factor sensitivities are, the consistency of the alpha, which market environments the strategy will thrive in, and in which scenarios the strategy will not work.

It creates a basis for discussion for when to deploy a strategy or how to protect your portfolio from certain outcomes.

---

Ok, REITheads, that's enough of that. Let's get to the good stuff.

So our back-test was terrible, and I stewed on it. I ruminated on it. I obsessed on it.

And then the a-ha moment. Of course. I took a bath and then cried eureka! The historical backtest only reinforces the divergence between public and private valuations. What matters is whether they will mean-revert in the future or not.

So we ran valuation analysis every which way. We looked at the implied cap rate, FFO and AFFO, NAV prem/discount, debt to gross asset value. We quantified Blackstone's operating efficiencies, capital sourcing and deal sourcing. We tried to quantify how much of the crappy public REIT back-test we could attribute for.

And at the end of it, what we saw was the inevitability of a price convergence. A massive delta between public and private REITs. A coming drawdown that private REIT investors are not prepared for.

So we designed a life raft for them. That life raft launched today. I don’t have a big fancy sales team, I don't have fat selling commissions to offer brokers. That’s not what we do.

I designed a life raft. Investors can grab it or not. I've done my part. I launched a fund that has the worst back-test anyone has ever seen.