Allocate Smartly

Super-Secret Proprietary Black Box Strategies

Note: This is a rare non-geeky, non-quantitative, stream of thought blog post.

Because we’re so deep into this world of Tactical Asset Allocation (TAA), we’re sometimes asked for our thoughts on such-and-such black box TAA strategy. By “black box” we mean a strategy for which the trading rules are not disclosed to investors (nor to us).

Black box strategies inherently sell on the idea that they have some super-secret hidden knowledge about the market, which you can participate in for a premium fee.

That could be the case, but we’ve been doing this too long to be anything but skeptical.

The genesis of Allocate Smartly (the root of our skepticism):

We replicate and track 70+ TAA strategies, but the genesis of what we do came long before we launched this platform. In fact, it came long before we were even interested in TAA at all.

Our earliest foray into quantitative modelling began 20+ years ago, and focused on “swing trading” strategies, which hold positions from less than a day up to, at most, a couple of weeks (back then it was just a geeky obsession, and not something we shared publicly).

Unlike TAA, these were relatively short-lived market inefficiencies, and as you would expect, they’ve mostly been traded out of the market, but it was a very instructive time for us and how we view trading strategies today.

We analyzed a lot of strategies trading all sorts of variations on the swing trading theme. What we found was that the majority were based on the same core observation: short-term mean-reversion. Excessive short-term weakness (i.e. overreaction) usually led to a short-term rebound, and vice versa (the classic example being Larry Connors’ RSI(2)).

There were fewer strategies that offered something truly unique; some unique factor that meant the stream of future returns were likely to be, for better or for worse, different.

Most only varied at the margins. The effectiveness of all those strategies that were based on that same core mean-reversion observation might differ in the short-term, but over the long-term, they lived (and eventually died) together.

Parallels with Tactical Asset Allocation:

There are parallels between our swing trading origin story and the TAA strategies we trade now.

The core observation being traded by many TAA strategies is “trend-following and momentum”. To oversimplify: assets going up over an extended period of time tend to continue going up.

The analogy is not perfect. Unlike swing trading strategies, trend-following and momentum have worked for essentially as long as financial markets have existed. That provides a level of comfort that they will continue to work in the future.

But where the analogy holds is that many of the strategies that we track are also only different at the margins, and we have to work to identify exposures that are truly unique. Why? Because by combining strategies that are truly unique, we achieve a higher degree of real diversification to weather future unknowns.

Some examples of unique exposures include (this list is far from exhaustive):

Like Consumer Reports™ does for washing machines:

We’ve cast a very wide net with the TAA strategies we track. Quite intentionally, none of these are our strategies. From the launch of this platform, we’ve always sought to broadly represent TAA as a whole – to serve the same function for TAA as Consumer Reports™ does for washing machines – an unbiased independent arbiter of performance across the entire spectrum of solutions.

Put another way, one could say TAA is good (you’re right) or bad (you’re wrong), but one cannot say the particular collection of strategies we track is good or bad. The strategies we track are simply a reflection of TAA as a trading style.

What does this have to do with black box TAA strategies?

When we’re presented with a black box TAA strategy that claims to have some super-secret knowledge about the market, our response is always the same: prove it.

More specifically, prove that you’re trading based on some factor that we’re not already capturing via the large body of tactical strategies we track. Because if your strategy is only different at the margins, we already have plenty of options for that exposure which we can actually analyze intelligently because the rules are available to us.

There’s only two ways to do that without revealing what’s in the black box.

  • An independently verified out of sample track record of sufficient length. Because we’re talking about long-term, beta heavy strategies like TAA, “sufficient length” is tricky to define.
    In lieu of knowing what’s in the black box and being able to do our own analysis, we think the absolute bare minimum to start to understand the performance of a TAA strategy would be 3 years of independently verified out of sample data (and even that’s just a pittance).
  • Provide a “toy example” of the supposed super-secret knowledge.
    A good example is Financial Mentor’s Optimum 3. The strategy is a black box to investors, but the author shared the rules with us allowing us to perform our own independent replication. We didn’t share those strategy rules, but we did share a simple toy example demonstrating how the strategy provides exposure that is unique to others we track.

Our skepticism is born out of many years of doing this type of analysis. We’ve seen too many “next big things” come and go to be impressed by anything other than something very tangible that we can independently analyze.

Two more points:

These didn’t fit neatly into the diatribe above, but we think that they are important points:

  • When a black box strategy is only different at the margins AND the backtested performance is significantly better than other similar strategies, that’s a strong sign of overfitting.
    Why? If you don’t have some unique exposure, it’s highly likely that the boost in performance came from adding some niche rule to the strategy that is overly fit to the past and unlikely to be robust to the future.
    Is an overfit strategy a bad strategy? Not necessarily. But it is especially likely that the strategy will fail to perform in the future as it has in the past.
  • Beware the “perpetually revised” strategy.
    Our thinking on various things has evolved over time. There’s nothing wrong with that. But we’ve all encountered developers that, when their strategy fails to perform, release a hot new version that corrects for all past mistakes with the promise that this one really will work!
    If the developer “locks in” past returns and shows their past poor performance forever, great. No issue there. But if they wipe the slate clean with a fresh backtest as if they never stumbled, that’s a bad joke and you should run the other way.

In summary:

Returning to the original question, what do we think of such-and-such black box TAA strategy?

We think that the developer needs to demonstrate why the strategy provides exposure that is unique to the large body of tactical strategies already available – not just at the margins, but substantially.

The developer can do that either through independently verified out of sample data of sufficient length, or by providing a toy model of their super-secret knowledge.

In the absence of that, we are thoroughly unimpressed and consider the black box to be purely a marketing tool.

New here?

We invite you to become a member for about a $1 a day, or take our platform for a test drive with a free membership. Put the industry’s best Tactical Asset Allocation strategies to the test, combine them into your own custom portfolio, and follow them in real-time. Learn more about what we do.

* * *

Edit 09/27/23: Just meditating on this stream of consciousness a bit more…

We can think of 3 cases where we added a strategy to the platform and the rules were withheld from members, but known to us. We independently replicated these 3 strategies with the same level of rigor members expect from us, but admittedly didn’t provide a good toy example like we did with Optimum3.

That’s a grey area: independently replicated by a trusted source (or at least that’s how we hope members view us), but no clear indicator of the strategy’s inner workings. We’ll commit to doing a better job providing a clearer peek under the hood if we add black box strategies to the platform in the future.