This post covers an issue rarely discussed in backtesting: the day’s last real-time price shown at 4pm ET often differs slightly from the day’s official closing price determined shortly after 4pm. This is not an Allocate Smartly issue; it’s an oddity of the exchanges. Every so often this difference can cause discrepancies between backtests based on the close and investors’ real-world experience.
Fortunately, because Tactical Asset Allocation is designed to capture broad trends and ignore day-to-day noise, TAA strategies have been robust over the long-term to issues like this. In fact, they’ve been robust to far larger perturbations, such as adding a full 1-day lag between signal and execution.
Having said that, we take the quality of our results very seriously, so any time there’s a discrepancy from what we signal in real-time, no matter how small or infrequent, it’s important to us. This issue bit us on one strategy last month, inspiring this deep dive.
Background:
Backtests of long-term strategies like the ones you find on this site often assume trades are executed at the closing price of the day, based on the most up-to-date data at that point in time. By doing so, strategy authors are able to create much longer backtests.
Here’s how the timing of everything works at Allocate Smartly (learn more):
- Starting at the open, we show each strategy’s expected allocation at today’s close.
- We continue to update the expected allocation throughout the day as asset prices change. Allocations tend to be stable through the day, but occasionally a strategy finds itself on the edge of this asset vs that asset and bounces back and forth.
- We assume that an investor executed the final allocation of the day at the closing price.
The allocation we show at 4pm ET is based on the last real-time price provided by the exchanges at 4pm. Later in the evening, between 11pm and midnight ET, we do a complete site refresh based on the official closing price.
There is sometimes a slight discrepancy between the two. On NYSE Arca, where most of the ETFs we model trade, that official closing price is usually determined about 10 minutes after the close.
This discrepancy between the last real-time price and the official close, on very rare occasion, causes a strategy to change the allocation we signaled in real-time at 4pm.
How common is this problem?
We follow 44 ETFs for the purpose of modelling the 60+ strategies on this site.
The average absolute difference between the last real-time price and the official close for those 44 ETFs over the last two years has been just 0.0057% (less than 1/100th of 1%). Individual ETFs have varied from 0.0002% (ETF: BIL) to 0.0344% (IWM).
We’re talking very small differences here, but those differences were enough to flip the position signaled in real-time by at least 5% in 14 instances this year.
That may seem like a lot, but remember, every monthly strategy signals an allocation 21 times a month (because of alternate trading days). In one year, we signal a total of 14,940 new allocations across the site. In other words, there are 14,940 opportunities for this issue to bite us.
That means this problem only arose 0.09% of the time, or once every 1,067 allocations signaled. This is a very uncommon issue (but an issue nonetheless).
What is the practical impact of this problem?
In the short-term, i.e. the signal for a particular month, it could be significant – sometimes for the better, sometimes for the worse. In the long-term however, it’s almost certainly a non-issue.
In past articles, we’ve analyzed far greater discrepancies, like adding a full 1-day lag between the time an allocation was signaled and when it was actually executed, and found no significant impact over the long-term.
Tactical Asset Allocation is designed to capture broad market trends, so short-term deviations like the one discussed here tend to wash over time.
Further, assuming that members are combining multiple strategies together into a Model Portfolio (a core feature of our platform), these short-term deviations are further diluted.
Which allocation is the “right” allocation? Should investors fix the discrepancy the next day?
In terms of what we officially stamp to the historical record for posterity, the “right” allocation is the one based on the official close.
In terms of what the strategy is actually trying to measure, both allocations are equally valid. If some insignificant noise is enough to flip a strategy from this position to that, neither position is more or less “right”. The difference is just an unfortunate byproduct of noise.
In our own personal trading, when issues like this come up, we usually split the difference. If noise caused a strategy to flip a % of the portfolio from asset A to asset B, we’ll split that % between the two. If the two positions are polar opposites, we may even leave that % in cash. We’ll use discretion to time when to fix the discrepancy.
Strictly adhering to a change we know was driven by noise just doesn’t make sense in the real-world.
A tangentially related issue: flip-flopping before the close
A different, but similar issue to the one presented here is a strategy flip-flopping between two allocations before the close as asset prices change throughout the day. TAA strategies tend not to do that due to their long-term nature, but occasionally a strategy does find itself on the edge of this asset vs that asset and bounces back and forth.
There’s a risk that the member executes an allocation late in the day thinking it’s the final allocation, only to have it change again before the close.
Our response is the same as in the previous section: (a) it’s uncommon, (b) TAA has been robust over the long-term to discrepancies like this, (c) those discrepancies are diluted if part of a broader Model Portfolio, and (d) practically speaking, both positions are equally “right”. See above.
Technically, this issue of flip-flopping before the close is unrelated to the issue being discussed in this article, but the remedy is the same, so we thought it worth mentioning.
Why not model strategies based on the last real-time price rather than the official close?
Our focus is always on providing accurate, actionable results, so we’re very willing to do that if it makes sense.
It would require us to maintain a separate data point for the close (in addition to the cash price and the dividend-adjusted price). This would complicate analysis and lead to lots of questions from members wondering why our results didn’t match other folks’ conventional analyses.
We think the juice just isn’t worth the squeeze because this hasn’t been a significant enough problem to justify making the platform more complex for users. It’s an edge case without a significant long-term impact. We have to balance accuracy with keeping the platform useable for members.
What about trading with a lag between signal and execution?
Some members choose to wait for the allocation signaled in the evening based on the official close and then execute trades the following day. It’s a simpler, less stressful approach to executing strategies. In a perfect world, we don’t think it’s the optimal approach, but we also know that it hasn’t had a significant impact over the long-term. See this deep dive.
This is the cause of the discrepancy in GTAA Agg. 6 (and Meta) in November:
We were inspired to write this deep dive because we were bitten by this issue on Faber’s GTAA Agg. 6 on 11/30/2021.
There was a 0.06% difference between the last real-time price and the official close for one ETF that caused 17% of the portfolio to move from cash (based on the last real-time price) to TLT (based on the official close). Agg. 6 was also held by our Meta Strategy, so it had a knock-on effect there too, causing a < 2% shift.
We wrote this post to give more context on not just that discrepancy, but also (rare, but inevitable) future ones.
New here?
We invite you to become a member for about a $1 a day, or take our platform for a test drive with a free membership. Put the industry’s best tactical asset allocation strategies to the test, combine them into your own custom portfolio, and follow them in real-time. Learn more about what we do.
