BACKTESTING RESULTS TABLE
By clicking the following links, you acknowledge you have read the disclaimer message above in its entirety and agree with it. Click HERE (or pdf version here) to see my backtesting results for my trading setups based on the weekly Commitments of Traders reports issued by the Commodity Futures Trading Commission. Be sure to click the tab for the explanatory notes.
Subscribe to:
Post Comments (Atom)
50 comments:
Hello.. I have only just found your blog and its great. Thank you for your efforts.
With regard to currencies, you only show CAD, YEN, GBP and the Dollar Index. How come you don't include the other currencies in the COT report i.e. CHF, EUR, MXD, NZD & AUD?
Hi Deirdre,
Thanks for your question. They're on my to-do list. I just haven't gotten to some of those yet, but should do so in the coming weeks.
Regards,
Alex
Hi Grant,
Thanks for your message. I don't believe there is a heating oil ETF. I regularly check Don Vialoux's regularly updated ETFs list at: DVTechTalk.com. (See "Special Reports.")
Regards,
Alex
Alex-
Great stuff. Thanks for your efforts and for sharing. A couple of questions for you: 1) what is your set up for the DJIA, it is not in the latest signals notes; 2) What is the difference between the set ups for the Nasdaq and Nasdaq-100. If I am reading your Latest Signals notes right they are both based on the Nas-100 dataset but giving different signal...? Thanks again.
Brian
Hi Brian,
Thanks for your comment. I haven't published those parameter values, but the main difference with the setups for the NASDAQ 100 data is the composite setup fades the small traders while the NASDAQ 100 setup fades the large specs. That's due to the testing results.
Regards,
Alex
Hello, Alex.
It looks very reasonable when your signals for SP500, NASDQ turn to bearish. However, I can not believe Dow should be bullish.
Also for HUI, and Gold 30, your signals are bearish. It seems not correct. Sorry to disappoint you.
Sincerely
GY Zhao
Hi Gy Zhao,
Thanks for your comment. While day-to-day action in the markets may lead you to think a signal is wrong today, tomorrow's action might make you think it's correct. The point is this isn't a short-term system that tries to catch minor fluctuations, but rather high-probability, longer-term moves. And of course, as I've pointed out elsewhere on this blog, not every signal is right. That's just not possible in mechanical trading. If you're looking for 100% accuracy, see if God will tell you what the markets will do.
Take care,
Alex
Hello, Alex:
SP, Nasdq, and Dow were bullish, they were dropping like rock;
Gold has been bearish, It has been going up like volcano erupts;
Thus you realized that some thing was wrong, then you modified SP and Nasdaq. However, Dow and Gold are still wrong. If people used your indicators to invest, they will loss a lot. Thus I suggest you modify your system further and get better results.
Sorry to criticize your system, but I think my suggestion is for you to get a usable system.
Sincerely
GY Zhao
Hi Guo,
Thanks for your comment. I've been going through a process of updating and revising all my trading setups. It has nothing to do with whether their current signal is correct or not. Rather, it's an important exercize to do from time to time with mechanical trading systems.
As I've mentioned elsewhere, no mechanical or any other kind of trading system gets it right every time. In fact, some of the most famous trading systems out there have a win ratio that's as low as 50 percent, believe it or not.
How do they make money? First, by having larger wins than losses on average, over the long run. And secondly, by good risk management. That's true no matter what kind of system you use.
That said, you have to find a system that's suited to your needs and personality. This one suits me just fine, though I understand if it's not for you.
All the best,
Alex
Alex-
How do you determine the "optimal" set-up for your COTS signal. I created a program for running all permutations of the input variables -- upper and lower standard deviations and moving average period--in order to find the inputs that maximize the historic return of the set-up and have found set-ups that would have produced higher total returns than th set-up you are using. For example for the S&P 500, I modified your set-up by fading the small traders and selling the S&P 500 short when the small trader net position is 0.55 standard deviations or more above its 21-week moving average, if the 50-week price moving average is sloping downward. And the setup goes long when the net position is 1.55 or more standard deviations below its 21-week moving average, but only if the price moving average is sloped up. Like your setup I assumed a cash position if both conditions aren't met. This set-up produced a return of over 1200% with 31 total trades (23 long/16 winning, and 8 short/6 winning). Is there something about my set-up that is statistically less robust than the set-up you are using?
Thanks,
Brian.
Hi Brian,
Firstly, let me applaud you for developing your program and finding a very interesting setup. Unfortunately, my tests show it returned a 260-percent profit, compared to 471 percent for the best setup I've found so far (detailed on the table above).
More importantly in my eyes, it doesn't score as well with the Student's t-test (83 percent confidence for beating the index, compared to 93 percent for my setup). Profitability confidence is quite good - 99.8 percent - but still lower than my setup's - 99.98 percent. On other tests like MAR, the Sharpe ratio, the one I'm using was also better.
That's not to say there aren't better setups lurking out there somewhere. I encourage you to look!
All the best,
Alex
Hi again Brian,
Oops, should have said the return from my best SP500 setup was 482% (not 471%). That's the number in the table up above minus 100 because the profit results in that table all start from a baseline of 100.
Regards,
Alex
Hi Alex,
I am trying to understand whether the COTS traders ie small traders and commercials both failed to predict the run up for gold lately Ie from around 700 to 900, the COTS setup seems to be bearish for gold, but the rise has been almost 30% or more from that price range/bearish signal.
Do you have any feel of what the insiders might not know re this ? If both the dumb and smart traders are wrong on a call, the COTS data and derived signals also will end up wrong. What are your thoughts to correct this - is it possible to develop a trading setup for such a pattern ?
Hi Ard,
Thanks for your comment. A few points: the gold large specs and small traders tend to be the "dumb money," in the sense that they are usually positioned the wrong way at market extremes. So the fact that the small traders didn't predict the recent gold rise shouldn't be a surprise. That's why the best setup I've seen so far for gold stocks is trading opposite to the small traders.
As for the commercial traders, their bearish signal in my gold setup from the Sept. 25 COTs report would now be down 20 percent (not 30 percent) if it hadn't been stopped out a couple of weeks ago when the stop was hit at the largest historic drawdown level (12 percent).
As you say, the commercials are usually correctly positioned in gold, so what went wrong this time? I'm not sure. But I do know the fact that the price has gone so far against their signal suggests gold's current rise is an extraordinary event outside statistical norms in the data seen since 1995. Could it mean a blow-off of unpredictable size? A sudden blow-up? I suspect we'll know only after the fact.
Take care,
Alex
Alex,
Thanks for providing your work for us all to see. It is very interesting. One question I have is how do you determine who to watch for each commodity (Commerical, Large or Small) since it varies in your excel per listing?
Hi Jin,
That's all based on the backtesting and other tests of statistical robustness to see which group of traders provided the best signals.
Regards,
Alex
Alex,
Does the group that you watch change over time or is it constant?
Jin
Hi Jin,
Once I've chosen the setup that seems best for a market, I stick with it generally until my next update. The plan is to update all of them on at least a yearly basis or so, and that's a process I'm going through right now.
Regards,
Alex
Alex,
If it is not secret, I would like to see a sample work spreadsheet that you have mentioned that you are pasting the data in it.
Thank you for the idea and your work.
Hi Bora,
Thanks for your message. Please refer to my "S&P 500 Spreadsheet and DIY Guide" page to download a sample spreadsheet.
Regards,
Alex
Hi,
I started reading your blog since last week. What will be the grain hitting the top ?
Aaron.
Hi,
I started reading your blog since last week.
What will the grain hitting the top ?
Aaron.
No idea, Aaron. Sorry,
Alex
Alex,
Do you take into account the current psychology and change in social moods? The bullish signal in a tech heavy index seems counterintuitive to the change in social mood where more money must be spent on necessities and less on "stuff" and the emerging negative attitudes towards debt.
I am also curious about how quickly the signals change from one direction to another? I would guess that the change would be slow if you are following large traders but quick if you are fading small traders.
Good blog though. I subscribe to the feed but have not commented before. Look forward to your response.
Hi Pinalspic,
Thanks for your comment. Mood doesn't figure in the data, except in so far as it might affect trader positioning. For the duration of signals, one way to check is to see how many trades there were for a setup. Those with more trades would have shorter durations. Haven't compared the large specs to the small traders.
Regards,
Alex
Alex,
Thank you again for your previous reply and sharing the information. I have another question to understand the robustness of your system. What percentages do apply rising, falling and consolidating markets when you check your historical data.
My experience is any system needs to dynamic according to a parameter as a function of trendiness. Say if market consolidates last three months the system needs to go swing trading, say if there is a trend; system needs to create kind of trailing stop loss by allowing stay long for small corrections.
Do you have dynamic parameters in your systems changing as a function of trend?
You may say using SD is already creates a dynamic system however what I am asking is a kind of coeefficient as a multiplier in front of SD.
Thank you
Hi Bora,
That's a good question. No, my system doesn't seek to identify specific types of markets. The type of system you identify can be useful, but you sacrifice adaptability when market conditions change, as they can and do frequently, usually only visible in hindsight. The ideal system is sensitive to current conditions (e.g., bull, bear markets) but also based on enough different conditions that it remains robust when conditions change.
Take care,
Alex
What is the rationale for connecting the Comex (actually CME sold that business to NYSE Euronext, and new contracts will appear on the LIFFE) gold futures to HUI?
Eurodollar Futures have little to do with the Treasury Bill rate (and Treasury Bill futures have all but disappeared). The Eurodollar futures settle to LIBOR, which represents a spread off of the Fed Funds target rate that banks will lend to each other. I humbly suggest you simply rename this Eurodollars.
Can you explain the difference between the Bank Index signal and the Treasury Bill (Eurodollar) signal?
I hope you know that Russell granted ICE an exclusive license for futures, so you need to include the ICE COTS data for your system. While the volume remains small relative to the CME, no Russell products will appear on the CME after the September contract, so the trading will gradually shift to ICE.
Can you add the two year treasury future (it is FAR more important than the five year future)?
Why no Euro currency (but you have the CAD, GBP, and JPY)? It is by far the most important currency (after the U.S. dollar) in the world.
I would like to see at least the major ag futures (corn, soybeans, and wheat) while realizing that it would take more of your time.
Could you gather all of your (real time) signals in some place? Maybe you don't want to just give us all of your hard work, but at least putting your signals since you started writing the blog would prove interesting to analyze.
Thank you very much for sharing your work with us.
Hi Kiwiguy,
Thanks for your message. Here are some answers to your questions:
- The Euro hasn't existed long enough to get much statistical validity for a trading setup. Same applies for any other new futures data. Off the top of my head, without doing any calculations, my guess is we need at least 10 years of weekly data before it can reliably be traded. Ideally, more than that.
- I have published signals for several agricultural commodities, but recently removed them from my table temporarily because I wasn't satisfied with their limited trading robustness. I'm going to take another look with improved testing I'm doing now and will no doubt come up with some better ones which I can share. There's little point in my mind taking the time to publish signals that aren't useful.
- Publishing all my past signals would take a massive amount of time that I just don't have at this point. This isn't a paid service. I'm doing this on a voluntary basis after all!
- The Bank Index signal functions with different parameter values than the other setup based on the Eurodollars data.
- Why connect Eurodollars to the T-Bill? Simple: I discovered a statistically robust signal between the two, regardless of what the perception may be about their lack of correlation.
- Same for HUI and gold. However, I find the best signals from the COTs data come from a security that is most closely linked to the data. So the best way to trade the gold signal is with bullion itself, not a different setup that reacts to gold stock prices. The trading results in the table I provide show some of these differences in robustness. I studied the HUI and XGD data merely out of intellectual curiosity.
Regards,
Alex
Thank you for your reply. I have some follow up comments and questions. :)
I think we have about nine and half years of Euro futures trading now (getting pretty close to having enough data).
I thought you might have all of your past signals in a spreadsheet that you could just make available. I guess I can satisfy my curiosity by keeping track of them myself going forward.
"However, I find the best signals from the COTs data come from a security that is most closely linked to the data."
That is why I thought you should just rename the "treasury bill" signal Eurodollars, because it makes the most sense to use Eurodollar futures to trade that signal (much more convenient and cheaper than trying to buy, or especially short, treasury bills).
Thanks for the gold explanation. I think I confused myself (believing you used the less active CBOT, now CME, gold futures). THOSE are the products the CME sold to NYSE Euronext (they are keeping the more active Comex futures). The notes in your spreadsheet refers the HUI signal as coming from Comex gold, and the plain gold signal as coming from Nymex gold, when they are actually the same.
One more question - how do you handle "mini" contracts when multiple sizes for a index or commodity exist? I know the equity indexes (most famously the S&P 500 and the Russell have them, but Dow futures trade in THREE different sizes, crude oil and natural gas have cash settled minis, and CBOT gold and silver have them as well).
Any thoughts on how you will handle the changeover of Russell futures to the ICE? Some small traders might not want to add another exchange, and simply stop trading the product. I believe it will definitely change the market participants, and thus make using historical open interest statistically suspect.
One more question - how do you measure the performance of Fed Funds futures? There are several different contracts (the first eight or so are fairly liquid), and they don't all behave the same.
Hi Kiwiguy,
For the Fed Funds, I use the 30-day Fed Funds cash price, as provided by the CRB (symbol FF).
Regards,
Alex
Hi again Kiwiguy,
For the Euro, the cash data I would use starts Jan. 1, 2002, so that's not nearly enough for statistical reliability in my opinion.
Regarding the 13-week T-Bill setup, it's named that because the signals and data are specifically for the T-Bill. I would retest the data if I were to apply this signal to another market as you suggest.
Regarding your gold/HUI question, the point is I am using the COT gold data, which comes from CMX, but my gold price data is from NYMEX (CRB symbol GC). This is specified so people can get exactly the same datasets if they want to look at this more closely.
Regarding mini contracts, I haven't found the COT data to be as useful. Its signals are in general inferior in backtesting.
Regarding the changeover in Russell futures, I will continue to follow any market listed by the CFTC until my testing shows it's no longer of use. Regular updates for this kind of mechanical system are required every 1 1/2 years or so. As well, if the number of traders falls below 20 in a market, it isn't listed that week in the COT report. If that starts happening consistently in a market - like it did last year for the S&P 400 Midcap index - I would drop it.
Regards,
Alex
"I use the 30-day Fed Funds cash price, as provided by the CRB (symbol FF)."
Do you have a link for this? Sorry, but Fed Funds futures have a lot of unique qualities, and I just want to understand how your system handles them.
"For the Euro, the cash data I would use starts Jan. 1, 2002, so that's not nearly enough for statistical reliability in my opinion."
The Euro came into existence on January 1, 1999. Why would you start in 2002?
"Regarding the 13-week T-Bill setup, it's named that because the signals and data are specifically for the T-Bill."
I think I understand that now, too. You use the COT reports for Eurodollars to estimate subsequent T-Bill performance, correct?
"I am using the COT gold data, which comes from CMX, but my gold price data is from NYMEX (CRB symbol GC)."
Comex is (a part of) Nymex. They merged fourteen years ago, I believe. I did look at the raw COT reports, and I guess they still do keep a separate report for Comex (nothing but copper, gold, and silver), so now I understand your references to them.
"Regarding mini contracts, I haven't found the COT data to be as useful."
So you don't use mini contracts at all, even for the S&P (where the dollar value of contracts traded is much higher for the mini)?
"Regarding the changeover in Russell futures, I will continue to follow any market listed by the CFTC until my testing shows it's no longer of use."
I keep reading about people who say they will stop trading the Russell 2000 when CME's listing rights expire. I just think you will have a big discontinuity as far as the validity of your data.
Thank you very much for answering all of my questions.
Hi Kiwiguy,
Thanks for your message. Here are some answers and comments:
- No, I don't have a link for the 30-day Fed Funds cash prices. The CRB sells the data.
- Regarding the Euro, my apologies. My CRB data does have trading data for the Euro index since 2002, but it also has the Euro/USD cross since 1999, as you say. With less than 10 years of data, this is pretty borderline in terms of a sample size. Using a single group of traders, the maximum moving-average period would be 34 weeks in order for the setup to have any statistical validity. That's not so bad, but if I were to used a combination of two groups of traders (meaning more trading rules), the maximum moving-average period would be 23 weeks. As well, a smaller sample size tends to have fewer variety in terms of markets conditions, making the setup less statistically robust. It's not to say the setup wouldn't be interesting - I think it would be - but that my limited time is perhaps best spent on more fruitful trading opportunities. But I do assure you I'll get to the Euro eventually!
- Regarding mini contracts, the volume isn't relevant if the utility of the setups is inferior.
- Regarding the Russell, if the number of traders falls below the minimum threshold for reporting by the CFTC, I'll have little choice but to drop the setup. We'll see what happens.
Regards,
Alex
Regarding Fed Funds, most people don't understand the process very well.
The Federal Open Market Committee of the Federal Reserve Board establishes a "Target Rate" for overnight loans that banks make to each other (the well known "Fed Funds Rate"). The New York Federal Reserve Bank affects the "realized" rate through the purchase and sale of government securities.
There is no such thing as "thirty day" Fed Funds (all loans are overnight).
The CBOT futures (which do have "thirty day" in the title) settle to the average of the realized rate (not the target rate) over the calendar month (maybe the reason they call it "thirty day," even though it still isn't), as published each day by the New York Fed.
You can find both recent and historical data on the both the realized and target rates at:
http://newyorkfed.org/markets/omo/dmm/fedfundsdata.cfm
Is this the data you have? If not, could you post a few days worth, and I will try to match it to another source.
Usually, the Federal Reserve Board communicates short term (meaning the next meeting of the Open Market Committee) changes in monetary policy fairly well, so near month Fed Fund futures generally offer little chance of significant price movement, as the market expects the communicated outcome.
In my opinion, the Euro and the two year note both offer much greater trading opportunities.
Thanks again.
Alex,
How is the percent return calculated for the index? Are you using the first date of the data as day one and last signal as the end date? I am trying to understand how you get 255% profit for S&P 500 when on 3/21/95 SPX was at 495 and last signal day it was at 980.
Hi PCP,
The return figures are as of the date of the last update, which is in one of the last columns on that table. I've clarified the notes to explain that better.
Regards,
Alex
Hi Alex,
great blog! Thanks for sharing your thoughts with all.
Two questions, as I am new here.
1. The calculations in the latest S&P 500 sheet are based on other MA, 1st. signal with 16wMA and 2nd. signal with 10wMA, but in the notes for your "COTs Beat the Markets" file you are using 10wMA for 1st. setup and 5wMA for 2nd.
Is it correct to assume, that you are adjustinf the formulas in the file from time to time and the notes are not always revised.
2. The explanation for Gold setup describes two times the same: "The setup trades on the same side as the large speculators. It goes long when the large speculator net percentage-of-open-interest position is -1.2 standard deviations below its five-week moving average or above. It sells gold short when the net position is -1.2 or more standard deviations below its five-week moving average." So same for sell and buy. I suppose also, that you are traiding against Large Speculators and not with them.
Thanks for your reply!
Andreas
Hi Andreas,
Thanks for your comment. Thanks also for mentioning the mistake in those notes. The second setup does use the 16-week moving average, as you mentioned. I've corrected that. For the gold setup, I've also clarified the description.
The setup trades on the same side as the large speculators, as you noted. It goes long when the large speculator net percentage-of-open-interest position hits -1.2 standard deviations below its five-week moving average or higher. It sells gold short when the net position hits -1.2 or more standard deviations below its five-week moving average or lower.
Hope that's clearer.
Regards,
Alex
Hi Alex,
thanks for fast reply. My first question is clear, but I do still have problems with Gold.
1. Why do we trade with Large Speculators? I thought these are the dumb money and COmmercials are the smart money in Gold market.
2. I am still confuse why you use two times "below...". I would expect one time "above its five-week moving average or higher" and the second time "below its five-week moving average or lower".
And, why to use the minus sign in front of 1.2?
Maybe it would be easier just to post the formulas in the cells EB and EC.
Regards,
Andreas
Hi Andreas,
The large specs are not always the dumb money. That's a common misperception. Sometimes, they have the best signal. I'll see about posting the gold setup on the site too at some point. The minus signs shouldn't be there. You're correct.
How it works is this: I go long if the net position is -1.2 standard deviations in relation to the average or above; I go short if it hits -1.2 SDs or below. So if it hits -1.2 right on the nose, it switches from long to short or from short to long, depending on what the current signal is. Hope that's clearer.
Regards,
Alex
Alex,
Are the stops given in your backtesting results appropriate for 1x or 2x leveraged ETFs?
Thanks,
Eugene
Hi Eugene,
If I use a 200-percent leveraged fund, I keep the same stop but halve the maximum portfolio allocation in order to maintain the same risk level.
Regards,
Alex
Hi Alex,
I noticed that the footnote for the Nikkei setup in the backtesting results refers to gold futures and options. Is this a typo?
Thank you for your continuing work on the COTs setups.
Best Regards,
Eugene
Hi Eugene - thanks for letting me know. I've corrected that now.
Take care,
Alex
Hello Alex,
First thanks for sharing all yor knowledges.
I have a question regarding the 30Y set-up.
When you say: "It goes long when the total open interest hits 0.05 standard deviations above the moving average or lower."
Do you mean that the setup goes long when the signal is > 0.05 and in this case, I thing that higher instead of lower would be more appropriate, or is it the opposite ?
Thanks for your help
All the best
Nicolas
Hi Nicholas,
The description is correct as is. The signal goes long when the COT position is 0.05 standard deviations above the moving average or lower than 0.05 standard deviations above the moving average. This setup has strange entry and exit criteria that are kind of upside down.
Regards,
Alex
Hi Alex, I'm still working on my model to try to see if I can get one of these to work (S&P), but it currently is "bear" S&P (5th october) if I use that setup (12wk moving average, +0.2/-1.2, then the lag on top of that..). I wanted to check - is the Standard deviation a 12 period also (same as MA)? Is it a simple moving average?
The net % for commercials I'm using is "(commercial open interest long - commercial open interest short)/Total open interest"
Any thoughts would be great. Cheers, John
Hi John,
Thanks for your comment and interest in the COT data. The standard deviation period is the same as the moving average period (12 weeks). To obtain the net % commercial position, I subtract the value in the BA cell from the value in the AZ cell. Hope that helps.
Regards,
Alex
Post a Comment