Improve trading strategies through Walk Forward Analysis
por Fernando Monera
Traditional optimization of trading strategies
Optimization is the process to adapt the parameters of a given strategy to a specific market. It is generally a good thing when done properly, but if not done correctly, there is a high risk to do it wrong and end up with a curve fitted system (I will go deeper on the traditional optimization of trading strategies on future posts).
What is curve fitting?
You can always find a combination of rules and trading parameters that fits perfectly to the available historical data, resulting in exceptional trading results based on those tests. But when those rules are tested on a live market, they fail and loose money very quickly.
The majority of the strategies commercially available suffer from this problem. Why? Because vendors sell the strategy based on pretty tests instead of robustness of the strategy. It is much easier to sell based on a nice equity curve than to sell based on the complex optimization process to increase robustness and reduce curve fitting. Sad but true.
The proper way to optimize
To avoid curve fitting, one must leave at least 30% of the available data out of the optimization process. For example, if you have data from 2000 to 2012 (12 years), the optimization process would be:
1. Optimize for 2000 – 2009: You will end up with the best parameters on this period.
2. Select the best parameter set: The criteria to choose it is important. For example, selecting the best AbsoluteProfit/RelativeDrawdown that has robust enough neighbor parameters is a good choice.
3. Test the parameter set on the out of sample period (2010-2012): If you get different results on this period there is something wrong on previous phases. You must return to the design phase.
Once you have a parameter set that works correctly on the out of sample period, you would run your strategy live.
The problems of traditional optimization
Traditional optimization is good, but there are obvious problems:
> The chosen parameter set is average quality: As it needs to survive a lot of market conditions (both on in-sample and out-of-sample data) it will not be really adapted to any one, so many trading opportunities will be missed.
> Degradation forces periodic reoptimizations: As the time passes, the chosen parameter set degrades. We will have more data and we will have more different market conditions.
> Most of the gains could be focused on small periods: You will find many strategies out there cuve fitted to recent market conditions but when testing them on previous years they fail.
> A dramatic and definitive change in a market will make the strategy reach the worst case scenario, loosing a lot of money.
There are ways to solve most of the above problems in a traditional way. But using Walk Forward hopefully we should be able to overcome or reduce the impact of all of them.
Walk Forward Analysis
With Walk Forward Analysis, instead of making one big optimization on in-sample data and testing it on the out-of-sample data, we will make a lot of small optimizations and testings on much smaller periods.
You can read the full entry in the blog ‘Profitable Auto Trading‘.