|  JUL 1988

The value of optimization by Louis P. Lukac and B. Wade Brorsen

The value of optimization by Louis P. Lukac and B. Wade Brorsen Optimization based on historical data is used widely in the futures industry to determine the most profitable parameter sets for a trading system. The underlying premise is that if some parameter set worked well in the past, it should work well in the future. Simply picking the best parameter set from historical data and stating the accompanying profit level (called in-sample profits) is of no real value because you cannot see into the future to know what the best parameter set will be. Actual or blind simulation (out-of-sample) results are always much worse than in-sample results, which has led many to question the value of optimization. There seems to be little agreement on how much data should be used in optimizing parameters, how frequently to re-optimize or whether optimization is even of any value at all. Even the large public futures fund managers who rely heavily on technical analysis cannot agree. A survey of public futures fund managers by B. Wade Brorsen and Scott H. Irwin in The Review of Futures Markets in 1987 showed that the data used by advisors to select parameters ranged from all the available historical data to as little as two years worth. Their frequency of re-optimization ranged from twice a year to once every five years. Several advisors did not optimize at all, they simply used the same parameter across all commodities. Large amounts of time and money are devoted to finding the optimal parameters for a system. With more and more money being managed technically, and more software available for optimization, the need to test the validity or limits of optimization grows greater. An invalid framework could lead to invalid results and a false sense of security, not to mention the potential loss of money. We tested various re-optimization strategies to determine if optimization is of any value for predicting correct sets of parameters and to determine if one re-optimization strategy is better than another. Specifically, we statistically compared 10 different re-optimizations to each other and to a random selection strategy using two different trading systems.

by Louis P. Lukac and B. Wade Brorsen