BUILDING RELIABLE TRADING SYSTEMS

Discussion in 'Philosophy and Strategy' started by wanghu065, Apr 24, 2014.

  1. rok likes this.
  2. 刚刚在Amazon上看到一些评论 :)

    Recommends poor and inappropriate development techniques, August 21, 2013
    By Dr. Howard B. Bandy

    In a few words -- worse than wrong.

    Chapter 1

    He gives examples of the returns and drawdowns of several money management organizations.

    Chapter 2

    His discussion of the standard error (sic) of samples of trades drawn from an in-sample backtest. Here we realize that he is treating the entire history of all stocks and commodity futures as if they have identical characteristics and are stationary for their entire lifetime.

    It is also the description of the BRAC methodology he recommends:
    Build, Rebuild, Compare.

    Beginning of methodology

    Build -- Using all available price history -- 20 years or more -- optimize the rules and parameters of the system. The objective function is maximum profit.
    Rebuild -- Using all but the most recent year of price history, repeat the optimization of the rules and parameters.
    Compare -- Compare the "best" system parameters between the two optimizations. Also compare the profit per trade between the two runs.

    From page 20: "If performance is similar, you can have pretty good confidence that your strategy will hold up in real trading."

    End of methodology.

    You understand correctly. If the optimized results from years 1975 through 2013 are about the same as those from years 1975 through 2012, you can have "pretty good confidence."

    All 37 commodities are treated as having identical characteristics. Profit per unit contract is treated as equivalent regardless of being wheat, platinum, TBond, S&P.

    The system optimization schedule is in three steps, performed in sequence:
    1. Determine the optimum parameter for the entry and exit rules -- say moving average lengths (he illustrates using a moving average crossover system).
    2. With the moving average lengths fixed, determine the best length for a trend filter based on a long-period moving average.
    3. With both sets fixed, determine the best level for a maximum loss exit.

    He regularly shows tables with rows for parameter values and columns for results. The parameter chosen is often at one of the ends of the table with no values beyond it.

    The results after all three passes for all 37 commodities for all price history is 6,429 trades, 2103 winners and 4326 losers.

    He suggests we will be surprised that the results without the final year are so similar to those of the entire history, and that we will take that as confirming our confidence in the system.

    In a (different) example of a breakout system run over the entire history of 37 commodities, 31% of trades were profitable, with an average profit of $33.85 (no mention of allowance for slippage or commission), and a standard deviation of $2,316.

    He proudly claims there is no out-of-sample testing. His criticism of out-of-sample testing shows that he does not understand proper testing procedures, or the unique characteristics of financial time series data.

    Other criticisms of other and better techniques are vacuous.

    Chpater 3

    After basing his earlier analysis on "every market acts the same," he states "markets are different."

    For one test he assigns stocks to one of two groups -- weak and strong. There is heavy selection bias. No criteria are given. He tests 2000 to date, acknowledging survivorship bias, but not adjusting for it, then stating "this is not considered a major flaw in the study."

    Chapter 4

    Tests of entries using traditional indicators -- moving average crossover, RSI, stochastics, etc -- with traditional lookback period.

    Chapter 5

    Tests of exits. He uses inappropriate settings for timed exits and profit target exits.

    Chapter 6

    Filters. Long lookback moving average, volatility, day-of-week.

    Chapter 7

    Money Management. He puts position size into the model code, fixes it one time, and assume it will remain at that value forever.

    Chapter 8

    Bar scoring. Divide the indicator values into bins. Examination of profit bin-by-bin.
    Bar Patterns.

    Chapter 10

    A very disorganized description of the use of Monte Carlo simulation. He confuses discrete variable with continuous variables and draws inappropriate results.

    Chapters 11 through 15

    More on money management. Fixed fraction, Fixed ratio.

    In summary --

    I will be presenting at the IFTA Conference in San Francisco in October. I have several slides outlining good techniques to use and poor techniques to avoid using. My Fitschen is using all of the techniques I recommend against and none of those I recommend favorably. (No flames, please)

    Save your money. Those people who wrote glowing reviews either did not open the book, do not understand trading system development themselves, or are really good friends of Mr. Fitschen.
     
  3. 粗略读了一下这本书,基本同意上面的评论。个人建议还是别在这本书上浪费时间了。
     
  4. 这评论写的还真详细。看来不用买了。