Update - Indicator and Strategy Evaluation

I just finished reading "Systems Trading for Spread Betting: An End-To-End Guide for Developing Spread Betting Systems" by Gary Ford and I wanted to take this opportunity to update my methodology for evaluating trading systems.  The book was an easy read with plenty of illustrations of how to use NinjaTrader.  Overall, I would say it was a good introduction to system evaluation, but more research is necessary to further develop my methodology.

The system evaluation methodology is broken up into three sections that include system setup, optimization, and evaluation.  Below is the list of steps that should be taken when evaluating the potential and robustness of a trading strategy:

System Setup:

• Select a basket of at least five different contracts that have a low correlation to use as trading instruments. Each instrument should be very liquid, have a small spread, and have enough volatility to generate a profit regularly.  It would be best if the contracts were from different sectors.  Any basic strategy or indicator should work on this basket of products.  Correlation testing will have to be done using Excel and can be performed on end of day data, even though we will be day trading.
• Obtain the tick level historical data for each contract going back to 2007.  This will insure that we are covering all major market behaviors (range bound, bull market, and bear market).
• Determine the average volume for each contract over the past few months on a 10 minute time frame.  Identify the times when the volume is at least 30 contracts a minute.  Use this information to determine the trading times for each contract. Periodically re-evaluate the volume analysis.
• Determine your trading time frame by deciding how often you want to  receive trading signals each day. If you want signals once a day, try starting with an hourly time frame.  If you want signals once an hour, try starting with a 5 minute time frame.  If you want signals once every 10 minutes, try starting with a tick chart.  This will just be a starting point for the analysis of your strategy.
• Determine your basic entry methodology.  Make sure that entries occur during times when the volume of the contract meets your trading requirements.  Consider avoiding major economic announcements and entering right before the market closes.
• Determine your basic exit methodology.  Will you be exiting the trade at the end of the day just before the market closes?  Make sure that the exits occur during times when the volume of the contract meets your trading requirements.  Consider exiting just before major economic announcements.
• Add a basic filter to help reduce the number of bad entry signals and whipsaws.
• Use the NinjaTrader Strategy Wizard to create the basic strategy.  Create input parameters for the different aspects of the strategy that you would like to optimize such as indicator period, stop loss size, profit target size, etc.  For a fully symmetrical system (always in the market long or short), do not plot your exit orders on the chart.
• Use the NinjaTrader Strategy Analyzer to backtest the data on a long duration time span (>250 trades).  This time span is used extensively during research and optimization phases of strategy development.  Pay close attention to the total net profit, percentage profitable, profit factor, Sharpe Ratio, average maximum excursion, and maximum drawdown.  Use the chart tab to evaluate if the trade is entering and exiting orders correctly.  Use the cumulative profit graph to evaluate the equity curve.  Use the drawdown graph to evaluate the severity and duration of strategy drawdowns.  Use the daily net profit graph to determine the consistency of daily returns.  Use the distribution graph to evaluate the the profitability of the strategy by time of day or day of the week.  Use the trades tab to export all the trade data for analysis in Excel.

System Optimization:

• Use the NinjaTrader Strategy Analyzer to optimize the strategy over the long duration backtest period. Optimization metrics should include maximum profit factor (preferred), maximum win/loss ratio, maximum average profit, minimum drawdown, and maximum net profit.
• Optimize strategy inputs until the parameter results are in the middle of the optimization range specified for each input.  Perform an optimization of the strategy inputs for each instrument.  Calculate the mean, median, and mode of each optimized input parameter for all instruments reducing the input parameter combinations down to a set of three.  Once again, backtest the three sets (mean, median, mode) of input parameters on the entire basket of instruments to determine which set is best.
• Repeat previous optimization procedure for many different time frames including ticks, minutes, and hours.
• Perform a walk forward optimization beginning with the oldest data in the long duration backtest period and moving forward up to the beginning of the out of sample backtest period.

System Evaluation:

• Use the NinjaTrader Strategy Analyzer to backtest the data on specific market condition time frames such as range bound, bull, or bear markets.  Use the date ranges in the Strategy Analyzer to specify important periods.
• Test the strategy using the simulated dynamic data feed to evaluate its performance on range bound markets, up trends, and downtrends.
• Use the NinjaTrader Strategy Analyzer to backtest the data on the out of sample period.  This time span is used to validate if the strategy performs consistently over the backtest periods.  The out of sample backtest is performed on the most recent historical data.
• Export trade backtest results for all instruments to Excel and perform analysis of equity curve to determine position sizing (% risk per trade), spread/slippage cushion, initial capital account size, and money management.  Compare equity curve analysis results to the same equity curve analysis results for a buy-and-hold strategy on SPY (S&P500).
• Monte Carlo analysis (more research required)
• Bootstrap analysis (more research required)
• Use the NinjaTrader Market Replay to backtest the strategy on a dynamic data feed to help evaluate entries and exits.
• Simulate the strategy on a real-time live data feed using a demo account to understand if a strategy is tradeable, trade signals are generated correctly, and if slippage/commissions seem reasonable.  Once you are consistently profitable for four weeks in a row with 4 out of 5 days showing a profit, and only hitting your max daily loss once a week, then you can begin thinking about trading the system live with real equity.
• Stochastic process control theory analysis to determine when to stop using the strategy (more research required)
• Trade system live with real money and monitor performance.

So I was able to add a lot more detail to the system evaluation procedure, but there is still more analysis that needs to be completed.  Specifically, I have ordered some more books on walk-forward, Monte Carlo, and bootstrap analyses.  The methodology will continue to constantly evolve, but I have a good foundation here to begin testing systems.  It will take a lot of work/time, but this will ensure that I am confident in the final product and that I am doing everything I can to manage my risk.

Trading Thoughts - When Bad Things Happen

I have done a lot of research on trading that has included all sorts of topics including indicators, analysis, risk management, psychology, etc.  Something I have rarely seen addressed though are the actual technical problems involved with trading full-time from home.  Things may be different if you work at an established company because if something does go wrong, there are usually measures in place to fix things quickly or prevent bad things from happening.

When I say bad things, I am referring to the technical aspects of trading not involved with the strategy itself.  I am referring to a plan of action if something breaks or the power goes out.  Below is a list of bad things that could happen:

• Computer hard drive crashes (no cpu for days = no money made in the market)
• Monitors fail
• Mouse breaks
• Virus attacks your computer and freezes the cpu
• Trading software strategy is out of sync with actual account positions
• Microsoft Windows update restarts your computer without your knowledge, shutting down your automated trading system
• Power to your home goes out
• Cable internet connection goes down
• DSL internet connection goes down
• Home burns down or is flooded (off-site backup of strategies)
• Brokerage server connection goes down
• Insane market behavior and flash crashes
• Futures market reaches limit move
• Brokerage company unexpectedly closes or goes bankrupt (ex. MF Global)

I am sure there or many more problems that can occur.  At some point, I need to develop emergency action plans to prevent or account for these scenarios.  I can't imagine how terrible it would be if some of these things occurred and I was not prepared.  Are you prepared?

Trading Thoughts - Trading vs. Planning

I was speaking to my fiance yesterday and she asked me how my trading was going.  I told her that I was pretty productive this weekend and I made a lot of progress towards getting organized and developing my trading plans.  She responded by saying that it seems I spend a majority of my time planning instead of trading.  This comment really made me think.  Should traders spend more of their time trading or planning?

Now I want to make it clear that when I say planning, I am not referring to a trading plan.  A trading plan is critical and you should not be trading live without one.  When I say planning, I mean scheduling out tasks and procedures such as what you are going to do for the day, how you are going to backtest an idea, what books you are going to read, etc.  For instance, I spent time this past weekend revising my daily trading schedule.  I needed to update it in order to make sure I was getting the most out of each day.  I also want to create a weekly schedule because there are so many trading related activities that I cannot fit all of them in everyday.  So I need to put time to creating a schedule to ensure that each week I am accomplishing all my objectives and covering all the areas of my trading activities.  This is what I am referring to when I say planning.  This task would be so much easier if I had a boss or coach devoted to scheduling my time in order to maximize my performance.  Since there is no such person, the responsibility is solely mine.

Trading vs planning is a difficult argument because there seems to be a balance between the two.  If you look at the extremes, neither option is good.  You would never want to spend all your time planning without trading because you can become lost in the search for the holy grail as well as become somewhat disconnected from the live, psychological action of the market.  On the flip side, it would be a terrible idea (and costly) to spend all your time trading without a plan because your actions would most likely be dominated by emotions and plus you would have no way of evaluating your performance against a specified benchmark or plan.  Therefore there must be a balance between the two.

This topic is probably less difficult for those lucky individuals who are trading full time.  Unfortunately for me, I  have a full-time job in engineering which takes up most of my week.  That leaves me with only the morning, evenings, and weekends to work on my trading.  There is little room for error because my free time is at a minimum.

Personally, I lean towards mechanical system trading as opposed to discretionary trading.  I believe that a majority of my time is well spent researching and testing methods to apply to the live market.  At the same time, most of my ideas come from either trying to trade the market are watching the live market throughout the day.  I do believe in performance evaluation, so i think it is critical to have some guidelines.  I don't think you can strictly evaluate performance by looking at your PnL statement for the day because there are many other factors that determine if you traded well.  I also believe in having standard procedures for researching topics, so time must be spent on these procedures.  Once they are in place, it makes researching a more efficient process and ultimately save you time.  One final note, I believe a plan is critical because it lays out a path to success.  In trading, one day I feel like i am gonna be a millionaire, the next day I feel like I am back at square one.  With a plan in place, I can feel a since of accomplishment regardless of if the outcome is profitable or not.  I believe that goes a long way to preventing burn out and hopelessness.

While planning is necessary, it is important to remember what you are planning for and that is why trading (whether sim or live) is important.  It would be like training for a race that you never run, it would be very easy to lose sight of your goal.  This is why it is necessary to also get as much screen time as possible.  It helps you familiarize yourself with your trading products.  It will also provide countless ideas for trading indicators and systems, which provides more ammo for your planning.

I guess what it all boils down to is loving the process.  Whether it is trading or planning, as long as you love what your are doing the rest doesn't really matter much.  When it becomes a chore or a burden, then something needs to change.  There should be something besides money that drives you to pursue a path to successful trading.

Trading Thoughts - Indicator and Strategy Evaluation

In an attempt to become more organized and more efficient, I have realized I need to determine a procedure for evaluating trading indicators and strategies.  The procedure should be a simple, repeatable, and robust process with clear milestones that determine whether further testing is warranted.

Unfortunately, there seem to be a countless number of ways to evaluate a trading system.  Thankfully, there is a whole niche of books devoted to trading system development and evaluation.  I have made a list of the books and will begin reading them this year with the goal of refining my personal system evaluation procedures and development processes.  I recently purchased "Systems Trading for Spread Betting: An End-To-End Guide for Developing Spread Betting Systems" by Gary Ford.  This seems like a good starter book because it walks through the fundamental steps of creating and evaluating a trading system.  Currently, I am not interested in spread betting, but I am always looking to learn something new.  The real bonus is that the author uses NinjaTrader for the system development and evaluation, which should significantly improve my platform skills.

I will eventually develop an entire post/plan for my evaluation procedures and add it to the side bar, but for now this is more of a kick-off post to highlight the various areas of system evaluation.  From what I have seen, below is a list of topics that my system evaluation procedure should address:

• Time frame analysis (ticks, minutes, hours, days, etc.)
• Day trading time analysis (what hours during the trading session)
• Contributions of each aspect of the indicator/strategy
• Entries and Exits
• Filters for volume and volatility
• Backtesting
• Risk management and position sizing
• Parameter optimization
• Walk Forward optimization
• Monte Carlo simulation
• Bootstrap analysis
• Stochastic process control theory analysis
• Simulated strategy trading
• Live trading

This is just an initial list of topics to be investigated.  Hopefully, once I have read a few books on the subject I will have the basic steps of my procedure in place and I can begin evaluating systems on this website.  My initial goal would be to evaluate at least one indicator/strategy a week.  The primary goal would be to familiarize myself with each procedural component to better understand its strengths and weaknesses.  Once that is accomplished, I can then focus on determining the true merits of a sample trading system.