How to Correctly BackTest, Collect Data, and Analyze A System | Forex Factory

Options

Permalinks
Print Thread

Similar Threads

Sail the Grid - System without the need to analyze the chart 292 replies

NFP System - Backtest data provided 385 replies

How to find and develop your system correctly 11 replies

Visual backtest result different from non-visual backtest 0 replies

You think you are good enough to analyze what system is this?? 7 replies

Attachments: How to Correctly BackTest, Collect Data, and Analyze A System
Exit Attachments

How to Correctly BackTest, Collect Data, and Analyze A System

  • #1
  • First Post: Edited 8:20pm Dec 30, 2016 8:06pm | Edited 8:20pm
Hello all,

I hope someone can help guide me in the right direction. And hope this help others as well. After losing about $7000 over the past two years, I research and found, I need to:

1. Define a trading system(s) for myself. I like resistance and support breakout trading.

2. Correctly manually backtest of +100 trades for each strategy idea.

3. collect the right data in spreadsheet. I welcome any spreadsheets please to use.

4. If and only if the backtesting reveals a profitable system from +100 trades, manually forward test with paper for +100 trades. Else go back to step 1

5. If and only if the system is profitable program the system(s) if profitable and automated backtest for +10 years of historical data using a platform. If profitable, proceed, else go back to step 1.

I have 3 breakout strategies I want to test.

My biggest problem is I don't know how to collect data to determine optimal stop loss (I prefer not losing more then $150 per trade), entry, and target (I prefer R:R 1 to 3).

Is this enough data to collect for data analysis?

Data Collection in spreadsheet in columns:
1. Time of Trade
2. Price
3. Long or Short
4. Contract Size
5. Win
6. Loss
7. MAE
8. MFE
9. Profit made if not break-even after up $150. No breakeven testing.
10. Max stop loss that could have prevent a loss for fake-outs.
11. Profit when trailing from previous low/high to catch a higher R:R

Thank you all. Really appreciate the help.
  • #2
  • Dec 31, 2016 5:25am Dec 31, 2016 5:25am
chanced upon your thread as I was doing a search on how to programmatically run backtesting on mt4. I am learning myself, but am currently at the break out moment.

1) REDUCE your trade size to a minimum. gamble in cents or a few dollars until you r profitable. why lose 7k when u can lose just 70bucks trying out the same trades

2) tip for determining SL is to draw your lines. trendlines, outer/inner, diagonal/horizontal. trade breakouts bouncing off these levels, set SLs just below them. If you can't plot these, start with ATR as a stop loss OR use something like daily pivots. when trading breakout retests, I typically set tight stops just below the breakout point. Often, I find.. starting off with identifying my tgt price.. then working backwards to determine my SL, finally arriving at a reasonable R:R, is better starting point than finding my SL first. If my tgt makes my R:R crappy, no trade

3) if you are trading on a discretionary basis via breakouts of trendlines, esp diagonal ones, you will know that it is mission impossible to program these (backtest). lay out your gambles in real-time. find a proper non-commercial mentor to follow (eg tradeciety.com). there is very little to gain switching back-n-forth in strategy A-Z.. I experienced this in my initial years. In the end u will realise all indicators r lagging. just watching price is enough once u understand what an indicator does. All that matters is the macrotrend, swing highs/lows, measured moves, where r u relative to daily pivots/ weekly/monthly open

good luck!
  • #3
  • Jan 2, 2017 11:31am Jan 2, 2017 11:31am
Backtesting to find one consistent setup that works is often a myth. Your missing too much info just looking at the chart. Such as is there a KEY news event everybody waiting on, A vote, A central banker speech. Are we in risk off and stocks are puking? Very hard to say what was gong on 45 days ago.

The best way to backtest is around key times of day.. London open, usa open, london close. But still difficult without all the info I mentioned above.

Part of being a trader is recognizing what the whole market is doing, not just the pair your trading.

If Euro is gaining on everything E/G E/U E/A E/CAD E/CHF All very strong. I'm less likly to take a short setup on E/u chart that day.

Backtesting your just missing too much info about rest of the market.
The only system that will work is one designed by and for yourself.
  • #4
  • Jan 2, 2017 12:48pm Jan 2, 2017 12:48pm
Quoting bloodpoodle
Backtesting to find one consistent setup that works is often a myth. Your missing too much info just looking at the chart. Such as is there a KEY news event everybody waiting on, A vote, A central banker speech. Are we in risk off and stocks are puking? Very hard to say what was gong on 45 days ago. The best way to backtest is around key times of day.. London open, usa open, london close. But still difficult without all the info I mentioned above. Part of being a trader is recognizing what the whole market is doing, not just the pair your trading....
Of the same persuasion and as one-&-one = 2, figure-it-out-only-on-a-must-know-basis programmer I came to the conclusion that there just too many discretionary trading variables to consider and besides the time MT4 backtesting too simplistic for reliable results.

However, to each his own, who may very well find merit in backtesting.
TSLA: 30-Dec-22 123.18 // 16-Jun-23 260.54 **111.51%**
  • #5
  • Jan 2, 2017 2:46pm Jan 2, 2017 2:46pm
Back testing and gathering data is all about asking the "right" questions. Those questions must come from you and how you perceive your trading to be or will be. There are many many questions and they are only limited by your imagination. I strongly applaud your effort to collect and analyze data on your trading as I believe it is the ONLY way to become consistently successful in the long run. You must also keep in mind that your data collection will never stop. You will always have new questions about your trading that your current data does not answer, and thus you will likely start new collection procedures to answer the new questions...and on it will go until you are a bazzillionaire.

First step would be: Learn to use a spread sheet to the best of your ability. Keeping track of data manually will become cumbersome and slow. Learn to use all those nerdy commands and formulaic expressions...they will pay off in the long run. Not only that, if it takes you awhile to find your stride in trading, you can contract out your excel services to the more lazy and make good money.

In my beginning, I decided to use the process called "scientific" process because I felt it worked the best. That process is as follows:

  1. Make an observation - this can be as complex or simple as you choose. I suggest making it as simple as possible.
  2. Ask the right questions - some examples:

  1. How often does this observation appear?
  2. What time frame is it seen the clearest?
  3. What percent of time does this observation create a chance for profit?
  4. If it looses, on average, how much does it lose?
  5. What happens immediately before and after this observation? Is it consistently the same?
  6. etc...etc...keep thinking

  1. Formulate a hypotheses - from your observations and the questions you ask about it, you now have the basis to formulate a hypotheses about your idea. WRITE IT DOWN! Take a little book and write down your observation and questions, and then write down your hypotheses. This simple action helps cement the concept in your mind and your reasons for doing what you do, the way you do it. That later becomes the basis for your disciplined trading action.
  2. Gather Data - NOW you have the basis to start gathering data that will become useful and meaningful. Based on the questions you ask yourself, you now know what data to start collecting. In addition to any of the above questions you might also want to collect data on the standard stuff everyone here asks about (but really doesn't understand). Time in trade; Average profit/loss; expectancy of signal; blah, blah, blah.

Once you have gathered a large enough sample of data you will have something to analyze and chew into. How big should the sample be? What ever you think is significant...remember you are looking at a HUGE 24 market that exchanges trillions of dollars every DAY. The standard "100 trades" is a good place to start of course, but you will soon find it to be inadequate. However it will get you started on the right path.

Again, sooner or later you will realize that you have more questions than answers about your observations, so make your data collection efforts expandable to include things you haven't thought of yet. Remember this is the start of a journey that has no end...so settle in and do it right from the start. This all by itself will save you hours of time starting and stopping because you didn't feel like doing it right or asked someone else to do it for you.

I am happy to help if I can

@xxxx§|[::::::::::::::::>
  • #6
  • Edited Jan 3, 2017 2:15am Jan 2, 2017 9:50pm | Edited Jan 3, 2017 2:15am
Excellent post above and apperars to be on target re OP objectives.

Came across this while researching c++ code for Sierra Chart - a multi broker charting and trading platform. Not quite what the OP is looking for but may be of interest to passersby.

Note:
I use Sierra chart as a trading and charting platform.
I have no financial connection with Sierra chart other than sub charges.
I have no financial connection with Prosoftware.

10 day trial available.

Inserted Video
http://www.profsoftware.com/scbt/home.htm
Attached Image
http://www.tradinggeeks.net/2014/05/...-excel-part-i/
TSLA: 30-Dec-22 123.18 // 16-Jun-23 260.54 **111.51%**
  • #7
  • Jan 4, 2017 10:01am Jan 4, 2017 10:01am
I was going through some records for last year and came across this power point from my mentor. I called and asked him if it was OK if I shared this power point and said it was fine. I think this will answer many question on how to go about exactly the goal you are looking for.

Good luck
Attached File(s)
File Type: pptx Trend Riding or Targeted Profit Taking.pptx   144 KB | 1,290 downloads
@xxxx§|[::::::::::::::::>
  • #8
  • Jan 21, 2017 11:58pm Jan 21, 2017 11:58pm
Quoting DonPato
Back testing and gathering data is all about asking the "right" questions. Those questions must come from you and how you perceive your trading to be or will be. There are many many questions and they are only limited by your imagination. I strongly applaud your effort to collect and analyze data on your trading as I believe it is the ONLY way to become consistently successful in the long run. You must also keep in mind that your data collection will never stop. You will always have new questions about your trading that your current data does not...
Thank you so much for your response. I will read in detail and respond if questions or comments. I am just now responding cause I been working alot lately.

Thank you
  • #9
  • Jan 22, 2017 12:03am Jan 22, 2017 12:03am
Quoting econbizer
chanced upon your thread as I was doing a search on how to programmatically run backtesting on mt4. I am learning myself, but am currently at the break out moment. 1) REDUCE your trade size to a minimum. gamble in cents or a few dollars until you r profitable. why lose 7k when u can lose just 70bucks trying out the same trades 2) tip for determining SL is to draw your lines. trendlines, outer/inner, diagonal/horizontal. trade breakouts bouncing off these levels, set SLs just below them. If you can't plot these, start with ATR as a stop loss OR use...
econbizer,

Apologize for the late response,

I really appreciate your response and it really helped me with my planning.
Thank you so much.
  • #10
  • Jan 23, 2017 11:38am Jan 23, 2017 11:38am
Quoting econbizer
chanced upon your thread as I was doing a search on how to programmatically run backtesting on mt4. I am learning myself, but am currently at the break out moment. 1) REDUCE your trade size to a minimum. gamble in cents or a few dollars until you r profitable. why lose 7k when u can lose just 70bucks trying out the same trades 2) tip for determining SL is to draw your lines. trade breakouts bouncing off these levels, set SLs just below them. If you can't plot these, start with ATR as a stop loss OR use something like daily pivots. when trading breakout...
econbizer,

I really thank you for sending me this comments and advice. From this I have adapted the following in my discretionary mechanical trading.

1. Reduce contract size from 3 to 1. For now, trading small til something works is achievable and less stressful. And it makes sense.

2. Identify my targets, SL, and RR before the trade.

3. This is a big one!! Draw the trendlines, horizontal lines, whatever line that may effect my profit target and SL before the trade.

Thank you
  • #11
  • Jan 23, 2017 11:50am Jan 23, 2017 11:50am
Quoting DonPato
Again, sooner or later you will realize that you have more questions than answers about your observations, so make your data collection efforts expandable to include things you haven't thought of yet. Remember this is the start of a journey that has no end...so settle in and do it right from the start. This all by itself will save you hours of time starting and stopping because you didn't feel like doing it right or asked someone else to do it for you. I am happy to help if I can
DonPato,

Thank you so much.

This is the stage i am at now, learning to what to include in my data collection and questions/metrics to answer for the system I would like to trade. The last I want to do is start tracking and on trade 224 for example, I realize, I did not ask the right question for my system and have to repeat over again.

I want to learn how to statically evaluate my system, so I can use this to trade with confidence and without hesitation. This will take sometime, but it should be fun if i know what I am doing.
  • #12
  • Jan 23, 2017 3:43pm Jan 23, 2017 3:43pm
Quoting goodoboy
DonPato, Thank you so much. This is the stage i am at now, learning to what to include in my data collection and questions/metrics to answer for the system I would like to trade. The last I want to do is start tracking and on trade 224 for example, I realize, I did not ask the right question for my system and have to repeat over again. I want to learn how to statically evaluate my system, so I can use this to trade with confidence and without hesitation. This will take sometime, but it should be fun if i know what I am doing.
There is no time like the present. If you have already started tracking some of your results...GREAT NEWS! You have some data that will be of use. You can start no matter where you currently are and begin asking the "right" questions. If your questions are something along the lines of: "How can I 'KNOW' that my set up is trustworthy?" Then you will need to trade your set up over at least 100 trades where EVERY criteria of your set up is met, then ask these questions: (1) how many times did my set up provide a chance (no matter how small) to profit? (2) On average, what kind of profit can I expect? (3) under what conditions does my set up work the best? (4) if my set up loses how much does it lose on average? (5) under what conditions is my setup more likely to lose?

Then you can go about tracking these results...you must clearly define you set up and hold that up as a ruler to measure your results. If you find your setup gives you a 50% or greater probability of making profit, then look to how much you lose when you do lose. If your losses are small, and your profits are at least 2x greater than your losses, you now have the basis for a solid system that you can now forward test over time. I would forward test more than just 100 trades but that will be up to you.

Always remember that your setup criteria must ALWAYS be met. If you modify or "tweek" anything about this criteria you must start again with a new study. Even one little tweek can invalidate all your previous data. If you find after 50 trades, that your criteria isn't what you thought it might be...then back to the drawing board for a new observation and more testing.

It is an extremely long, painstaking, and at time aggravating process...but its WAY better than losing your money "shooting from the hip".
@xxxx§|[::::::::::::::::>
  • #13
  • Aug 29, 2021 5:04pm Aug 29, 2021 5:04pm
DonPlato is giving the best answers there are. Its mt 5th year researching and reading on Financial Markets and the only thing l really haven't done is Backtest all my findings. I am fully prepared to spend the coming weeks to backtesting!

We shall meet at the Top!
  • #14
  • Last Post: Sep 6, 2021 1:50am Sep 6, 2021 1:50am
The important of the system is the design. Sometimes it can't be tested.

We do backtest just make sure it work as design. Not prove the system is profit.
0 traders viewing now