Assessing the Profitability of All Possible Trading Strategies

Hello,

I would like to develop a trading algorithm that is consistent with the random walk hypothesis; in order to do this, we must assume market fluctuations are fundamentally unpredictable.

A successful system that makes this assumption would be profitable under virtually every condition in the market since the market direction becomes irrelevant. In addition, this profitability would exist regardless of whether the assumption was true (but the same cannot be said about systems that rely on market prediction).

Furthermore, an approach of this kind would allow us to simplify trading to the point where we should be able to quantify all the possible actions of a trader. This is because, if we assume market fluctuations are purely random, it becomes easy to ignore trading strategies that depend on technical/fundamental analysis and thus, trading simply becomes a combination of various operations (buy/sell) and techniques (such as hedging).

If all the actions of a trader can be reduced to a finite set of elements, it is reasonable to wonder if one could analyze the profitability of all possible configurations of these elements. An analysis of this kind would be a very efficient way to assess the profitability of nearly every possible trading system (we would no longer need to waste time mindlessly searching for an optimal strategy/Holy Grail).

For instance, one should be able to systematically evaluate the profitability of all possible combinations of trading strategies through a series of mass computations involving multiple large/sparse systems of linear equations.

I won’t go into much detail at the moment-- but I would like to know if anyone in this community is aware of such an analysis? Has research in this area ever been attempted? If not, would anyone else be interested in investigating this topic?

Thanks!

Your idea boils down to the question "can a system generate a positive expectancy from a pure RW". Then to find such a system.

If you assume the pure RW, no hedging can exist with any other instrument as the hedging exists only if there is cointegration. There is no cointegration between two RW (only spurious correlation). If by hedging you mean going long and short at the same time on the same instrument you only change the net exposure.

So we are left with actually only two possible actions for the trader: increase his net exposure or decrease his net exposure. Exposure is negative when you're net short. Any strategy is a modulation of the exposure over the time.

If you had access to a quantum computer perhaps... The exposure can be changed up to at every tick and the size of the change goes from 1 microlot to your full available margin. Just increasing or decreasing by 1 unit every minute makes 1,152,921,504,606,846,976 combinasions for the first hour!

Also I see no reason for linear algebra to work since a RW is a fractal.

I don't think exist a strategy that would generate profit in a random environment, because the very random fact, added to the real costs of trading (spread) and more important the real fact that the capital is a finite number (for example you lose 50%, then need to make +100% to recover), this is essentially what makes profitable trading so difficult to be sustained.

Sustained profits from a "random" strategy..? I hope you'll have success on that. But i don't think it's possible.

I wish you well but I don't fancy your chances.

The problem with random price series is that they hold no predictive power at all and it can be demonstrated statistically that a symmetrical trading strategy based on point change will tend towards 0 expectancy in the short, medium and long term as the expected value of a model of this nature has no change irrespective of the periods under consideration. It gets even worse when a small bias such as frictional costs are incorporated as the zero profit expectation turns negative.

It only becomes a game changer in my opinion if the price series can be demonstrated to have an inbuilt positive bias and that's where non-random components of price action have predictive power.

David Aronson in 'Evidence-Based Technical Analysis' has used 25 years of historical data to test 6,400 binary buy/sell rules on the S&P500 so it is worthwhile having a read of this book if you are interested in rigurous statistical testing.

I was among the first to see your thread but I have delayed my posting here (even thought of not posting whatsoever). After I have seen the above opinions, I decided to first of all wish you luck in all your endeavors and second of all to give my opinion:

Our personality and character gives us the urge sometimes to over complicate things especially in this century of fast moving and glamorous technology.
My advice to you and to everyone when it comes to trading is not to complicate things - save yourself the time. There has been many others that tried random walk theory and other theories for achieving profitability in trading. Probably this is an evolutionary step in your journey to trading successful.

Trading is simple enough when you have the experience/knowledge and clarity of mind but not that simple that everyone could figure it out for themselves - thus the losing percent in retail markets.
Indeed mathematics have something to say in the markets, but try not to over complicate it.

I admire your logic KV, and your ambition and probably in the end you will succeed in becoming profitable and trading for a living.

Just to get you going - combine different 'theories' in trading like market orders, volume price and time. Do not forget: the market is made of ticks not H1 bars.

First of all random walk doesnt' mean RW is unprofitable.
I share similar ideas but I havent' proved it yet.I am going to do some tests.Can you share some academic papers if you know any ?

A RW is bad at visiting its state space. In clear text: "in some time the price isn't going to be really far from where it is now". It implies that the best target is the current price. It also means that if the price moves significantly it won't come back soon: no mean reversion. The variance (error on this prediction) increases linearly with the time. In summary the SL is very big and the TP is close to zero. If the price goes against you the probability of winning goes down. If the price goes with you, you shall take profit because of the high probability of returning to the starting point.

How could it be profitable?

Nice PipMeUP. That's something to chomp on. :-)

I was going to write an essay explaining exactly how I believe this could be done; but I think it might be best for now if I just present the gist of my idea through a few examples.

First, I would like to clarify that I am not trying to create a trading algorithm that attempts to replicate/simulate the price movements of a random walk or some complex fractal pattern. I might have overemphasized the role the random walk hypothesis would play in this assessment. I mentioned the idea of a random walk as a way to stress the fact that this assessment would not rely on any kind of market prediction. If we approach trading this way, we can simplify trading to its most essential elements.

In its simplest form, trading is merely the interaction between the actions of a trader and the outcomes of the market. If we can quantify all possible actions of a trader, as well as all possible market outcomes, we should be able to compute the profitability of all the possible trading configurations; that is, all the interactions that can exist between the actions of a trader and the outcomes of the market.

An analysis of this kind can be divided into three steps (for simplicity):

1. Simplify all the outcomes in the market.
2. Simplify all the actions of a trader.
3. Assess the profitability of all possible configurations of these elements. The goal would be a system that is profitable for all (or most) of the market outcomes. It could also be a system that has a greater probabilistic chance of arriving at a net profit over time.


STEP 1: SIMPLIFY ALL THE OUTCOMES IN THE MARKET
We are able to reduce all possible outcomes in the market to a finite set of outcomes; this assumption holds regardless of whether the market is a purely stochastic process. This is achievable if we begin with a simple form of hedging; we place two pending orders of different types at equal distances-- we place their respective TP/SL such that the TP of one order is the SL of the other and vice versa. This creates a closed system and it is the first step to isolating all conceivable outcomes. A [possible] example of this setup is seen below.

Attached Image

If we use the previous hedging setup, it becomes easy to quantify all the possible outcomes in the market. An illustration and description of these outcomes can be seen below. Each numerical value of n represents a possible market scenario.

Attached Image

1. Let n=0 (blue line) be the scenario in which one of the pending orders is executed and subsequently closed via TP.
2. Let n=1 (red line) be the scenario in which both pending orders are executed and subsequently closed.
3. Let n=2, n=3, n=any positive integer… (same idea extended further).
4. n to infinity (green line) simply acknowledges that the price could oscillate within the small interval for a very long period of time. This is unlikely to happen but should be considered.

This is a possible way in which one would be able to simplify the nature of the market. With this hedging setup, the market direction becomes completely irrelevant; you are really only concerned with the market volatility as well as the ability to open and close trades that would be profitable for all the scenarios (values of n).


MATRIX EXAMPLE
I have explained the general idea behind the first step. Rather than go into great detail about the other steps, I will present an example that should hopefully clarify most of the reasoning behind this assessment as well as clarify how linear algebra is relevant to this idea.

Given the hedging setup above, how would it be possible to arrive at a net profit for every outcome in the market; that is, for every value of n (n ∈ ℤ+)?

A possible way in which one could always arrive at a profit would be through the use of martingale betting strategy. Read the caveat about martingale (at the end of the post). We can use linear algebra to assess the trading scenario.

Attached Image

The above matrix represents the martingale strategy for the previous hedging setup. Each row represents the market outcome-- the value of n (0, 1, 2, 3 and 4). Each column/variable represents an order (if the first order is a sell stop [1st column for n=0], then the second order [2nd column for n=1] would be a buy stop etc.). The solution to each variable represents the trading volume size (lot size) for each order.

Standard Gaussian elimination will give the following solution:

Attached Image

If you plug the numbers into the matrix, everything works perfectly. It is important to realize that this matrix example only has 5 rows (n=4). This means, in this example, the system will fail if the value of n is grater than 4.

This is a brief example that should illustrate the importance of linear algebra. I probably didn’t need to create a matrix and perform Gauss–Jordan elimination to determine the trading volume sizes of the martingale setup in my example; anyone could discover this with a bit of thinking. However, when the trading systems/setups become more complicated, it becomes a very useful tool.

Perhaps the greatest benefit of using matrix analysis is its ability to prove the impossibility of a particular setup. Before I began to use this, I was wasting a lot of time trying to discover a hedging setup that would be profitable. However, the matrix representation of the hedging system was inconsistent; I was trying to do something impossible, but only found out after using a matrix.


CONCLUSION
From here, you should begin to see how it might be possible to assess the profitability of all possible trading strategies. In the above example, we have reduced all the market outcomes to a set of finite outcomes. An issue that was not discussed, however, was the simplification of all the actions of a trader.

I have defined several of the most fundamental operations in trading to be the following (tell me if there are more). Under the assumption the market is purely random, trading is nothing more than a combination of all these different operations.

1. Trading Volume (Lot Size)
2. TP/SL line (Take Profit /Stop Loss)
3. Order Type (Four Different Order Types)
4. Partial Close
5. Initial Execution Price & Transition Line(s)

Thus, if we manipulate all of these different types of operations, if we consider all the possible combinations and permutations of these actions and how they could interact with all the possible outcomes in the market, if we then use matrix analysis to determine whether there is a valid set of trading volume sizes that allow for a realistic profit— if we consider all these factors, it is reasonable to wonder if we could, through a series of extensive computations, arrive at a trading system that consistently outperforms random chance and does so with utmost efficiency.


SOME CAVEATS
I apologize if my explanation is not very clear. I am still working on an essay/text that should hopefully explain the idea in more detail (I threw this post together as a tentative explanation).

I understand the hedging setup oversimplifies certain elements of trading. For instance, the example I presented does not take into account the bid/ask spread or CFD expiration. These are problems that should be looked into. However, I do not feel they are significant enough to vitiate the entire idea.

In my examples, the term hedging is used loosely to refer to the idea of going both long and short; I am not using the word in any formal financial sense.

I would like to stress-- martingale betting strategy, as it has been presented in my example, is a horrible idea since you are at great risk of total loss due to running out of leverage. I only used this as an example since I feel most people are familiar with the idea.

I should also stress-- this idea is quite straightforward! If it seems like I am overcomplicating trading it is likely because I did not explain the idea in a clear enough way. I say this primarily to caution those with an intransigent attitude toward trading; that is, those who think there is only one way to trade and believe that any novel approach is fanciful or naïve.

Thanks,

James

Sorry but I don't get it. You say the rows of your matrix encode a scenario. The first row is (1,0,0,0,0,1) but the scenario n=0 is made of only one single buy. Why are there two 1's in this row? Also the columns encode orders. The first column (1,-2,1,-2,1) contains 5 orders. Two of them are 2 lots shorts. Your example only contain 2 pending orders. And what is the vector x supposed to solve?

find some decoding books for electricity (audio, vision,.....).

Ok I don’t blame you.

I will try to breakdown the structure of the matrix and explain how each row and column is derived from the hedging setup in the previous post.

First, it is important to understand that the matrix in my example is an augmented matrix – meaning that the far right column is not a variable it is what the equations must equal. This must be a numerical value (it's a constant not a variable).

I should also stress the fact that the matrix is operating with ratios rather than pips. The hedging setup has a profit to loss ratio of 1:1. The ratio between the two execution lines for the pending orders and the TP/SL lines for both orders are the same. This is why I like that particular hedging setup; it’s easy to work with.

I have drawn out an explanation that should hopefully clarify the philosophy behind the matrix.



This is how it works:

For the scenario n=0, the profit value is +X.

For the scenario n=1, the profit value is –X if the lot size for both pending orders are equal (this should make sense if you visualize the scenario). However, we can apply martingale strategy and have the lot size of this new order (the buystop) be larger in order to compensate for (and surpass) the loss from the first order (the sellstop).

For the scenario n=2, n=3, n=4… we apply the same logic. We must continue to open new orders to compensate (and surpass) the previous loss. We must do this until the market volatility is sufficient to close all orders via the TP/SL line. When this happens the cycle repeats.

Remember: none of the orders close until a TP/SL line is hit. The orders just keep piling up.

No.
The first column represents only one order. In my example it happened to be a sellstop.
The coefficient of the variable X1 is what you are seeing in the first column. It will oscillate between +1 and -2 depending on the scenario (the value of n). This is because, for the sellstop, the order is in profit for the n values of 0, 2 and 4 but the order is in loss for the n values of 1 and 3.

There is some very heavy duty thought processes going on in there KV427 and I hope that some of the eggheads on this forum can assist.

I must admit I was suckered into the analysis by paralysis phase for a number of years playing around with Martingale variations with a hedge. It was so addictive that it blinded me to some critical assumptions about sample size with respect to the Law of Large Numbers and data mining bias that if it wasn't for excruciating statistical analysis I would still be doing this today, such was the apparrent logic of my assumptions. If I had spent some time reading Hanover's and others posts, I would have saved myself a lot of trouble and frustration.

Not to say at all that this is happenning to you.....in fact it is not reading your assumptions in post 9 however I wonder how you can apply this given the huge variance in techniques over significantly different timeframes. Do your simplification assumptions hold for any timeframe eg. Your closed system captures methods on a single timeframe, but does this extend to include all time horizons and thereby all possible actions of a participant?.....as such extensive time went down the tube with me that I don't wish that for anyone......but.......there is always that finite chance (and in your case probably a good one) that you may be onto something however your matrix and analysis is simply beyond me. I hope there is someone here that can assist.

I have spent most of my life trading in the institutional setting, and have spent hundreds of hours chatting with quants. There are several problems solving the system you propose. Efficient Market Theory specifically rules out specific predictions of market direction. (though I suspect you have read it, a great primer on EMT is Dr. Burton Malkiel's "A Random Walk Down Wall Street") But there are further problems which have dogged quantitative market analysis. To date, the most successful algo trading has been High Frequency Trading, which is essentially overwhelming the market with volume and forcing it to move in a certain direction, followed by taking profits in the 2-3 tick area. Results from quant trading systems have lagged human traders, thus far. Contributing to the quant problems are:

1. Randomness: Depending on the market you intend to trade, 30-50% of the market movement is random. Given this level of random trading, your standard deviation of error would be astronomical; actually, it would (by itself) render any predictive value nearly useless, at least from a mathematical equation standpoint. However, you might study reversion to the mean theory and standard deviation, specifically using a 200+ SMA and sigma 2 and sigma 3. I trade this system very efficiently, as a scalper.

2. Variable Definition: For any equation to have validity, you must have clearly defined variables. This becomes problematic when identifying variables that affect the movement of market price. If you try to identify a few, you develop a cascading set of variables that can run into the hundreds. (ie- volume, trend, trading velocity, breadth, price action, hundreds of global issues, currency pricing, the Fed, High Frequency Trading, dark pool trading, zero sum trading, unexpected industry-specific issues...I think you get the idea). You might read Bill Williams "Trading Chaos" which had a profound effect upon my trading and algorithms. As market pricing is really a function of public perception of a limited or expansive understanding of individual variables, you will find yourself led to game theory. Some individuals may invest on rumor and others invest on solid mathematical and trading principles; yet either one has a decent chance at winning in the short term.

3. Long Tails: Benoit Mandelbrot's "The Misbehavior of Markets" should be required reading for any potential trading system. Mandelbrot is a mathematician who did much to debunk the theoretical underpinnings of Efficient Market Theory, along with the housing and credit crisis of 2007-2008. Clearly, there are inefficiencies in the market which cause pricing errors of significant magnitude. As you might surmise, I use the Chaos Theory school of trading as an underpinning for my trading system. Long tails are those surprise events which occur with increasing frequency in the market and cause catastrophic movement one way or the other. Often these long terms are baseless fears as evidenced by the 300 point drop the day a single individual entered the US with Ebola. It caused quite a stir, then the market rebounded back to it's previous level. You would need a heck of a stop to ride that one out.

As an alternative to your EMT view, you might consider identifying specific variables that have the "greatest" effect on the markets you trade and then use an equation that shows either convergence in your measurements or divergence in your measurements. Forex will cause some problems with this approach because of the lack of transparency and outright manipulation of currency prices. There are currently 80+ Federal investigations of price manipulation in the Forex market in the US alone. Volume is going to be one of your most important variables and actual volume and implied volume are 2 very different animals. I trade futures and use 4 specific variables and convergence/divergence to initiate trades. I should add that I am a dyed-in-the-wool scalper.

I have identified 6 specific trade entries that I use over the last 30 years. Over the past 7 years, I have kept an Excel spreadsheet on each of these entries and detailed data about each trade. I identify the time of day, day of the week, trending-sideways-counter-trend, trade size, entry/exit, and a set of proprietary analysis-based algos to each trade. I have more than 10k lines of trades and know the approximate probability of each entry succeeding or failing. I think it's important to understand that even the highest probability trade has a component of failure. For example, I have 3 trades that have an 80% chance of succeeding and take those trades without flinching, yet there is still a 20% chance of failure. In essence, trading is more of a probability exercise than anything else. Further, I know the average run of each of these entries. For example the CL contract usually moves 26 ticks, on average, before profit taking sets in or it retraces. That being said, there is no guarantee that each move in my set of entries will be 26 ticks, but I will usually hold for 20 ticks because I know probability favors this length. So, parts of trading are very subjective while others others are objective. As a trader, you can only trade the highest probability. By the way, this system will make you an excellent trader.

I hope I have given you some ideas and direction. Identify the most important variables, isolate and measure those variables and trade when they converge. I suspect that you will have a good deal of trial and error until you hit on the combination of variables that provide the most satisfactory results. You might try back testing, but the last ten years will probably provide you with aberrant results because of the unstable conditions, government intervention, and volatility that have occurred. Best of luck and I hope it helps in your quest.

What are you on about?

Hi,

It is an (easy) understandable idea but with the martingale as base it will never be profitable. So what is left without the martingale....
There is really nothing left because its all based on martingale, the whole hedging doesn't make any sense without it.

I made some notes on your paper, you probably already knew this.

This idea is based on a thing I don't really know the word for but it's like this:
Assume the market is random, there are 2 ways up or down so the chance is 50/50. So enter random, have RR of 1:2 and you would be profitable. But we all know this doesn't work. Does this prove that the market is not random? No. The assumption is not valid.

So how do we make it profitable? To make it profitable n should be finite TLDR n=x.

OK. But why do you require the target to be (1,1,1,1,1) ? You only need the values to be non negative
For example using (1, 1/4, 1/8, 1/16, 1/32) your solution becomes (1, 2.25, 3.625, 7.0625, 17.65625) -still a martingale-

Quoting PipMeUp

Disliked

You only need the values to be non negative

Ignored

Yes. Exactly!

I used 1s because it was easy— but yes it could be any number greater than or equal to zero (actually, negative numbers [representing a loss] should also be considered but I will discuss this later).

In fact, if we make the final column (1, 0, 0, 0, 0), the solution becomes (1, 2, 3, 6, 12).
The goal here would be to minimize the exponential growth of martingale as much as possible. The zero would represent a scenario in which we break-even (however in reality I don’t think it would be this ideal because of the bid/ask spread).

The thinking would be-- if you can’t make a profit on your first position, ignore the idea of a profit and just focus on loss prevention. A system that uses this philosophy would close trades that are either profitable or result in a net profit/loss of zero; of course, if you could arrive at a strategy that respects these conditions, your equity would always trend in the positive direction.

Quoting Jysticim

Disliked

So what is left without the martingale.... There is really nothing left because its all based on martingale, the whole hedging doesn't make any sense without it.

Ignored

Not necessarily.

GOING BEYOND THE MARTINGALE MATRIX
Although I am sure we can eventually make use of some component of the martingale strategy (this includes the idea of a buffered martingale or a linear growth [non exponential] martingale)— it is important to remember that I only used that martingale matrix because it was an easy way to explain the general idea.

The game becomes a bit more interesting when we consider the buy and sell limit orders. This, in addition to TP/SL line modification (and many other operations I have not yet discussed), increases the number of system configurations.

I have posted two other images that should explain this idea further (it's easier since FF does not support mathematical text).





The first example [Example A] was one of my first attempts to create the break-even system I proposed above. But it does not work-- that particular setup is impossible since the matrix is inconsistent (many variations of this setup are inconsistent as well). However, we should not conclude that the entire idea of a profit or break-even system is impossible; this might only be revealed after a more extensive analysis of all possible configurations.

With a bit of tweaking, we get a system that works but it is not as ideal as what I was trying to do in example A – I will discuss this later (it allows for negative outcomes).

I’ll leave it at that for now.

Hey KV427, thanks for the thread! I tried solving the matrix by hand and after a bit of effort was successful. But I'm really rusty with matrix math, and it's too easy to make errors with the calculation, and now my brain hurts so I moved into R. Here is the baseline case using sell / buy stops just to show that I have the correct numbers:


So I tried some break-evens as an outcome, and the lot sizes still increase too fast as you can see:


So here is the question: do we really need a positive outcome every time? What if we allowed for some losses? If it really is a random walk, then it is a 50/50 proposition at each step. If we consider one round as a total outcome then I see the below as either a win on the first time, or a small loss or even odds loss at one of the points below, or a bigger loss of 5. It still comes out to a 50/50 payout and I don't really like how the "equity curve" looks, so it's time for a more devious plan.

Exactly!

If we allow for the possibility of a few losses, the number of configurations is greatly increased.

I have found a few systems that use this reasoning and may be profitable (statistically, in the long run); they use a combination of all 4-order types (sell/buy stop + sell/buy limit) as well as a modification of the TP/SL & execution line(s).

Modification of the TP/SL & execution line(s) simply refers to the idea of changing the ratios in the initial hedging setup; this is a variable and can be modified.

We should also consider the idea of a partial close and how it would be encoded in the matrix representation of a hedging setup.

I have included more detail on this topic in post 25.