Neural network strategy

Basic premise: since major forex markets are chaotic then they exhibit entropy and will never preserve order long-term. In a nutshell, you bet on that and make profit. Easy-peasy.

This makes sense, too. If the markets were becoming less chaotic then it would slowly become more and more regularized and predictable until prices would become simply a function of x and easy to calculate at any x. This also makes sense because currently markets are grounded in reality which is infinitely chaotic and will imbue unpredictability. It would require unspeakable amounts of order to make chaotic markets gain order even over the short-term - at best they can maintain the level of disorder already present.

What's required:

1. Linux,
2. one or many GPU cards capable of parallel computing (ideally with more and faster CUDA cores)
3. good, clean, reliable forex data
4. free Linux software for neural networks - tensorflow, python, etc.
5. some understanding of and experience with: programming, artificial recurrent neural networks (like LSTM), statistics and probability, finance, math, forex (for example, if you can implement a five-dimensional matrix in python real quick it will speed things up)
6. time
7. watchfulness and attention to (and love for) numbers

EDIT: you can supposedly use Google Collab service with access to free GPU and no setup. You can at least learn there. https://colab.research.google.com/?utm_source=scs-index

Steps:

1. Prepare the data - clean, fill gaps, check for accuracy - no comparing apples with oranges, consider data source and data's essence, add data to compensate for neural network's shortcomings). This step is very important.
2. Load the data in, prep for the neural network (standardize carefully, deal with outliers, decide on input data window, test data window, etc)
3. Run the neural network to find a statistical model predicting the next day's close - the key word here is statistical
4. Bet against it
5. Profit
6. Optional - optimize

This link is a good tutorial to get started with neural network programming: https://machinelearningmastery.com/m...g-lstms-keras/ that I found very helpful, the entire site is. But you must think for yourself and specifically about your project's goals when you program.

Because of how statistics works, this will require trading at every data point, that's the only way to capture the probability and the profits - after the backtesting confirms this model. I confirmed it but you should see for yourself.

This can and has been done on an older GPU card. Imagine, if we could have a supercomputer?

All of this in other words - this may be a mathematical way of saying and proving that every single model based on price (and its derivative indicators) will fail in the long-run to be profitable. That is, price has no predictive value.

If you are completely new to neural networks - try to have a neural network predict a simple relationship you feed it. I feel it's a good way to get started. Have it learn and predict that the next value in a sequence is addition of two values in different columns a step before. Then try a more complex relationship. Then try conditional or fuzzy logic.

I'm in!

Doing something similar. Could work. Will implement neural network and try.

Here my experiments so far with idea "bet against it".

I'm in too

That's kind of up this alley conceptually. Neural network makes it a pretty hands off experience...like it automatically kind of finds the best 'bad EA,' in your jargon (which is actually a good EA because it makes money on a local stretch extracting whatever ordered patterns in data) and then you know it will always fail statistically. So NN just generalizes things so it's always the best "bad EA" you can get and thus maximizing whatever edge you can get (which isn't big so you want to capitalize on as much of it as possible?).
EAs are sorely deficient, very limiting if you wanna analyze data. But it's a stepping stone I have gone through as well 14 years ago.

Costos,

Have you ever created a system that works long term with this? I worked with a team that implimented something like this for manufactoring but that is a contained envvironment compared to the wild wild west of forex.

Guys I have one question.
What if use NN not to predict next candle close or new day close.
For example I have very good entry point. They are perfect most time. But there are let's say 10% chance that it will be not good.
So the task is: to find best possible approach to average the entry, or reverse it.
Сan this be done using a neural network?

I created this a couple of months ago in September. Was focused on other things before. I put it aside for a little bit since then.
I did backtesting for several years (needs to go further) but no live testing. I have a 7 year old graphics card (it's not a bad card but still) so it's annoying and discouraging to keep working on this. If you change something that affects calculations you must wait days for backtest to see results.
But interestingly, I was relatively happy even with 3 years of backtesting for interesting reasons. 3 years covers the pre-pandemic period and the pandemic itself. As I was starting to analyze the data, I was seeing interesting things and dynamics but the pandemic seemed to disrupt any profitable patterns and trends that were emerging. When this final theory/strategy solidified and I ran the backtests the pandemic period was no longer an adverse effect and expectedly became the disruptive event increasing disorder and creating profit.
That's great. Indeed, forex is a great challenge.

I guess everyone got instantaneously filthy rich off of this strategy and is downing champagne bottles. Now that we've cracked the markets on the technical side - I have not mentioned anything about neural networks on fundamental data.

There are 2 kind of "AI":

1. Give some input to AI, train it (optimize), then run in real life - and here works concept "bet against it"
2. Implement that AI re-trains itself (self-learning) - here no idea how it would work, I have no resources to implement that kind of AI

Real AI is maybe 2nd.

But I am able to work only with 1st approach.

Regarding "fundamental data" AI - just take some financial feeds, get history, train AI and get again some optimized version which eventually will fail. I see no benefit for nr 1. to use "fundamental data" - the result is anyway just optimized setups which will fail because of fluctuations in long term. Red news now means nothing... random reaction. Sometimes markets move huge and sometimes do not react at all. And sometimes move in the opposite way as expected (manipulations?). If "red news" would work "fundamentally right" - all would be rich.

For nr 1. purpose "bet against it" it is ok to work only with technical data I guess. Anyway, can be used any set of inputs for Neural network. It can be anything. Even astrological info may be used to define data inputs.. All we need is to develop "bad" overoptimized EA and bet against it.

can see here i play arround with neural networks EA

The lines are blurry. You can take your AI #1 that you trained yourself and just write a script that will do all the steps automatically. It will depend on the specific task and goals at hand, computing capabilities, constraints, etc.

There's a major difference between technical data and fundamental data. Technical data you know precisely, conslusively, entirely - you have the price. The price represents the price. With fundamental data you know nada. You should know everything, right? But you know nothing. With fundamentals:

1. You do not know the true state of economic affairs. You get some numbers from some reports compiled by god knows who with god knows what kind of manipulations. If you ever worked in accounting or finance you might know how the numbers are massaged and changed around on the books and moved between entities, etc. There are also powerful players that can manipulate markets/businesses via inconspicuously exerting control over the aforementioned two and society in general.
2. You do not know the true state of economic affairs. The economic reports the public receives are horrendous. They are not designed to give an accurate picture of the economic reality. They are missing important metrics, and they might have a lot of things upside down - for example, human capital in financial reports is treated as an expense and a liability mostly (https://hbr.org/2019/10/the-problem-...tead-of-assets).
3. You do not know the true state of economic affairs. There currently exists no strong cohesive method to tie ground economic reality with the reports. To some extent due to objective reasons but mostly likely to prevent anyone from knowing the true state of economy and to profit from that.

There are already issues with fundamental data, as you can see. Getting data for human behavior is even more daunting - Google has it all haha. Google can very quickly check the entire internet they crawled for what affects the markets in which way. Getting data for weather might be the only data that's relatively clean and available.

Other than that, it's all easy-peasy, too. Of course, there are clever ways to deal with problems and various exploits.

I must qualify something. I only had a chance to work with EURUSD pair as the output for the neural network. It seems every pair behaves differently, so it's natural to wonder if and how things will be for other pairs. I think EURUSD is the largest market. That'd the best one to focus on for this approach then because that would it more chaotic (as well as more organized locally).

This system seems to be built on a fallacy.
You build a poorly performing strategy and then bet against it. Consider an extreme variant of this. Namely moving average crossovers. It can be demonstrated through backtesting that it's a losing strategy. Betting against this losing strategy will not yield a winning strategy because the crossover strategy is not the worst one there is.

The neural network strategy is not the worst either. Sometimes it will have winners.
Your task therefore is to come up with either the best of the best or worst of the worst strategy and invert the latter if you find one.
Your neural net idea doesn't fit the bill.

Great news, everyone. Free GPU and zero setup is offered by Google. https://colab.research.google.com/?utm_source=scs-index
You can learn how to code a neural network, test it, play with it for free.
Of course, everything you do then is on Google servers, and I don't know if you want to share your Forex models with Google once you get to that stage. But you can definitely learn for free.

Following.

Side note - I am slowing looking into machine learning when it comes to trading and been in the same sphere, is there any resources that you recommend for learning, been books or websites or videos?

I mentioned a couple of things already. I would start with reading up on statistics and then machine learning theory and basically really think about what you are trying to accomplish and how the tools at hand are able to accomplish it.

Machine learning is a way to find relationships. If you cannot even establish that hints of relationships exist within data with simple statistics then machine learning is kind of superfluous. As a trite example - machine learning is great at categorizing images; in this task you already know there's something on the image to classify, well, because you can see it, it has specific clear (and relatively simple) features and can categorize it.

Interestingly, you can treat price movement and other relevant data as pixels of an image but you haven't even established there's a category there within the data that you can tell apart from others clearly at all - there is not an unequivocal pattern that means buy or sell in 70% of cases, for example (or we would all be rich).

This is a profitable strategy. No one seems to be interested in discussing a perpetually profitable neural network strategy?
[I am the original source. Do I still need a resume to get a job now?]

You're absolutely right.

Only the second is "AI"; the first isn't at all.


Again, you're absolutely right, in the sense that almost nobody in a forum where the members are trading spot forex is actually able to discuss it.

There are other places where many are interested in discussing it, and able to: for example, Crédit Suisse First Boston has an entire department dedicated to evaluating profitable neural network strategies, and when they find them (which they do occasionally) they're eager to lease them on a profit-sharing basis (which, again, they have sometimes done). Their evidential standards for "perpetually profitable" tend to be rather higher than the examples found in forums, though - as you can imagine.

I feel like this is the place where people can help people learn and grow. I learned a bit a long time ago here and would love to have been learning even more then and now.

I wonder if anyone has been able to get this to run and work?