Predatory High Frequency Trading Machine Learning Thread | Forex Factory

Options

Go to
Permalinks
Print Thread

Similar Threads

Algorithmic Quant Trading (Machine Learning + Stat-Arb) 25 replies

Machine Learning Trading Algos 10 replies

Trading and machine learning 4 replies

My most recent advancements into machine learning 16 replies

Predatory and Parasitic Trading 36 replies

Attachments: Predatory High Frequency Trading Machine Learning Thread
Exit Attachments

Predatory High Frequency Trading Machine Learning Thread

  • Poll

  • Is predatory HFT ethically wrong?
  •  
  •  
  •  
  •  
  • #1
  • First Post: May 14, 2017 11:26am May 14, 2017 11:26am
Hello, as I said on my personal trading journal found here: ( https://www.forexfactory.com/showthread.php?t=627863 ) I am developing a new stat-arb strategy. I will be focusing on finding patterns in tick volume in order to track and predict large flows of capital between currencies. I thought it would be appropriate to form a discussion on predatory high frequency trading. I myself am not a market maker/scalper, due to the broker I use, but if you are a market maker/scalper feel free to discuss how your trading is going/provide personal insight.

I am exploring the use of neural networks when finding patterns in data. I am currently sampling and modeling data on 5m bars, but trading on 333ms increments. The reason for rounding to 5m bars is computational. It simply takes to much computing to do tick data. One day I hope to try tick data, but it's likely I will need to lease a cloud based vm. I've used cheap vms in the past to scrape and record tick data, but it can take 20-60 secs to backtest only 16 hours of tick data. This makes it impractical for neural networks.

I'm doing some visualizations of the data as it researches, so i can get a better idea of what some of the relationships plot out looking like. Here is an example.
Attached Image


Inserted Video
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #2
  • May 14, 2017 12:03pm May 14, 2017 12:03pm
I will test the models built in a few days to verifify they are not victims of over sampling. I expect to begin small scale (10USD) live trading my research in 3-5 days after checking out of sample data.
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #3
  • May 14, 2017 2:25pm May 14, 2017 2:25pm
added a poll
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #4
  • May 14, 2017 3:53pm May 14, 2017 3:53pm
I am finding success in my own strategy researching ai using order flow as a data input.
Inserted Video


Is anyone else finding success also? If so let's trade insights
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #5
  • May 14, 2017 4:08pm May 14, 2017 4:08pm
What is your objective? To see if a series of infinite regressions can do a better job than you or most people at trading? Obviously. The human mind is a finite series of regressions.

Or is it to see how much you can make from letting a computer do the dirty work of figuring things out because you don't know how to do it yourself?

If you knew how to think like a computer then you wouldn't be wasting your time on this.
"The fun is in the hunt. Not the kill."
  • #6
  • May 14, 2017 4:19pm May 14, 2017 4:19pm
Quoting Redeflect
What is your objective? To see if a series of infinite regressions can do a better job than you or most people at trading? Obviously. Or is it to see how much you can make from letting a computer do the dirty work of figuring things out because you don't know how to do it yourself? If you knew how to think like a computer then you wouldn't be wasting your time on this.

Good question, my objective is to build a time series forecasting algorithm. I input data I think may be relevant (in this case order flow) and the machine finds a very large number of signals with a high positive/negative correlation to the asset price. These signals are combined into a linear factor model and estimate expected value by finding bizarre patterns in volume. However if I suspect for example google search trends can predict commodity price, I could take relationships in search results, and price assets with them.

Now if you look at my trade explorer, I developed a macro strategy that I came up with manually (the hard way). It is mildly successful with a sharpe ratio ~1.25 on percentage returns. I used 3 macro indicators, forecasting them using auto regression models and thus forecast future exchange rates.

This computer program is very arguably far far stupider than most humans, it just is designed to do a few calculations, and thus can do them much faster. It invents 3-4 strategies a second, and then tests them in OOS (out of sample data) to make sure it isn't cherry picked.
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #7
  • May 14, 2017 4:44pm May 14, 2017 4:44pm
Quoting aroxalot
{quote} Good question, my objective is to build a time series forecasting algorithm. I input data I think may be relevant (in this case order flow) and the machine finds a very large number of signals with a high positive/negative correlation to the asset price. These signals are combined into a linear factor model and estimate expected value by finding bizarre patterns in volume. However if I suspect for example google search trends can predict commodity price, I could take relationships in search results, and price assets with them. Now if you...
Well, then, you're going to need a lot more hardware and quite a few quantum computers. A minimum of one. You'd need an AIG to run the process... and money wouldn't matter anymore at that point. The moment that a single human being makes a single irrational decision (buy high, sell low) it starts to fudge the data. When you start considering a massive group of irrational investors the data gets very fuzzy. Sure; you can find statistical correlations and define a few series of outputs with reasonable probabilities but you need to constantly re-define the parameters and interpret input that only an AIG is capable of analyzing.

Basically; nothing short of an AIG can interpret black swan events because they sort of violate all existing models. If you have a computer that forecast Trump's presidency and the market's reaction then you know what you have to do... destroy it. Immediately. Before it kills us all.
"The fun is in the hunt. Not the kill."
  • #8
  • May 14, 2017 5:57pm May 14, 2017 5:57pm
Quoting Redeflect
{quote} Well, then, you're going to need a lot more hardware and quite a few quantum computers. A minimum of one. You'd need an AIG to run the process... and money wouldn't matter anymore at that point. The moment that a single human being makes a single irrational decision (buy high, sell low) it starts to fudge the data. When you start considering a massive group of irrational investors the data gets very fuzzy. Sure; you can find statistical correlations and define a few series of outputs with reasonable probabilities but you need to constantly...

LOL AT TRUMP.

Sentiment is a factor that has been quantified. Plenty of companies specialize in mining for sentiment over a companies favorability with consumers, and these same services can be applied to FX, commodities, etc. I wrote an EA for a client who used bitcoin sentiment measured via google search results that found a way of predicting "black swan/6 sigma" events. He made 200% one week intraday stat-arb, Just regression models. Check out my YT channel, I have a video displaying some of the research i did for him.

Also, do you have any highschool/college education in statistics or computer science? High school doesn't count if it isn't AP or IB.
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #9
  • Edited 7:23pm May 14, 2017 6:48pm | Edited 7:23pm
Quoting aroxalot
{quote} LOL AT TRUMP. Sentiment is a factor that has been quantified. Plenty of companies specialize in mining for sentiment over a companies favorability with consumers, and these same services can be applied to FX, commodities, etc. I wrote an EA for a client who used bitcoin sentiment measured via google search results that found a way of predicting "black swan/6 sigma" events. He made 200% one week intraday stat-arb, Just regression models. Check out my YT channel, I have a video displaying some of the research i did for him. Also, do you have...
College for programming(C++). My highschool sweetheart went to Bryant University in the US for Actuarial Mathematics and graduated just shy of Suma Cum Laude. I was still helping her reason her way through some assignments before she graduated(I had a passion for physics).

I worked through all this rationale for over a decade for the explicit purpose of doing everything you've spoken of BEFORE I placed a single trade. The problem came to light when I started studying psychology. The answer came from quantum mechanics. At the end of the day we are all still bound by time. If, for instance, 911 were to happen again then what's the sentiment? You can't obtain accurate data for black swans before they occur. Some? Sure. But many are freak occurances that result from humans being assinine. Quantum entanglement's a real bitch. There comes a point where even an AI can only perceive with so much statistical accuracy. Nothing is omniscient.

My 'education' is minimal. I always preferred thinking and assimilating useful information over regurgitating lectures.

My question is; if you're already profiting then why not just go work at Two Sigma? It's probably a lot more entertaining(or boring depending on your perspective) to trade alongside other quants.

The point being that you can refine the process indefinitely but the dimension of time will always invalidate any computer's ability to accurately model the universe and it's impact on human psychology. The butterfly effect is entirely intertwined with chaos theory.

If you REALLY need the answer then it's 42.
"The fun is in the hunt. Not the kill."
  • #10
  • May 15, 2017 10:49am May 15, 2017 10:49am
Quoting Redeflect
... If you knew how to think like a computer then you wouldn't be wasting your time on this.
Good line.

But no comment on topic either way.
  • #11
  • May 15, 2017 11:13am May 15, 2017 11:13am
Quoting Redeflect
{quote} College for programming(C++). My highschool sweetheart went to Bryant University in the US for Actuarial Mathematics and graduated just shy of Suma Cum Laude. I was still helping her reason her way through some assignments before she graduated(I had a passion for physics). I worked through all this rationale for over a decade for the explicit purpose of doing everything you've spoken of BEFORE I placed a single trade. The problem came to light when I started studying psychology. The answer came from quantum mechanics. At the end of the day...
I can't work at two sigma because Iam seventeen and haven't finished highschool. As for black swans, the predatory strategies find 30-40+ opportunities per month per pattern recognition model. You dont have to be right all the time with neural networks, I measure the signal against price as a correlation coefficient. Thus the models always assume some error. The computer jus keeps discovering better and better models. It likely won't be perfect, and doesnt have to be. Just good enough to net a profit.
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #12
  • May 15, 2017 12:13pm May 15, 2017 12:13pm
Quoting Redeflect
{quote} College for programming(C++). My highschool sweetheart went to Bryant University in the US for Actuarial Mathematics and graduated just shy of Suma Cum Laude. I was still helping her reason her way through some assignments before she graduated(I had a passion for physics). I worked through all this rationale for over a decade for the explicit purpose of doing everything you've spoken of BEFORE I placed a single trade. The problem came to light when I started studying psychology. The answer came from quantum mechanics. At the end of the day...
Also i already addressed the hardware problem, using cloud based vms i could acces lot
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #13
  • May 15, 2017 12:18pm May 15, 2017 12:18pm
Quoting aroxalot
{quote} I can't work at two sigma because Iam seventeen and haven't finished highschool. As for black swans, the predatory strategies find 30-40+ opportunities per month per pattern recognition model. You dont have to be right all the time with neural networks, I measure the signal against price as a correlation coefficient. Thus the models always assume some error. The computer jus keeps discovering better and better models. It likely won't be perfect, and doesnt have to be. Just good enough to net a profit.
That's boring. The money is incredibly easy to obtain. Figuring out how to obtain it is always the fun part. Figuring out what you should do with it; not so much. It usually leads to quite a bit of discontent. I wish you the best of luck.

Also, I highly doubt anyone who said 'No' is happy with their lives. SunTrader understands.
"The fun is in the hunt. Not the kill."
  • #14
  • May 15, 2017 12:22pm May 15, 2017 12:22pm
Hi, the problem is that you are looking for patterns in set of fractal data (https://en.wikipedia.org/wiki/fractal).
But in fractal data there are by definition no patterns.

Here a nice TED talk about HFT

Inserted Video


  • #15
  • May 15, 2017 12:49pm May 15, 2017 12:49pm
A fractal is a pattern!!!!
  • #16
  • May 15, 2017 12:52pm May 15, 2017 12:52pm
Quoting fantasma
Hi, the problem is that you are looking for patterns in set of fractal data (https://en.wikipedia.org/wiki/fractal). But in fractal data there are by definition no patterns. Here a nice TED talk about HFT https://youtu.be/fawD7Nj74NA
Excellent video. Maybe some new traders will find it enlightening.
"The fun is in the hunt. Not the kill."
  • #17
  • May 15, 2017 2:26pm May 15, 2017 2:26pm
Quoting aroxalot
{quote} Also i already addressed the hardware problem, using cloud based vms i could acces lot
Try approaching it from a different angle. The hardware problem isn't the problem. The input is the problem. The data is already corrupted.
"The fun is in the hunt. Not the kill."
  • #18
  • May 15, 2017 10:42pm May 15, 2017 10:42pm
I am starting implementation. How are scalpers/hftraders doing today?
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #19
  • May 16, 2017 10:11am May 16, 2017 10:11am
Tested it, and it made it's first trade, netted 60 pips on the peso, but had two losing trades and only one winner.
Quant Trader - My Blog: quantstop.blogspot (dot) com
  • #20
  • May 16, 2017 9:27pm May 16, 2017 9:27pm
Quoting Redeflect
{quote} Try approaching it from a different angle. The hardware problem isn't the problem. The input is the problem. The data is already corrupted.
The data is surely corrupted think back to Greece having a ghost deal with Goldman sachs for a currency swap which made them look 3 billion dollars better off from getting 2 for 1 euros on the market, while they secretly owe Gold man and then no one gets in trouble. Logic would say that gold man has black deals with every other country and so no one will rock the boat because they are in the same boat. Meaning that the the exchange rate is seriously out, because these deals only half is on the exchange the other half is unknown. So than the market is now carrying skewed sovereign risk, imagine if all Euro countries owe large sums of money to the bank off the books, yet have healthy current accounts because they bank poured liquidity into their market/currency for a back deal loan. (leading to the conclusion we need more data, lots more because finding black deal money would be finding the error values in a proper fundamental regression on the market, random walk with a drift, least squared linear regression or so on)
0 traders viewing now