Currency Scan

I ended up just 0.002 behind the leaders in the hedge fund competition. looking forward to the next round. today though I've got to make some headway on the modeling that actually matters.

With the machine learning stuff basically on autopilot now, it's a matter of the most efficient integration (calling 240 complex models into Excel), I've got to start thinking about which fundamental model to put into the scan. There are a lot of different approaches to chose from and to be honest I'm not exactly looking for the most accurate as I think that's a waste of time. There currently is no good answer for any of the exchange rate puzzles, so I'm really more interested in tracking where we are from a cyclical standpoint to where the model thinks we should be. This will help me add a fundamental overshoot/undershoot which along with the technical measurements will give me more insight into potential periods of extreme asymmetry. That's my comfort zone as a trader. I prefer not to risk capital until I feel the risk profile is skewed. That's no guarantee, a bias can persist for a very long time and if you get into to it too early, you can find your trade under water for weeks and weeks... not a very comfortable thing. Of the different type of trading this works best for me at this stage of the game where I'm risking less and less and am more concerned with capital preservation than accumulation.

So with that in mind I'm going to start grinding through some of my old stuff and a bunch of new stuff I've recently added to the Kindle. I'll Add any of the research here that I find interesting or insightful.

Here is what's on deck for today: Bubbles and Crashes, Abreu & Brunnermeier, 2003
https://www.princeton.edu/~markus/re...es_crashes.pdf

Currently sitting in 46th place (out of 353) for this months hedge fund competition.... which means nothing. Everything changes when the balance of the data drops. Last month saw my model take a dive, it was clearly overfit, and I wasn't surprised about that. I actually had a chat with the CEO a few days before the comp ended and told him I thought there was going to be a lot of heartburn for myself and others if the balance of the data of was more evenly distributed. He just laughed and said "I can't comment on the bias..." It did well enough in terms of the fit, only .002 behind the top models, but it tanked in the ranks.

Ultimately this process is about making sure the models I make for myself and what I put into the scan are robust. This is helping me (forcing me) to consider new things and experiment. One thing I can see right away is, a lot of what happens in a comp can't possibly make it into production. The more I dig into the competitive frameworks, the more complex and unrealistic they become to implement in a production environment. With that in mind I've set up a second account at the comp where I'm testing a "production" only style model (look for the "Xcrape" score). Clearly I want it to do well and if it will stay within .003 or .004 of the leaders I'll be happy. The goal for this model is to reduce the impact of "new data". It also has to be easily implementable. I'm not dealing with single forecasts in my work, I have to have 24 of these running every day and as a rule I ensemble 10 to 12 of these together for each pair, so the framework has to be manageable through the entire process.

It's been a pretty interesting experience so far. Going to knock out a couple of tests this morning then get back to the fundamental stuff that I want to integrate... slow grind forward. - JP

Here is a good piece which digs into the disconnect between fundamental data and the magnitude of exchange rate pricing (section 6 and Fig. 3).
https://www.ecb.europa.eu/pub/pdf/sc...61b5118f3345f8

Thanks for the thread and for sharing your vision! Lots of information for slow reading. Can you please post your Currency Scans here on regular basis? Thanks.

I wish there was a way for us to automate some of our posting. I know that opens an entire can of worms, would make life a lot easier though I get updates from the scan right now about 4 times a day, I could set it up to post as often as people wanted to see it, every 15 minutes or so is possible - just not sure it's useful at that level and that frequency jacks up the data-costs pretty high. The difficulty for me is all the other coding and research I'm involved in that supports my trading/modeling, makes it hard to stop and do the forum thing. I'm not sure how some of these guys post as much as they do!

Not a lot happening right now in the short term momentum setups that interest me. The two longer term setups (AUD/CAD and GBP/CHF) have some potential but I wouldn't jump into that AUD trade until this bearish momentum subsides. The GBP could be a decent short if it breaks lower, it has some room to run.



On a completely different note, I've submitted my final models for this months competition at Numerai.
The comp model (ObjectScience) is currently in 43 spot out of @ 450 participants. This is the work it until it fits model and hope not a lot changes. There could be a ton of variance in the balance of the data set and like last month I could see this thing TANK!!! That's the fun of swinging for the fences. This model is still primarily an ensembled GP model built from a 10 K-fold Cross Validated structure. That model was generated in C++. Next month I'll be blending those results with a nueral net done in c# and a boosted tree algo in python (xgboost specifically). I would like to see a little less sensitivity to changes in the dataset bias... we'll see

The production model "xcrape" has also been uploaded. This is another ensembled GP 10 K deal that doesn't take into account at all how it fits the partial "test" set. It should suffer much less shock with the balance of the data hits, but right now it's sitting just above 230.

Still working on the dll for excel that will insert the machine learning models into the scan. After that I'll get the fundamental forecast knocked out. The issue with that is less the math as much as it is grabbing the data, trying to keep data costs down but not really digging the idea of grabbing links from all over the world that are prone to being changed and breaking stuff. It will get sorted.

After that I've got a pile of new ML research and economtric modeling in R and python I want to get sorted so I can generate some of my macro/fundamental analysis a little faster and make it look a little better.

Finally have Python and R integrated into Visual Studio, the xgboost install on the python side was miserable (was easy in R), but it's in there.



This will make development a little smoother. For R (in this setting) I decided to give Microsoft's version of R and their multithreaded app a try... there should be a significant boost in processing. Locally I'm working off of a 12 core workstation. If that starts sucking wind under some of the larger, more complex data-sets I'll setup one of the large 32 core monsters out on AWS. Their R-server is also an option locally if I decide to chain a couple of these workstations together. https://www.microsoft.com/en-us/serv...ucts/r-server/





Baby steps first, want to squeeze all the juice I can out of the old horse I have sitting on the desktop.

Now I just need to generate the laundry list of base projects that I'll need in the different environments for production.

The GBP/CHF showing a little movement lower, had a tiny break followed by some bearish momentum. This will kill the longer term signal in the scan but there is still plenty of asymmetry in the short-term studies (APL).

Interesting, thanks for your info. Will monitor GBP/CHF, now it's at key historical levels. I see breaking below 1.4080 may confirm bearish outlook (Fib-50 from last noticeable leg and nearby long term Fib-50 at 1.4120 may add confluence too)
Here is my picture if it's appropriate for your scientific thread

You're more than welcome to post in here.

There is a lot of intraday asymmetry right now, which as we grind into summer becomes more relevant (as volatility levels tend to fall off). There are a handful of short-term setups here worth digging into a little deeper.

DISCLAIMER: I'M NO LONGER TRADING

Without going into a lot of detail I've decided to suspend my trading activity, probably permanently. I just simply no longer trust the Retail Brokers. My most recent broker was my last stab here in the US and I've found myself incredibly disappointed and frustrated with some things that have occurred in my account that have gone a long ways to minimize my returns. I think it's safe to say, if you're a profitable, low frequency trader, you're not really what the industry wants. Because I'm not willing or really able to risk large amounts of money on exchange rate speculation any more, which would allow me a to use a more robust institutional system, I'm not left with many options. I guess my time with FX is done. Tough to see it go, I've spent a lot of time and effort on it over the years, and was really happy with how the Currency Scan turned out. But I guess most of that work, in hindsight, has been for nothing. With no broker and no data feed, I'll no longer be able to provide the analysis, something I'm really sorry about as I felt it added value not only to myself but to those who also followed it.

Going forward I'll probably focus on the data-science and machine learning side of things, as I'm able, just to keep my mind busy. I appreciate all the interest in the work I've done and I wish you all well. - JP

Just a question, if I may ask: I see that you started trading in 2000. Today it is 2016. Has it taken 16 years of time and hard work to recognize that you can not trust any retail brokers? Handling different brokers platforms I saw quite soon that they did not deserve any trust AT ALL. How come did that come to your knowledge so late?

Regards and all the best!

Jos

I spent a lot of my time on better platforms, where I had direct access to over a dozen different pools of liquidity. I think also, I always hoped as the industry matured as regulation found its way in, things would improve, that the clients would be treated better. It's sad that they aren't. I came on when retail FX was just really emerging and I had always hoped to see it grow into something respectable... But it's not hard to look at the bigger picture, all of finance/trading, and really see that the interest of the clients are and always have been a distant second to corporate and personal profits. This was the only thing I've ever wanted to do, since I was 15, speculating on exchange rates. That was it. Next year I'll be 50, and in hindsight, it feels like a mistake. Don't get me wrong, I've learned a lot and there have been times when the journey was very rewarding, but on the whole, the experience and the people I've encountered have left me wanting on every level, institutional and retail both, where they only real difference at times, is the cost of the suit.

As an aside to all this, it's possible for me to run a demo account and still produce the CS, I haven't decided if I want to bother with it or not and I wanted to be really transparent about what I am, and what I'm not doing... I may play around with it later, adjust some of the code to ease the db linking etc. Going to be hard to see it sit there and watch all that work gather dust. But that's a decision for another day.

Given consideration to futures via thinkorswim?

(came into the same conclusion with "CHFgate"...retail forex is a scam imo....from my own research my opinion is that retail FX brokers merely mark up or mark down a futures pricing stream in their own interest...sounds over simplified I know...but, why can't most retail brokers keep up with news while the futures market just eats 'em for breakfast while maintaining liquidity for quick entries/exits..??)

Hello, Have you looked into the nadex platform ? seems to be pretty good.

Looking for the silver lining in all of this... with most of the summer camping/hiking season behind me and not focused so much on my own model building and trading, it's given me some time to work on the Numerai modeling a little more. In early rounds (competing as ObjectScience) I was getting hammered with a lot of variance between the public leader-board and the private leader-board. Basically I was over-fitting to the test data. In last weeks round I sat down and got after some proper modeling, paying more attention to the robustness of the process and less to the score. It paid off. I still ended up scoring well enough to be above the "Controlling Capital" cut and managed to keep the difference between private and public LB's to .00083. There is still room for improvement from a structure stand point, but this will probably only improve variance a small amount and computational costs will go way up. At this point I'll probably focus on feature engineering a little more and try to improve the forecast themselves. At the moment I'm sitting at #33 out of 230, so basically in the top 15%... definitely want to do better than that.

Continuing to grind at Numeai. Haven't really improved much on the 15% placement even though the models etc are getting better. As I get better so does the competition. I've also started my comp at Kaggle, lucked out on a partner and we're currently in the top 4%. I was a top 20% on my own in that one - nice boost.

A lot of this working will be going directly into the scan, either for production or simply as an example of what I can do... not sure to what end. Hoping to get it back on here for you sooner than later though. Hope everyone had a good summer. - JP

Can we have regular currency pairs analysis over here? I would like to see daily chart, it will be more helpful for newbie like me. Thanks for thread post.