Peering down the algorithmic trading rabbit hole

PYTHON, ALGORITHMS, and FOREX

I'm starting this thread in the hopes that it may be beneficial to someone else later who may decide to go a similar route. To be honest, I enjoy manual trading and everything that comes with it. I've spent a majority of my time trading Stocks and Options but I've always been interested in FX (The no PDT rule actually attracted me).

That said, as a software engineer by day, I'm naturally drawn to the aspect of algorithmic trading simply for the love of the science behind it. As it encompasses everything I am interested in such as programming, trading, finances, algorithms, and statistics.

Oddly enough, over the years I have coded in more languages than I care to list here . But I've coded long enough to experience that there comes a point when most languages seem the same; it's the frameworks used with them that will drive you nuts. Even so, the only time you may notice differences language-wise is when you are trying to do something extremely complex, streamlined, and or with fast execution.

But as fate would have it, I've never coded in Python. I had always planned to pick it up but never got around to it. Go figure. So this is a perfect opportunity to pick up the language on the fly. I figure a few days...a week tops. The language seems extremely simple.

As far as my goals, I already trade with Oanda and I've been reviewing the API docs over at QuantConnect . I have also set up an account over at Quantopian as well. Simply checking out the communities and gauging their overall activity levels. More than likely I'll probably spend the majority of my time initially learning via QuantConnect simply because I'll be able to run my scripts against my own account, eventually. And, QuantConnect also offers C# as a language choice; that was initially attractive to me. But, I promised myself I would at least give Python a go as it seems to be the "favorite" in many algorithmic trading circles. Not to mention, there are a ton of examples in Python and the majority of other platforms offers that language as well. (Note: I am still up in the air though, I would imagine that C# executes faster with a more complex script because python has some bloat...but that's another discussion and that's what bench tests are for.)

More importantly, I would like to keep this thread updated with information detailing how "I" got from A to B and everything in between along the way. I am literally starting from scratch and have no idea what the heck I'm doing or how to get started other than opening an account at QuantConnect .

Regardless, just like any other trader, I want to become more knowledgeable about FX and work on becoming consistently profitable. The motivating factor behind this endeavor is a simple one. I am making the assumption that I might achieve a level of knowledge that I may not have if I had simply continued casually trading and growing via that experience. Although, one may argue that I would gain the same or a similar level of knowledge over time regardless. I would just reach it faster taking this route. For what it's worth, I am not talking about the knowledge gained from trading per se, as nothing can replace taking numerous live trades. I'm more or less talking about the knowledge of FOREX and trading as it relates to FOREX as a subject

Anyway, please feel free to leave any comments, ask questions, or provide advice. I'm new here so I'm definitely looking to make friends as well.

And if this was a TL;DR situation that's cool too!

Cheers

My first step is to just get the Python Language out of the way. I've looked at a few video tutorials online like Udemy, etc. I have also taken a look at some of the official Python tutorials. But I've decided that I just want to go through a small sized book just to get the basics.

Damn, there are a lot of Python books out there and from where I sit there are quite a few books out there that just seem like garbage and or cover everything but what you want to achieve. So I've decided on the book titled Python Crash Course . The book has a little over 500+ plus pages, it's filled with code to write which I like and decent reviews.

However, if I decide to stick with Python versus using C# then I've eyed a book I would honestly rather start with titled Fluent Python. Yet based on the current reviews it's somewhat nuanced and one should at least have a cursory level of Python experience. I can respect that.

As I begin going through this python text I'm also going to start with a few of the tutorials over QuantConnect. With the goal to just reverse engineer them initially and see what else I can figure out via discovery. But, I really won't give the API docs over at QuantConnect a hard go of it until I'm done with Python.

I'll check back later this weekend and give an early review of the Python book.

Cheers

Algo helps you to test every strategy ever known. Hope you manage to find that workable trading system.

In reference to utilizing backtesting. If past behavior is indicative of future behavior then one would hope for as much

But honestly, I am just looking forward to learning more about it and what is involved. Preferably would enjoy arriving at some ah-ha moments or even coming away from it a better trader than I am today; that would be ideal.

As I suspected Python is pretty simple to learn . I admit that I'm probably more biased than most when it comes to accessing difficulty because I write code all day. So what is easy for me may be slightly difficult for another. Using that as a given. I would still easily suggest a brand new coder start with Python first as compared to other languages out there. If for no other reason you will get to the business of creating programs much faster -- creating something that is actually doing work.

More importantly, early on in the book, I am able to see why Python would be such a great language to write algorithms with as the language just seems built for it. Although, I'm still holding my reservations about the overall execution speed in comparison to other languages.

As far as the book, it is extremely BASIC, again biased. But it's doable for what I need.

One of my favorite communities is over at StackOverFlow. I find myself there when I need to get clarification on programmatic syntax. So I'll say this much, there is such a thing as being "Pythonic" and if you want your code to be accepted in the community in a serious way, then you should aim to do things the "Python way".

Anyway...

My current environment stack looks like this:
1. Installed the latest version of Python 3.x
2. Sublime Text 3
3. Anaconda Plugin for Sublime Text

Sublime Text is an awesome text editor. It works natively with Python allowing you to build your code and execute it in the editor. Executed results appear in the console window below. Very convenient.

If you have ever done any coding before and you know what intellisense is (Visual Studio) you'll definitely want to install Anaconda. The anaconda plugin basically allows you to dot off an object and see all of the available methods, etc. Saves you time from having to look up stuff in the API. Very convenient.

At the rate I'm going I would probably be done with this book in a few days. But I'm flying out of town for two days next week for work. So I guess we'll see how it goes.

At best, by next weekend I figure I should easily be able to navigate my way around the Quantconnect docs using Python.

Cheers

If you come to the point where you need to speed up Python, there is Cython or PyPy (a JIT compiler).

Interesting. I'll definitely have to look into those.

Without knowing anything about Cython or Pypy. I'm assuming that the speed is due to the JIT converting the Python code to C. Therefore, the resulting binary would have the benefit of the increased speed.

Maybe that would work or maybe not as it depends on what you are allowed to deploy to either platform, respectively, QuantConnect and Quantopian. If we're dropping DLL's/Libs then it definitely sounds like something worth looking into.

Thanks!

An interesting week to say the least...

After recouping my personal time and getting my bearings back after a hectic out-of-state travel marathon for work. It has taken me a while to get back into the right head space for this endeavor, but nonetheless, I think I've found my groove again

In regards to learning Python. I may have been a little too overly optimistic as to the speed by which I would pick up the language Python is definitely an easy language to learn but there is a significant amount to learn, especially if you want to apply it properly.

One of the sections in the book I've wanted to explore the most was dictionaries, that has been a fascinating read. However, to round out my learning I still need to cover how Python handles functions, classes, exceptions, and testing. I figure another week and then I'm done with this book. At which point, I will begin to go through book #2 Fluent Python by O'Reilly. Unless there is something extremely significant about book #2 I doubt I will journal about my experience with it. Although, I will give my review of the book in this journal when I finish it

More importantly, I am almost comfortable enough to read Python examples with full understanding and I can do about 70% of my own Python coding on both the Quant Connect and Quantopian platforms. For me at least, that is actually a good sign as I'd rather be spending most of the time learning the API's than trudging the learning of Python.

Once I'm finished with the 1st Python book, my plans are to center the majority of my posts are around my endeavors in algorithmic trading, what I like about each platform, personal ideas, tests, and results, etc. I definitely did not intend for this journal to be about coding Python; it will about using Python for automation. Although, to be honest, at this stage of the journey it cannot be helped because if one is deciding to start from square one then this seems to be path thus far.

Still miles to go...

Cheers

Great to see a thread about algorithmic trading! Especially one coming from a ground up approach.

I'd recommend taking a look at the 'Python for Finance' (Yves Hilpisch) and 'Python for Data Science' (Wes MicKinney) books, especially the chapters about Pandas, numpy, and scipy.

Wait, is it still 2018? Right.

I can't believe it's been over a year since I have updated this thread.

To be honest, this was one of those "unfinished" things that has always nagged me. The programmer in me is still interested in learning more about algorithmic trading, simply for the pure science of it.

The end goal is to see if this is even something I would like to continue pursuing trading wise, let alone making a viable EA. If I some how managed to develop even a half decent trading strategy with a decent win rate, that would be awesome. But for now this endeavor remains strictly exploratory and informational.

Why did I seemingly abandon this effort?

My day job during 2018 was pretty hectic, the releasing of a major software version took up more time than I anticipated. In addition, trying to pick up another language like Python added an additional tail. Learning the language wasn't hard, however, had I stuck with a language I've used for years, I could have simply focused on the API and started learning how to write algorithms right away.

Going forward my focus will primarily rely on the C# language, using Oanda's Algo Lab's platform. Once I've gotten the basics down, and know my way around. I will more than likely revisit Python and determine if there are any advantages I may be missing out on (I'm sure there are).

More importantly, this is actually the perfect time for me to pick this all back up. I have about a month collectively of vacation time due to the holiday season. So there is no reason that I can foresee by which I can't develop a decent base going into the new year.

Should be interesting, at least for me ...and just maybe I, once I'm far enough along, I can close this chapter or at least make good on "unfinished business".

Cheers...

Today I spent a little over an hour going through the initial Boot Camp tutorials at QuantConnect. Creating simple algorithms via a template for a Buy and Hold strategy for equities and another for FX. FWIW, the tutorials were extremely basic and I expected as much since it is a beginner track.

That said, the overall setup for development and testing is cloud based and seems well put together. Nice size community. But, I was already getting frustrated from the experience.



The responsiveness of their code IDE's intellisense seems to be extremely slow. If you know anything about coding. Imagine dotting off of an object and waiting for what seems like an eternity for it's data members to appear. I don't know if it is because I'm on a free account, or if it's simply just a slow AF environment. Whatever the reason, it was beyond annoying. And for me, that type of coding experience was almost a deal breaker.

In addition, another issue I had was with some of the compiler error messages. Honestly, I know that my frustration stemmed from my current lack of knowledge and exposure to the Quant API. However, some of the compiler/test messages in conjunction with the instructions were both confusing and non helpful at times.

Surprisingly, I even had some difficulty quickly finding API related data on objects I was asked to use. Either, I was looking in the wrong place or I just don't have the full lay of the land down yet. Personally, I'm used to being able to find what I need on MSDN or StackoverFlow quickly. But their API docs just seemed to not provide at times the data I would have expected for built in objects. Again...maybe it's me, I sucked at searching their site, and oh yeah, it's only the first day.

By now, after about an hour or so I was literally sitting at my computer thinking, "This shit isn't fun let alone interesting at all.". I swear, I started looking at other algo platforms, and heavily considered going to back to working on a EA for MT4. But, my options are extremely limited. I want to create algo's mainly for FX, use Oanda as my broker, and I prefer to use C#. So I'm pretty much boxed in.



Anyway, right before I said screw it and was seriously going to just focus my efforts using MQL4. I saw a post in the forums on QuantConnect that caught my attention. It appears that you can develop your scripts offline using Visual Studio via a QuantConnect plugin. That in itself, solves the majority of my issues right there. For one, Visual Studio is the environment I use everyday. Secondly, it's extremely fast and provides me with responsive intellisense of which was lacking in the cloud IDE offered by Quant.

The only caveat is that there seems to be some what of a tedious setup to get this all going but, it's definitely worth a shot.

Once I get everything in working order I'll update my progress and provide any tips, and useful instructions on how to get setup locally; in the event someone else ever decides to go this route

I know I must sound like a pretentious whinny little brat but I did find a few Reddit posts with similar complaints. Nothing like confirmation bias right? LOL.

Next Steps:
1. Setup a local development environment using Visual Studio
2. Create a custom process pipeline that pushes my local files to the cloud (Quant) and GitHub for personal backup
3. Continue the current boot camp path

What a difference a few days make

Long story short. Nothing never works out how "you" plan. I am starting to become a firm believer in that fact.

Let's just say the amount of drama I went through attempting to setup my offline environment did not go as planned. So instead of publicly bad mouthing "anyone" and or a process, let's just say I scraped the entire idea and caved. Just too much trouble for what it is worth.

However, I didn't let any of those obstacles deter me from pushing onward. We're still in this...



Funny story...I ended up going back to Python. I swear, I think the universe is punk'n me.

Anyway, I decided that instead of attempting to learn the Python language first, and then go through the API docs and tutorials. It just makes more sense to pursue them both in parallel. Honestly, this is no different than an other time at work where I may have to pick up a unknown language on the fly. I really should have did this in the very beginning. Caveat: If you have never coded before, don't go this route. Learn your language and how to code first, trust me.


Oh well, over the last few days, so far so good. Python is extremely straight forward and getting passed the occasional nuance and quirks are minimal.

One positive take-away is that I still kinda get to use Visual Studio, well Visual Code, which works well for offline Python work. It's not Visual Studio but it works just as good all the same.



More importantly, I've gotten through two of the boot camp modules. One of which was Buy and Hold Equities and the other, which was nothing but a horse of different color, Buy and Hold Forex. All and all pretty interesting. The tutorials at least at this stage does a lot of hand holding. Expected I guess. Nothing too serious or challenging at this stage.

Here is an example:

All you're really doing here is setting up a initial deposit, the bookend dates for the back test, and the parameters for the pair you're trading. We also setup the broker and subsequently execute a market order for 2000 AUDUSD.

Easy peasy

-----------------------------------------------------------------------------------
class BootCampTask(QCAlgorithm):

def Initialize(self):
self.SetCash(100000)
self.SetStartDate(2017, 5, 1)
self.SetEndDate(2017, 5, 31)

self.AddForex("AUDUSD", Resolution.Hour, Market.Oanda)

self.SetBrokerageModel(BrokerageName.OandaBrokerage)

def OnData(self, data):

if not self.Portfolio.Invested:
self.MarketOrder("AUDUSD", 2000)
-----------------------------------------------------------------------------------

Personally, I get the feeling early on that the boot camp is the 50,000 ft view. In the end, I don't think one is expected to walk away and be able to develop a decent entry level algorithm. From what I've seen that will come from pursuing the API docs and actually developing a strategy. The boot camp seems thorough enough and you will definitely be familiar with the API at a base level. Of which should be enough to spark any ideas on how to do something, and or how to use the other complimenting tutorials. There are tons of examples and shared scripts; I'll hold my judgement for now.

As far as boot camp modules, next up I'm looking at something like Buy and Hold with a trailing stop, Momentum-based tactical allocation, Opening Range Breakout, etc. At this point, I'm just taking notes, and working my way through it all.

Definitely looking forward to getting to the point of developing a simple strategy that I came up with and watching it run on demo. How long before I get there? Who knows...but we will