Vibe Trading

https://www.rfastrat.com/module1 I haven't updated the site in the last couple of weeks, but I have tested over 100 hypotheses and a lot more variations.

bad day at the office today

someone asked if I backtested?

Are we actually better than Pardo? A self-audit

I went down a rabbit hole this week that I want to write up, partly to organize my own thinking and partly because I suspect a lot of people building systematic strategies have never actually checked their methodology against the person who wrote the book on it. That person is Robert Pardo. If you trade mechanical systems and you have never heard the name, you have almost certainly used his ideas anyway. Pardo is a CTA who has been running systematic futures since the early 1980s, but his real footprint is methodological. His book, "The Evaluation and Optimization of Trading Strategies" (2008, the rewrite of his 1992 original), is the canonical text on how to test a system without fooling yourself. The single biggest idea in it is walk-forward analysis: optimize your parameters on an in-sample window, then trade them unchanged on the next out-of-sample window, roll forward, repeat, and stitch the out-of-sample segments into one continuous equity curve. If the system only works on the data you fit it to, walk-forward exposes it. If it survives, you have something closer to real. So the question I asked myself was simple: is my process as good as his, or better? Not "do I feel confident," but actually mapped, pillar by pillar.

Here is where I landed.

On the core pillars, I match him. I run walk-forward folds on every hypothesis. I demand a parameter plateau rather than a lone profitable spike, which is exactly his robustness argument restated as a plus-or-minus-20-percent stability gate. I require large samples and few parameters because every degree of freedom you add is another knob the optimiser can use to memorise noise. I bake in realistic costs: slippage, commissions, and on top of that the specific drawdown rules of the funded-account platform I trade. That last part is arguably stricter than Pardo's generic cost treatment, because my constraints are not negotiable. A strategy that backtests beautifully but violates a trailing drawdown rule is dead on arrival, and my simulator knows it. Then there are two places where I think I am genuinely ahead of him, and I want to be careful here because it is not because I am smarter. It is because the field learned two painful lessons after his book was written.

The first is multiple-comparison correction. Pardo's framework has no real defense against the modern failure mode, which is: I generate eleven variants of an idea, walk-forward-test all of them, and the best one looks fantastic. Of course, it does. With enough variants, something will pass walk-forward by luck alone. The fix is to penalize for the number of tests, a Bonferroni-style threshold. This is not academic for me. It has killed live candidates. One pullback strategy looked like a clean winner on its best variant and died the moment I asked, "but how many variants did you try to get here?" Walk-forward alone would have waved it through. The correction caught it. The second is explicit falsification of individual filters. Pardo would accept a multi-filter system if the whole thing held up out of sample. I no longer trust that. Now every filter has to earn its place independently: turn it off, see if the edge survives, and run a cohort designed to break it. The last time I did this on a candidate, two of the three filters I was about to ship were worthless. One was actively net-negative, one was redundant, and only the third was doing real work. The combined system passed walk-forward with all three on. The ablation is what told the truth. So the scorecard is: match on the classic pillars, ahead on multiple-testing and falsification. But I owe Pardo one honest concession, because a self-audit that only flatters you is not an audit. He has a clean single-number smell test that I do not compute explicitly: walk-forward efficiency, the ratio of out-of-sample profit to in-sample profit. If your out-of-sample performance is less than roughly half of in-sample, you are overfit, even if most of your folds are technically positive. I report richer information than that ratio, fold win rates and out-of-sample significance, but I do not produce that one compact degradation number, and there is value in a single scalar you can glance at and feel your stomach drop. I am going to add it. It costs nothing and it is a good gut check.

The broader takeaway, and the reason I wrote this up: the discipline that separates a real edge from a flattering backtest has not fundamentally changed since 1992. Pardo had it right. What has changed is that we now generate hypotheses by the hundred, which means the ways to fool yourself have multiplied, and the correction has to multiply with them. Walk-forward is necessary. It is not sufficient. The honest version of the process is Pardo plus a hard accounting of how many times you rolled the dice before you got the number you wanted. If you build systems and you have never deliberately tried to falsify your own filters or correct for how many variants you tested, that is the gap. Close it before the market closes it for you.

Realise I have been quiet, so thought I should do an update. Today was a good day , and yesterday also fired off a winner. I didn't trade last Friday and missed the whole meltdown move, but as it turns out the data that was collected showed no trades would have been triggered, so I missed nothing. The reason for not trading wasn't an act of genius that predicted market turmoil; I honestly forgot to turn the algo on after it had been forced to quit the day before.

The reason for the force quit (after market close) was that the Ai found some bugs, but it wasn't until the weekend that some serious bug squashing happened. Again, I had been living with a few ticking time bombs, which meant that I was not getting good trades set up or completed. It unfortunately took a few weird occurrences to be logged before the problem was brought to light. Since the fixes, we have had no known new problems occur. Though I can't help but feel there is still another Gremlin waiting to emerge.

Whilst the AI was running through its own code, trying to fix its own mistakes, I was doing some research into a new trader (who has been around since the 90's) who is on YouTube and does scalp. I was compelled to run a deep dive on him and back-engineer a lot of what he shows on screen. The purpose was to try to be a better trader for it, so I am now using my version of what he shows with some AI-augmented additions. This is being used on my Lucid Account, which I have kept discretionary, so the Topstep can be Algo run without me needing to touch it.

very cool, is it live anywhere?

hi, the Topstep Algo is Trading an XFA account. I am discretionary scalping a Lucid account

Having 9 years of data to backtest still doesn't cover eventualities where the US President triggers the algos of the institutional buyers. Would be super useful if all elected officials had their social media accounts removed, as per what they are trying to do with under 16-year-olds.

I built this today to start a quantitative research project for the NQ. It's not yet fully useful, but the basic idea is done. I am just looking into ways to make it tradeable. If it is tradeable, then it becomes useful. Anything else is just more noise!

Is it going to be public ?

Yes, if it is useful to me, I may as well share it. The one thing it has to do is tell me who is in control. I doubt it would be able to deal with tweets from Trump, but it should be able to get down to 10-min chunks of who is winning, Bulls or Bears. I may have said it before in previous posts, I have learned the hard way that taking a trade is easy, following a setup is easy, working out whether the market is Bullish or Bearish is actually quite hard, and only easy in hindsight with most traditional methods. And as 99% of what is on this website is discretionary trading and not programmable, most traders get chopped up by following a strategy in the wrong market conditions. I should know, because I am one of those guys too.

the wrong market conditions point is interesting because the strategy itself does not change, but the trader's ability to recognize when to apply it does. that recognition tends to break down exactly when it matters most, after a drawdown, in choppy conditions, when you need the trade to work. do you track what your state is like when you misread the conditions versus when you call it right?

yes.

In fact, yesterday I had to manually intervene. When I queried whether the current regime conditions had been present in the 6-9 years of backtest data, the answer came back as an emphatic no. The reason I felt this current market was different from others we had seen since the COVID days was that the amount of drawdown i was seeing occur. My intuition was correct, but the data backed it up. The net result was that I was able to make an informed decision on whether to let the strategies bleed out or whether to pull the plug, or, as I did, step in and recover the accounts. There have been DD periods where I asked the same question and the AI came back with "this is within expected outcomes", so I left it alone.

Now that we have a totally novel regime (in recent history), we were able to make an adjustment to the code, but before i did that, I had to teach it that when this type of drift occurs, we have to protect the capital more. So we now have a line in the sand that when the balance goes lower than the threshold, we only trade the current A+ setups and not every strategy gets to participate. My idea is to generally not manually intervene, because they are designed to recover if we leave them alone, but that is based on a different type of market from what we have now. My idea is to put the brakes on the strategies that are really suffering when the market regime is novel.

Hope that makes sense.