Vibe Trading

https://www.rfastrat.com/module1 I haven't updated the site in the last couple of weeks, but I have tested over 100 hypotheses and a lot more variations.

bad day at the office today

someone asked if I backtested?

Are we actually better than Pardo? A self-audit

I went down a rabbit hole this week that I want to write up, partly to organize my own thinking and partly because I suspect a lot of people building systematic strategies have never actually checked their methodology against the person who wrote the book on it. That person is Robert Pardo. If you trade mechanical systems and you have never heard the name, you have almost certainly used his ideas anyway. Pardo is a CTA who has been running systematic futures since the early 1980s, but his real footprint is methodological. His book, "The Evaluation and Optimization of Trading Strategies" (2008, the rewrite of his 1992 original), is the canonical text on how to test a system without fooling yourself. The single biggest idea in it is walk-forward analysis: optimize your parameters on an in-sample window, then trade them unchanged on the next out-of-sample window, roll forward, repeat, and stitch the out-of-sample segments into one continuous equity curve. If the system only works on the data you fit it to, walk-forward exposes it. If it survives, you have something closer to real. So the question I asked myself was simple: is my process as good as his, or better? Not "do I feel confident," but actually mapped, pillar by pillar.

Here is where I landed.

On the core pillars, I match him. I run walk-forward folds on every hypothesis. I demand a parameter plateau rather than a lone profitable spike, which is exactly his robustness argument restated as a plus-or-minus-20-percent stability gate. I require large samples and few parameters because every degree of freedom you add is another knob the optimiser can use to memorise noise. I bake in realistic costs: slippage, commissions, and on top of that the specific drawdown rules of the funded-account platform I trade. That last part is arguably stricter than Pardo's generic cost treatment, because my constraints are not negotiable. A strategy that backtests beautifully but violates a trailing drawdown rule is dead on arrival, and my simulator knows it. Then there are two places where I think I am genuinely ahead of him, and I want to be careful here because it is not because I am smarter. It is because the field learned two painful lessons after his book was written.

The first is multiple-comparison correction. Pardo's framework has no real defense against the modern failure mode, which is: I generate eleven variants of an idea, walk-forward-test all of them, and the best one looks fantastic. Of course, it does. With enough variants, something will pass walk-forward by luck alone. The fix is to penalize for the number of tests, a Bonferroni-style threshold. This is not academic for me. It has killed live candidates. One pullback strategy looked like a clean winner on its best variant and died the moment I asked, "but how many variants did you try to get here?" Walk-forward alone would have waved it through. The correction caught it. The second is explicit falsification of individual filters. Pardo would accept a multi-filter system if the whole thing held up out of sample. I no longer trust that. Now every filter has to earn its place independently: turn it off, see if the edge survives, and run a cohort designed to break it. The last time I did this on a candidate, two of the three filters I was about to ship were worthless. One was actively net-negative, one was redundant, and only the third was doing real work. The combined system passed walk-forward with all three on. The ablation is what told the truth. So the scorecard is: match on the classic pillars, ahead on multiple-testing and falsification. But I owe Pardo one honest concession, because a self-audit that only flatters you is not an audit. He has a clean single-number smell test that I do not compute explicitly: walk-forward efficiency, the ratio of out-of-sample profit to in-sample profit. If your out-of-sample performance is less than roughly half of in-sample, you are overfit, even if most of your folds are technically positive. I report richer information than that ratio, fold win rates and out-of-sample significance, but I do not produce that one compact degradation number, and there is value in a single scalar you can glance at and feel your stomach drop. I am going to add it. It costs nothing and it is a good gut check.

The broader takeaway, and the reason I wrote this up: the discipline that separates a real edge from a flattering backtest has not fundamentally changed since 1992. Pardo had it right. What has changed is that we now generate hypotheses by the hundred, which means the ways to fool yourself have multiplied, and the correction has to multiply with them. Walk-forward is necessary. It is not sufficient. The honest version of the process is Pardo plus a hard accounting of how many times you rolled the dice before you got the number you wanted. If you build systems and you have never deliberately tried to falsify your own filters or correct for how many variants you tested, that is the gap. Close it before the market closes it for you.

Realise I have been quiet, so thought I should do an update. Today was a good day , and yesterday also fired off a winner. I didn't trade last Friday and missed the whole meltdown move, but as it turns out the data that was collected showed no trades would have been triggered, so I missed nothing. The reason for not trading wasn't an act of genious that predicted market turmoil; I honestly forgot to turn the algo on after it had been forced to quit the day before.

The reason for the force quit (after market close) was that the Ai found some bugs, but it wasn't until the weekend that some serious bug squashing happened. Again, I had been living with a few ticking time bombs, which meant that I was not getting good trades set up or completed. It unfortunately took a few weird occurrences to be logged before the problem was brought to light. Since the fixes, we have had no known new problems occur. Though I can't help but feel there is still another Gremlin waiting to emerge.

Whilst the AI was running through its own code, trying to fix its own mistakes, I was doing some research into a new trader (who has been around since the 90's) who is on YouTube and does scalp. I was compelled to run a deep dive on him and back-engineer a lot of what he shows on screen. The purpose was to try and be a better trader for it, so I am now using my version of what he shows with some Ai augmented additions. This is being used on my Lucid Account which I have kept discretionary, so the Topstep can be Algo run without me needing to to touch it.