DislikedInstead of that DMB analysis, can't we use the standard train/cross-validate/test dataset splitting? Train on the train set, tune hyper-parameters on the validation set, test on the test set.Ignored
But even if you did the above you would either way need to do the DMB analysis to see if your cross-validation/testing results are also likely the result of random chance. This means that you would need to perform a DMB analysis by performing the entire search process train/cross-validation/testing on random price series, etc.
There is no free lunch, you cannot skip DMB analysis.