19. Negative Discovery: The other side of a model

Years ago while pitching 4pm estimated prices for non-US stocks to replace the local closing prices to a mutual fund company president I said the estimated price is closer to next day’s open 65% of the time. He countered saying, doing so incorrectly modified the local price 35% of the time - and, moreover, local price had been  used since the beginning of the industry,  The company’s general counsel, however, supported my pitch with the reciprocal view that not using the estimate meant using the wrong price 65% of the time. I used this equivalent, but more convincing argument, in all subsequent meetings. Recalling this event, while reading Captain Cook and the art of negative discovery of the southern continent, made me reexamine my results.

I have focused on performance during 9 years of history when trades   occurred on 315 days - but why not also focus on non-model day performance.
​
​Consider these 1921 days. On them net return was -11.50%. (Some days had very large losses allowing 53% positive return on 53% of the days yet net for all days was negative.)

​The table shows this non-model day weakness has persisted through history.

Results are one-sided. Column 6 shows that in every year but one model day win ratio was higher by at least 12%. Column 9 shows that in every year, average daily return was at least .08% higher. Column 12 shows that in only 2 years was model net return not the highest - and this excess was achieved by trading on only 15% of the days. Particularly compelling is the full period  return difference (60.2% vs -11.5%)  in the last row. 

(Strong results but my physicist friend’s suggestion, why not  "Just buy-and-hold SPY" has merit. Its return over 2236 days (holding 24 hours/day) was 119% while model return over 315 days (holding only 4 hours/day) was 60.2%. Now time is he wrong variable to measure risk (Post 8) but still it's pleasant to earn such a return with so little time exposure. And a trade-war market collapse is more likely during the longer period.)

So is there any reason, besides the hassle of trading every day, to not use the model? One major concern is "data snooping."  The model is based on GDP-weighted momentum in the morning but I've refined the model to also depend on several other variables - volatility, ETF premium,... - which weren't obvious until I started testing. This snooping, looking a test results then refining the model, casts a shadow on it. To address this a pure statistician, whose papers are among the world's most referenced, has agreed to review my model for snooping. Results will be posted.