trying to figure out if my crypto signals are actually robust or just ahh

ive been working more on systematic crypto strategies lately and something thats been bothering me is how easy it is to get decent-looking backtests that dont really mean anything. like even with walk forward testing and basic cost assumptions, its still pretty easy to end up with something that looks stable but is probably just fitting to specific regimes or microstructure quirks in the data.

these days im trying to approach it more from a signal validation perspective instead of jumping straight into full strategy design. stuff like checking cross-sectional consistency, running the same feature logic across different assets without re-tuning, and looking at how sensitive the signal is to small parameter changes. ive also been experimenting with testing signals on platforms like alphanova where u can see how they behave on unseen data, and comparing that with more constrained setups like numerai just to sanity check if anything actually generalizes.

but still though it feels like the hard part isnt building signals but figuring out if theyre real in the first place. im in need of advice, thanks