To be fair, the *real* reason "turn it off and turn it back on again" works is more interesting than that. It's because systems are complex, and the more changes you make to a system, the less likely it is that the programmer foresaw the exact set of changes that led to whatever configuration it's in at the moment.
Programmers are pretty good at testing state 0 though (i.e. when you turned it on). So returning to that known state usually makes everything better.