Because the hydrogen atoms we care about are bound in galactic orbit, and that means they have a relatively low maximum relative velocity to us. If they weren't, they would all fly out into the intergalactic void in geologically short timescales, star formation would cease, and we'd have no such radiation problems after all.