Because he should have remained in his ir miklat until the mot HaKohen HaGadol; but after the mot HaKohen HaGadol the rotze'ach shall return into his eretz achuzzat.