And Yosef was there until the mot Herod. This was to fulfill what had been spoken by Hashem through the Navi, UMIMITZRAYIM KARATI LIVNI (Out of Egypt have I called my Son).