Joseph took Mary with him. She was his promised wife, and she was carrying a baby inside her at that time.