Abraham was really upset by what Sarah said. Ishmael was his son, so Abraham reckoned it was wrong to send him away.