Our ancestors worshiped the idols of those people, which resulted in their being destroyed/exiled/taken to another country [MET].