Because, if you did that, those people would cause your children to stop worshiping Yahweh and to worship other gods. If that happens, Yahweh will be very angry with you and he will get rid of you immediately.