For a ben Adam indeed ought not to be covered on the rosh, being the demut HASHEM and the kavod Hashem, and the Isha being the kavod (glory), the glorious reflection of Adam.