Pants used to be more like chaps, a separate article for each leg, that were fastened together about the waist. Other garments covered your bits and butt over and under the pants. But that's why they're called a pair, even if they're permanently attached now.