The more expensive the clothes, the more "forgiving" the sizes. A 10 at Walmart could be a 2/4 at Saks. Designers aren't stupid, even though their clothes may be off the wall weird.
When clothes are purchased at Talbots or some similar type brand like Land's End, Eddie Bauer, etc, who were at one time almost exclusively mail order through catalog, the catalogs had in the middle a size chart. So many inches here, was one size, there another. Women measured their bust, waist, hips and it all correlated with a specific size. Petite pants had a 25-27" inseam, regulars were 29-30", longs were 32"+. Any woman under 5'2" was considered a "petite".
Now days, 5'4" is considered petite. But I'm not even sure that's right either.
I'm 5'4" and have such a hard time finding clothes to fit. I used to buy longs in pants, now I buy regulars and sometimes even petites for a skinny straight leg, and they are long enough in the stride. Regular dresses don't fit at all, and petites are too short in the sleeves and just don't fit properly. And I have no clue how tops are being sized anymore. I used to wear a small, now I'm anywhere from a medium to an XL in order to not have them skin tight through the chest, but then they are so huge through the shoulders and sleeves, and there is so much fabric I can't tuck it without looking like a pillow stuffed into the pants.

I think men's clothes are so easy to buy! I pick up stuff all the time for M and for my son. Waist is one size and match the length to it for pants, shirts all I have to do is match the neck/sleeve length or just the neck and the rest will fit. It's not fair!!!

I think it also has to do with the fact that so many clothes are made in SE Asia, Phillipines, and China...all those people are midget size compared to many Americans.