gender-based fashion question

Why is it that stripes and plaid are gender-neutral, but polka-dots are uniformly feminine? Is there anything particularly female about dots? I mean, they are round - is that all it takes to be off-limit for male clothes (except, I guess, neckties)?