There's always an extra air of confidence present when you wear something unconventional. Or at least there should be.
For example, when you're in highschool or university and start dating a new guy and you wear his watch. Designers always like to capture a bit of nostalgia and often recreate this look and feeling as well. French greats Dior and Chanel both have mastered the "boyfriend" watch.
Some things are truly unisex, like fragrances from Creed, Tom Ford, and Calvin Klein. American Apparel has an entire unisex section. And many scarves can be worn by both sexes.
Refined, a little rugged, and definitely bold, wearing something unconventional is a great way to spice up your wardrobe. Just be confident and natural. Accessories are supposed to accentuate, and not overpower an ensemble. Remember, Coco Chanel gave us one of the best pieces of advice on fashion - she said, "wear whatever you want, and just before you leave, take one thing off"...less is more, no matter how well you master a certain look or vibe.