Bawbie, I think I'd give a pass for the stocking hat, unless you're going hipster and wearing it inside.
The philosophy is that if you're head to toe in one team gear, it looks like you're wearing a uniform or something, and that you're on the team or part of the organization. Since you're not, it's like you're playing dress-up. Kids can play dress up, a grown-*** man should not. An exception is given for game days when you're not in the general public, but I'll admit I'd even struggle with feeling weird getting to and from the game that way.
WX: as a student manager in college, you'd be fine. As a grown-*** man, you're not.

wink:
Really-though...to each his own, but personally, I do think it looks more grown-up to be a little more subtle than going full on with the 'walking billboard' look. I'd even be hesitant to wear an ISU shirt with a cardinals hat, I think I'd just do one or the other.