Steady on Nick - take a couple of deep breaths and you'll be fine!
I really wouldn't worry about what you wear - they'll be much more interested in the person than the clothes and I expect that, like Stu, most people's dress will depend on what else they're doing that day before or after the interview. If it were me I wouldn't buy any clothes I wasn't planning on buying anyway.
And as for the X-Factor comparison I wouldn't read too much into that. People get lazy these days and describe things by comparison to other things. This is a reality TV show and, from what I've read, there is an element of competition involved, but I think the similarity ends there.
Try to approach the whole thing as a novel experience and not worry too much about the end result - easier said than done I know. And remember there's a lot of people who'll be rooting for you on Friday and who won't think one iota less of you if it doesn't happen.
Now have a Mrs Fish cake (

) and an early night...
Dave P.