Believe it or not, Simon Cowell is blaming himself (more specifically his big mouth) for the big shakeup at The X-Factor. Since the firing of Paula Abdul and the disappointment that has plagued Paula’s fans, Simon has felt…well, responsible because he was the one who set unreasonable expectations for the show.

Cowell told Extra, “I would’ve liked to have kept [Paula]. She was very gracious, and I said that to her. She understands it’s business; it’s never personal.”

And as far as letting go Scherzinger? He really doesn’t think she’s punching her fist through walls. “I don’t think she’s mad. I think she’s disappointed.” Well, yeah. Unfortunately it was her big mouth and her actions that got her canned.

So why in the hell did he have to shake up the roster? Cowell explains, “It’s only because I opened my big mouth and said we were going to get 20 million [viewers]. If I hadn’t said that, everybody would be saying the show was a huge success.”

And why did he decide to ONLY keep L.A. Reid? As far as Cowell is concerned, “There’s not a record executive out there who is as good as him for the job.” True. I adore Reid.

Here is what he said about potential new judges:

As for the Beyonce rumors, they are “complete nonsense.”

He would love to have Madonna but she would be “expensive” (can you imagine the salary she’d be asking for? Cripes, she is charging her fans $300 to see her live in concert)!

So what about Katy Perry? Cowell thinks she would “fun” and “feisty.”

Even though Nick Cannon pretty much shot down rumors that his wife Mariah Carey would take on the X-Factor judge role, Simon says the door is still open. He said, “I would love her. but you can’t expect someone who’s just had two kids to make the kind of commitment you would need on this show.” But oh how we would LOVE her, despite her being too legendary for the show. Wink, wink.

Image via Getty Images