Credit: Jordan Strauss/

When Simon Cowell let Paula Abdul go from The X Factor's judging panel, it was strictly business -- and not at all personal.

In fact, the reality TV mogul says he was overruled when it came to cutting Abdul loose along with mentor Nicole Scherzinger and host Steve Jones.

"I would've liked to have kept her," Cowell, 52, tells Extra of his longtime colleague Abdul, 49, with whom he judged American Idol for eight seasons. "She was very gracious and I said that to her. She understands it's business; it's never personal."

With Abdul and Scherzinger given the axe, Antonio "L.A." Reid is the only judge left to sit next to Cowell for season two until the vacancies are filled. "There's not a record executive out there who is as good as him for the job," Cowell confirms of the Epic Records head.

In terms of potential new judges, Cowell laughed off reports that he offered new mom Beyonce $100 million to take part. "I have no idea where this rumor came from," he told Extra. "It's complete nonsense."

So who is on Cowell's wish list? He tells Extra Madonna could be "great" but "expensive" and that newly single Katy Perry "would be fun -- and she's feisty."

Tell Us: Who would you like to see as a judge on The X Factor?