"He announced he's leaving on my first day!" the new judge says in an episode of The Ellen DeGeneres Show to air Tuesday. "I'm trying not to take it personally."
DeGeneres continues, "But seriously, I am going to be very, very sad to see him go, because I think he's made the show what it is. He's a huge part of that show but he wanted a change.
"I wish him all the luck in the world hosting The Tonight Show," she quips.
Cowell announced Monday that the 9th season — which debuts Tuesday on Fox — would be his last. He plans to focus on bringing his UK hit The X Factor stateside.
"I felt like doing something different," Cowell explained at Fox's Winter Television Critics' Association press event in Pasadena, Calif. "I want to leave Idol this year bigger and better than it's been before."
Sign up now for the Us Weekly newsletter to get breaking celebrity news, hot pics and more delivered straight to your inbox!