And in very uncharacteristic style, the sharp-tongued Brit got teary-eyed as he recalled the fun and laughter they'd shared on the set of the hit Fox TV show.
Having vowed he wouldn't return, Cowell, 56, took his former work pals, including host Ryan Seacrest, completely by surprise as he sneaked up on them while they said their goodbyes in front of the studio audience.
And it wouldn't have been a proper Cowell Idol moment without a bit of bickering with Abdul!
"I just thought this would be a good opportunity on behalf of Randy and myself for Paula to apologize over the years for being so mean to the contestants," Simon joked, taking the microphone after his shock arrival. "When we were trying to be nurturing, helpful, now is the time for you to apologize."
Taking the joke in good spirits, Abdul, 53, claimed innocence, before Cowell swapped his teasing for some (unusually emotional!) sincerity.
"I'd like to say on behalf of us, thank you, America, for inviting us into your homes, thank you to the contestants … and," he added, gesturing to his pals on stage, "for you guys being actually my best friends now, and I'm going to miss you. I'm feeling quite emotional now."
Watch the touching moment in the clip above.
Sign up now for the Us Weekly newsletter to get breaking celebrity news, hot pics and more delivered straight to your inbox!