"Yesterday was a horrible day," the former Simon & Garfunkel musician, 75, tweeted Wednesday, December 28. "Carrie was a special, wonderful girl. It's too soon."
Simon and Fisher were married for one year from 1983 to 1984. After their divorce, however, they dated on and off for almost a decade.
Back in 2012, the Star Wars actress called Simon one of the loves of her life. "He's a great artist so that was great," she told the Today show's Hoda Kotb at the time. "I was quite young and so it was like being an apprentice to someone who was brilliant."
Fisher also had a high profile yet brief romance with Dan Aykroyd. The Ghostbusters actor, 64, proposed to Fisher on the set of their 1980 film The Blues Brothers. He paid tribute to Fisher Tuesday.
"Our beautiful brilliant funny wise kind and generous Blues Sister – so ironic – Xmas was Carrie's favorite thing next to babies," he tweeted.
"Our beautiful brilliant funny wise kind and generous Blues Sister -so ironic – Xmas was Carrie's favorite thing next to babies."
— Dan Aykroyd (@dan_aykroyd) December 27, 2016
The When Harry Met Sally actress, who welcomed daughter Billie Lourd with casting agent Bryan Lourd in 1992, joked about her former flames on Today. "If you could line up all of your previous relationships and you can ask them all, 'If you had to boil it down to one reason that this relationship didn't work… what would the common thread be?" Kotb asked.
"I think they would say that I was not sort of something — just too much. I'm not a typical wife," she replied. "If you look at me and think, 'Oh, that's my girlfriend' — maybe that flies. But if you look at me and say, 'Oh, this is my wife' — it looks weirder."
Sign up now for the Us Weekly newsletter to get breaking celebrity news, hot pics and more delivered straight to your inbox!