"I think he sexually assaulted a child, and I don't think that's right," Sarandon, 69, said during Variety and Kering's Women in Motion talk with Thelma & Louise costar Geena Davis.
"I have nothing good to say about him," the Oscar winner added. "I don't want to go there."
The director has been a topic of discussion at the film festival this week after French comedian Laurent Lafitte made a joke at the premiere of Allen's new film, Café Society, that appeared to reference allegations that the Oscar winner sexually abused his stepdaughter Dylan Farrow.
Referring to director Roman Polanski, who fled to Europe after pleading guilty to unlawful sex with a 13-year-old girl in 1977, master of ceremonies Lafitte addressed Allen saying, "You've shot so many of your films here in Europe, and yet in the U.S. you haven't even been convicted of rape."
The jab came days after Ronan Farrow, Allen's biological son with ex Mia Farrow, penned a column for The Hollywood Reporter questioning why the media hadn't thoroughly investigated the allegations that Allen abused Dylan, Ronan's sibling, when she was just 7 years old.
Dylan chronicled her memories of sexual abuse for a New York Times story in 2014, but a lawyer for Allen denied the claims.
"We are witnessing a sea change in how we talk about sexual assault and abuse," Ronan wrote in his column on May 11. "But there is more work to do to build a culture where women like my sister are no longer treated as if they are invisible. It's time to ask some hard questions."
Sign up now for the Us Weekly newsletter to get breaking celebrity news, hot pics and more delivered straight to your inbox!