Before he was a hotshot Oscar-nominated movie star, Jake Gyllenhaal was facing boatloads of bad auditions and rejection, he recalled on the Monday, March 21, episode of The Tonight Show Starring Jimmy Fallon.

The Brokeback Mountain actor, 35, sat down with Fallon, 41, to detail one of the worst audition experiences he’s had to date — one in which acclaimed director Peter Jackson chewed him out as the “worst actor” he’d ever seen.

Gyllenhaal explained that he was called in to do the audition via “a conference call from, like, 14 agents” who insisted that he would be great for the part of Frodo in the Lord of the Rings trilogy (the first film, The Fellowship of the Ring, premiered in 2001.)

“They called me up, and they were like, ‘Oh, guess what!’ And I was like ‘What!’” he recounted, imitating the gruff agents’ voices as they told him about the part. “‘… and when they said they needed a hobbit, we thought, Jake!’”

Once at the audition, however, Gyllenhaal explained that he was confused by the instructions, which featured “lots of stage directions” but “no lines.”

After he shuffled over to an imaginary chest and feigned picking up a ring, however, he said Jackson was less than impressed.

“Literally, Peter Jackson was like [facepalm],” Gyllenhaal said, adding that the director told him, point blank: “You are the worst actor that I have ever seen. Did anyone tell you you’re supposed to have an accent?”

Needless to say, Gyllenhaal didn’t get the part (it went to Elijah Wood), though he did go on to nab a nomination for a Best Supporting Actor Oscar in 2006 for his part in Ang Lee’s Brokeback Mountain.

See what other roles Gyllenhaal auditioned for and ultimately lost out on in the clip above!

Can't get enough of Us? Sign up now for the Us Weekly newsletter packed with the latest celeb news, hot pics and more!