Charlie Hunnam Credit: Jon Kopaloff/FilmMagic

It's tough being the master of the universe. Charlie Hunnam provided yet another explanation as to why he bailed on playing Christian Grey in the Fifty Shades of Grey movie. According to the Sons of Anarchy hunk, scheduling conflicts — and anxiety issues — were big contributors that led to him gracefully bowing out of the role.

"I wouldn't have signed up in the first place if I didn't want to do it," Hunnam, 34, told Us Weekly at a Television Critics' Association party last month. "It was a really, really difficult time in my life. I had already been attached to Crimson Peak for about 14 months, which I said, I was going to do with Guillermo [del Toro]."

Hunnam agreed to portray E.L. James' sexy billionaire in the film adaptation after meeting with director Sam Taylor-Johnson. "Sam and I just fell in love, creatively," Hunnam recalled. "We started talking about this thing and I just felt so excited. I fell in love with the character and started seeing him and becoming intoxicated, which is the way I work: I find a character I love and I start thinking about it all the time."

Unfortunately for Hunnam, he was playing three characters during that time period. "I was going to finish [filming] Sons at like 11 p.m. Friday night, get on the plane Saturday morning to Vancouver for Fifty, missing the whole first week of rehearsal and start shooting Monday morning," Hunnam recalled. "And I was going to shoot that film, wrap that on the Wednesday and the following Monday I was going to start shooting Crimson Peak in Toronto. I just had like ... frankly, something of a nervous breakdown."

Two months after Hunnam was cast as Grey, the actor dropped out. "I just said, 'I can't ... I can't ...,'" he told Us. "To fail on such a grand scale — because Fifty's going to be massive, it's going to be huge. I really didn't want to fail on such a grand scale, and I just couldn't transition from Jax Teller to Christian Grey in 48 hours. It's painful because I loved the character and I wanted to do it. I'm also a man of my word."

Once Upon a Time hunk Jamie Dornan was cast instead in Hunnam's place, opposite Dakota Johnson as heroine Anastasia Steele. The decision, though, was still met with protests. "It was just a s--- situation," Hunnam recalled of bailing on the role. "I was really, really sorry to do it. It was one of those damned if you do, damned if you don't situations." See behind-the-scenes pics from the Fifty Shades set.