Sorry, not sorry. Jeffrey Dean Morgan says in a new behind-the-scenes interview that fans of The Walking Dead are sure to hate him after watching the Sunday, October 23, premiere of the hit AMC show.

“I’ve always kind of been the guy that people like on shows, so this has been the first time that I’ve done something that I just know people are going to despise me,” he says in the interview with a smile.

But despite the inevitable backlash that he’ll garner from the season premiere, Morgan, 50, says that he’s having the best time playing Negan in the horror drama.

“Having been a fan of the show and the comic book for so long, I sort of had an idea of what Negan was going to be and no one seemed to stop what I was doing,” he says. “But it’s tricky bringing him to life. That being said, I’ve never had more fun. I couldn’t be happier.”

On the zombie-apocalypse series, Morgan plays an outsider who wields a barbed wire–covered baseball bat, and who has come to lead a group called the Saviors. The actor told Entertainment Weekly that the relationship his character has with the others is “odd,” at best.

“A lot of them, I think, follow him purely out of fear,” Morgan told the publication. “I think that Negan has kept these people alive, certainly.”

“After the apocalypse hits and the zombies start taking over the world, you see Negan trying to help people,” Morgan explained of the 48-page preamble creator Robert Kirkman gave him to understand the character a bit more. “And no one listens to him, and they get killed, and so finally he gets to the point that he becomes a little bit of a dictator.”

“Negan’s always a foulmouthed son of a bitch, so he always is colorful with his language, but I think you see him kind of make a shift because he starts off really trying to help people and save people, and when they don’t listen to him and they’re constantly getting killed, he becomes a little bit more brutal in his tactics, I guess,” Morgan concluded.

The Walking Dead airs on AMC Sundays at 9 p.m. ET.

Want stories like these delivered straight to your inbox? Sign up now for the Us Weekly newsletter!