Pretty Little Liars returns for a fourth season on June 11 -- and according to series star Keegan Allen, it's going to be a nail-biter! The 23-year-old actor recently visited Us Weekly's New York City offices to discuss the evolution of his character Toby Cavanaugh's relationship with Spencer Hastings (Troian Bellisario) and how his secret past with Alison DiLaurentis (Sasha Pieterse) will impact their future together.

Throughout most of Season 3, Spencer believed her boyfriend had betrayed her by working with the A-Team. Towards the end, Allen recalled, "It got revealed that he was helping Spencer. He was trying to go in and sleuth for her, and it kind of backfired in his face."

Spencer believes Toby's intentions are pure, Allen explained. "They are together," he said of the TV couple, "but as every season of Pretty Little Liars starts, houses of glass will break."

As Spencer and her friends (played by Lucy Hale, Shay Mitchell and Ashley Benson) try to identify the mysterious Red Coat in Season 4, secrets from Toby's past will come to light. "You'll be seeing some flashbacks with Toby and Alison that kind of explain why he didn't like the girls that much, and especially why he hated Alison," Allen explained. "He doesn't really have any friends and he really trusts a lot in Spencer. So when he was on the A-Team, he kind of lost everybody and lost everything."

Not wanting to spoil any secrets, Allen cryptically told Us that Toby still "has an axe to grind with the girls."

For more of what to expect during the show's fourth season, watch the video above now. Pretty Little Liars returns Tuesday, June 11, at 8 p.m. EST on ABC Family.