Credit: Jesse Grant/

Levi Johnston says he is planning to file for legal joint custody of his 10-month-old son with former fiancee Brisol Palin.

Johnston, 19, has claimed in the past that the Palin family has prevented him from being alone with son Tripp. And things have only gotten worse.

"Not working. I'm done," he tells The Insider. "It's going to have to go to court. They just finally pushed me over the edge."

Johnston's feud with the Palin family heated up last month when he said on CBS's The Early Show that Sarah Palin referred to her infant son Trig, who has Down syndrome, as "retarded."

The former Alaska Gov. shot back, saying: "We, like many, are appalled at the inflammatory statements being made or implied. Trig is our 'blessed little angel' who knows it and is lovingly called that every day of his life."

She also slammed Johnston for stripping down for Playgirl (the shoot takes place next week): "Those who would sell their body for money reflect a desperate need for attention and are likely to say and do anything for even more attention."