Kelsey Grammer, Wife Kayte Welcome Baby Girl Faith Evangeline Elisa

Actor Kelsey Grammer (R) and Kayte Walsh arrive at the 69th Annual Golden Globe Awards held at the Beverly Hilton Hotel on January 15, 2012 in Beverly Hills, California. Credit: Jason Merritt/Getty Images

Even as they celebrate the newest addition to their family, Kelsey and Kayte Grammer are mourning the loss of her twin.

The couple of two years reveals that the Friday morning arrival of Kelsey and Kayte's new daughter, Faith Evangeline Elisa, was bittersweet.

"We are thrilled," the couple tells Us Weekly in a statement about their baby girl, who weighed in at 6lbs, 2 oz. at birth. "Mother and child are in excellent health."

Earlier this year, the pair had announced Kayte was carrying twins.

"Tragically, we lost the little boy shortly thereafter," the couple says. "This was not something we cared to make known publicly at the time. It was unspeakably painful and we know that people will understand our desire to keep the news private then, as we know they will respect our privacy in this matter now."

Still, the Grammers -- who married in February 2011 -- say they're focused on moving forward from their tragic loss.

"A glorious birth with a lingering sadness is ours today. We choose to celebrate the life that has been given us," they share. "We proudly announce Faith to the world today looking forward to the days ahead and the children yet to come."