Rob Lowe is leaving “Brothers & Sisters”

Rob Lowe, who left NBC’s The West Wing in the middle of its run, is also leaving ABC’s Brothers & Sisters early.

That makes even more interesting a remark Lowe recently made to outgoing Tonight Show host Conan O’Brien when he guested on the show: “As someone who has left a celebrated NBC franchise myself… it can be done,” Lowe said, referencing O’Brien losing his Tonight hosting gig. “Believe me, if my experience can you give you anything, it’s that in a couple of years, nobody will even remember The Tonight Show, just like they don’t remember The West Wing.”

Lowe, who plays U.S. Sen. Roibert McCallister on Brothers & Sisters, is said to have felt underutilized this season – his fourth on the show. His character had a heart attack last season but this year has mostly been helping wife Kitty (Calista Flockhart) battle cancer.

Last season the couple had some serious marital problems but were closer than ever this year. Speculation is the Lowe’s character might be killed off – perhaps a fatal heart attack given his health history.

However his exit is handled, Lowe will likely remain at ABC with a show that is more built around the former Brat Pack member whose many film credits include St. Elmo’s Fire, Wayne’s World, Austin Powers: The Spy Who Shagged Me, About Last Night, Class, About Last Night, Square Dance, The Hotel New Hampshire, a remake of Suddenly Last Summer and the recent comedy The Invention of Lying.

“We would like to keep Rob at ABC,” said network president Steve McPherson in a statement. “We feel he should be anchoring his own franchise. We are in talks now.”

Lowe has appeared in two major hit shows and also had the leads to the successful miniseries The Stand and Salem’s Lot. But the two series built around him – Lyons Den on NBC and Dr. Vegas on CBS – both lasted less than a season. He also turned down the part of  Dr. McDreamy on Grey’s Anatomy that made Patrick Dempsey a hot star again.

Ironically, there is speculation that Lowe could find himself working for Grey’s creator Shondra Rhimes after all in a new medical drama she is creating for ABC.