Birkhead said that when he ended the relationship, Smith told him that he might "feel really dumb" in a couple of months, but wouldn't further elaborate. He suspected she was pregnant -- which turned out to be the case -- and said he felt certain he was the father.
By May, Smith's pregnancy had leaked to the tabloid press -- with Birkhead mentioned as the father. Birkhead said this irritated Howard K. Stern.
Not long afterwards, Birkhead said, Stern and Smith fled to South Carolina to get help from Ben Thompson, the wealthy developer with whom she had had a brief fling. He was buying a house in the Bahamas, which turned out to be a perfect place for Smith to assert her parental rights over Birkhead.
Thompson set Smith and Stern up in a house called "Horizons," which is where they have been living since the summer. Birkhead has insisted it was a calculated plan to keep him out of the picture.
"She was telling people down there that I was a stalker and that it (the baby) was not my child, and that I wouldn't leave her alone," he said. "And she was telling them it was their child, that it was Ben Thompson's child."
But that couldn't be, according to a sworn statement filed in the Bahamas by Thompson, which states that he underwent a vasectomy years ago. Thompson also states in the declaration that Smith "admitted to me that Larry Birkhead was, in fact, the father of her child."
That was not known on Sept. 7, 2006, when Anna Nicole Smith gave birth to a baby girl at a hospital in the Bahamas. Two days later, her son Daniel flew down, and soon after, Birkhead got a disturbing phone call.
"A friend of my sister's in the Bahamas told me that (Smith) had given birth to the baby … but somebody had died," he said. "And I, I thought maybe she died. I didn't know what happened. I said, 'Can somebody help find out what happened?'"
"Before it hit the news, I found out that it was Daniel," Birkhead continued. "Then people started saying, 'Congratulations, the baby -- the baby's born. And I said, 'What happened to Daniel?'"
A forensic pathologist hired by the family later concluded that Daniel Smith died accidentally after taking a lethal combination of two anti-depressants and methadone.
Two weeks later, Howard K. Stern went on "Larry King Live" to talk about the recent events and to make a surprising claim: He said he was the father of Smith's baby.
Birkhead told "20/20's" Deborah Roberts that Smith and Stern never had a romantic relationship -- or even casual sex.
"Howard is seeing this as an opportunity because her phone is ringing," he said. "People want to interview [her] and pay big bucks for photos."
In fact, Smith has cashed in on her recent notoriety, getting a reported $400,000 for photos of Daniel in her hospital room the day before he died, and up to $1 million from an interview on "Entertainment Tonight" that included home video footage of her giving birth.
Birkhead said he wants to be a parent to the baby he believes is his daughter.
"I want a life with my child that we were planning on … I'm not just going to act like we didn't have a child and go away," he said
Debra Opri, Birkhead's attorney, has filed a lawsuit demanding a DNA sample from the baby for a paternity test.