After a tearful and at times humorous funeral at a New York synagogue, the flag-draped casket of New Jersey Democratic Sen. Frank Lautenberg has started the trip towards his final resting place.

Lautenberg's casket will be brought via Amtrak, a service which he championed fiercely over the years, from Secaucus, N.J., to Washington, D.C., this afternoon.

The love of both places where he spent his life and career - New Jersey and Washington - was underscored in the eulogies given by the Lautenberg family and from top Democratic political leaders, former Secretary of State Hillary Clinton, Sen. Bob Menendez, D-N.J., and Vice President Joe Biden.

"As Frank would say it's not where you sit that counts but it's where you stand and there's never any doubt where he stood," Secretary Clinton said recalling their days in the Senate together.

The senator passed away Monday at the age of 89 due to complications from viral pneumonia after a 28-year career in the Senate, holding the title for the last actively serving veteran of World War II in the Senate.

"Frank had great character, an exceptional character," Biden, who served with him for years in the Senate said. "We saw that in the way he lived his life but by how he died - serving the people of New Jersey until the very end."

The senator recently announced that he would retire and not run again in his 2014 reelection race, a decision the vice president said the senator took very seriously and shared some of their personal discussions.

"He said 'Joe I don't think I can run - my legs,' He said 'my legs.' And it was clear to me he desperately, desperately wanted to run again," Biden said. "And I think the reason is not because he wanted to be senator, but your dad never quit. He never quit anything. He never gave up. He never gave in."

Although he had been absent from the Senate for some time dealing with his illness, the senator voted on two last key votes and was taking meetings with other senators as recent as two weeks ago. The senator's children said he had expressed regret recently announcing in February of this year that he would retire and not seek another term.

"On days that were good he would say 'I never should have made that retirement speech'," his daughter Ellen Lautenberg said to laughter from the crowd at the tenacious Senator.

The senator's rabbi said just recently when the senator became even more ill he had a discussion about the end of his life. The senator told his rabbi he'd like to be remembered as someone who came from humble beginnings and "wanted to do good."

Menendez told his wife that he may have loved Lady Gaga's music but he really loved his wife.

"Even Lady Gaga couldn't' lift a finger to you," he told Lautenberg's widow to laughter from the crowd.

Forty-one senators and six members of the House of Representatives attended the funeral today, as well as New Jersey Gov. Chris Christie.

The senator's casket will lie in repose on the Lincoln Catafalque in the Capitol on Thursday and he will be buried Friday at Arlington National Cemetery.