Biden said he's still uneasy about the White House butlers and other household staff doing things for him.
"I don’t know about you all, but I was raised in a way that you didn’t look for anybody to wait on you," he said. "And it’s where I find myself extremely self-conscious for wonderful people who work in the White House. But someone standing there and hands me my suit coat."
Remarking it was likely he would "get in trouble here" for saying it, he admitted living in the White House is a bit like living in a "gilded cage, in terms of being able to walk outside and do things."
He described much more fondly his time at the vice president's official residence at the Naval Observatory, a compound of 80 acres complete with a swimming pool and a place where he said he could wander the grounds freely.
He said that the White House is "very different" and has a "sense of history" to it, mentioning the Lincoln bedroom.
Biden also said that during his eight-year tenure as vice president with President Barack Obama, he had never been in the upstairs residence.
"What I didn't realize, I had been in the Oval Office a hundred times as vice president -- more than that -- every morning for the initial meetings," Biden said. "But I had never been up in the residence."
However, former White House photographer Pete Souza, who documented the eight years of Obama's time at 1600 Pennsylvania Avenue through a camera lens, posted a photo on Instagram of Biden in the residence with Obama in 2011.
He later took to Twitter Tuesday night to say that Biden had been in the residence "dozens of times," although he said Biden may have meant that he had not been in all parts of the private living quarters.
"He was in the residence dozens of times," Souza said in a tweet responding to a journalist. "I think he meant he was never in the bedrooms or kitchen or solarium etc."
ABC News' Justin Gomez contributed to this report.