The family moved in with relatives, then found an old house in Lindale, Texas, on the verge of being torn down, and talked the owner into letting them live there in exchange for fixing it up.
"Some of the windows were boarded up, and we never did get central heat," Lambert recalled, laughing. "So when you had to go to the bathroom in the middle of the night, you had to run, because it was freezing!"
Room by room, her parents made it a home. They planted a big garden, hunted game, and provided most of their own food.
Even Lambert's more well-off friends loved to hang out there.
That home still seems to be where her heart is.
One of the most powerful songs on Lambert's CMA-nominated album, "Revolution," is a ballad called, "The House That Built Me."
Lambert didn't write the song, but as soon as she heard it, she knew she had to make it her own.
"I know they say you can't go home again," she sings. "I just had to come back one last time."
The song showcases a softer, gentler and more mature side of the fiery singer, whose sometimes angry lyrics prompted Esquire magazine to name her the "Most Terrifying Woman of the Year" in 2008.
"I feel like I've started to grow up and be more of a woman instead of this crazy girl," she told Roberts. "I sort of feel settled, so I'm kind of ready to move to the next step in life."
ABC News' Alan Weisman contributed to this story.