For more of the exclusive interview about Loughner's friendship with Zachary Olser, watch ABC's "Good Morning America" tomorrow.
Osler, who spent a lot of time at the Loughner's home during high school, described the family's home as "cold, dark and unpleasant" and added that he always felt "unwelcomed."
Littered with musical instruments and a computer in the middle of the room, Osler said he never observed a particularly loving relationship between the Loughners. Loughner once told Osler that he loved his pet dog Hannah more than his parents.
"They did very few family activities together besides going to car races and one family vacation to the beach, and they never decorated their house for the holidays," recalled Osler. "Jared's parents were religious but Jared proclaimed himself as an atheist."
Meanwhile, in the search for clues to understanding why Jared Loughner allegedly plotted an attack on Giffords' constituent event Saturday, his relationships with his parents and home environment are of increasing interest.
Pima County Sheriff Clarence Dupnik said Monday that he couldn't comment specifically on Loughner's upbringing or mental health, but he noted that his was a "somewhat dysfunctional family."
Other neighbors painted a picture of a single-child home that was intensely private and increasingly insular and standoffish as Jared Loughner grew older.
George Gayam, who has lived next door to Randy Loughner for 30 years on North Soledad St., described their early relationship as amicable and engaging, like "average neighbors."
Randy married Amy Totman in 1986 and two years later they had their first and only son, Jared. He then became a stay at home dad, while Amy worked for Pima County, neighbors said.
Gayam, 82, recalled how his grandchildren interacted with the Loughners as they were growing up, playing with Jared in the yard and later sharing a passion for cars with Randy.
"When I was probably 16 or 17, I had a Mustang. Randy had his hot rod. We'd talk shop. He'd help me out. I'd help him out, and everything was great," said Gayam's grandson Rick Dahlstrom.
But around 15 years ago the dynamic abruptly changed, Dahlstrom said.
"There was times when we'd be out with other neighbor kids, and Jared wouldn't be allowed out. He'd be watching from the window or door," he said. "They all became very isolated. Randy was isolated, Amy wasn't out anymore. Something changed. They just kept to themselves."
"We used to talk, you know, though not a lot," said Gayam. "But recently there was always some choice words said at times or gestures when someone was driving by. There's no real rhyme or reason as to why."
Neighbors said the Loughners' behavior perplexed them and made them uncomfortable.
"Contempt breeds contempt," said a female neighbor who asked to remain anonymous because she scared of Randy. "The family was contemptuous. It wasn't the son. It was the father."
But Smith, who also lives on the block and is reported to be the only neighbor in contact with the Loughners since the shooting, suggested earlier this week that the family's isolation was rooted in a sense of personal privacy.
"They don't really interact with the neighborhood because they're very private people," he said.
ABC News' Desiree Adib contributed to this report.