Speaking at the National Cathedral, the president said it began when as a youth, he saw his home senator speaking on television during the Watergate hearings.
"I can't say that I understood everything that was being discussed, but I knew the issues were important. I knew they spoke to some basic way about who we were and who we might be as Americans," he said.
"The person who fascinated me most was this man of Japanese descent with one arm, speaking in this portly baritone, full of dignity and grace. And maybe he captivated my attention because my mom explained that this was our senator and that he was upholding what our government was all about."
Senator Inouye passed away Monday at the age of 88 as the longest serving sitting senator and the second-longest serving senator in U.S. history. He became Hawaii's senator in 1962, three years after the state joined the United States. His last word was reportedly "aloha."
President Obama said he watched the Watergate hearings while, at age 11, he took his first vacation to mainland U.S. But between trips to Disneyland and his mother's home state of Kansas, Obama was drawn to Inouye for other reasons.
"Maybe it was a boyhood fascination with the story of how he'd lost his arm in war," he continued. "But I think it was more than that. Now here I was, a young boy with a white mom, a black father, raised in Indonesia and Hawaii, and I was beginning to sense how fitting into the world might not be as simple as it might seem."
Inouye was a Medal of Honor veteran who served in the European theater in World War II when some Japanese Americans faced prejudices at home, with many detained in camps for years out of fear of foreign infiltrators. But the senator's rise from war hero to an accomplished lawmaker for the Aloha State told a young Obama what was possible in life.
"I might never have considered a career in public service. I might not be standing here today."
The president said it was a "privilege" to serve with Inouye in the Senate; a trip to Inouye's Capitol Hill office was one of the first places he visited when elected to Senate in 2005.
Senator Inouye was also honored by Vice President Biden, Sen. Harry Reid, Veteran's Affairs Sec. Eric Shinseki, and former President Bill Clinton at the service.