The singer touched the hearts of millions of music lovers with the 1958 song "Chega de Saudade," best known for the lyrics: "Go away my sadness and tell her than without her, it can't be... as I can't suffer anymore."
Jazz legend Miles Davis once said the singer "could read a newspaper and sound marvelous."
Born in 1931 in Juazeiro, a small city in the north of Bahia state, Joao Gilberto got his first guitar from his grandfather when he turned 14, and soon after formed his first band, Enamorados do Ritmo (Lovers of Rhythm).
After moving to Salvador, the capital of Bahia, in 1947, he soon gave up school to dedicate his life to music. He was hired by a vocal quintet, Os Garotos da Lua (Boys of the moon) and moved to Rio in 1950, leaving the band a year later.
It was while he was in Rio de Janeiro that Gilberto released the legendary record "Chega de Saudade," which introduced his romantic voice to a wider audience, complete with musical arrangements from another Brazilian music icon, Antonio Carlos Jobim.
According to Patrick Forge, a musician and producer, Joao Gilberto was a major influence on U.S. jazz.
"Without Joao Gilberto's Bossa Nova, lounge music would not exist," Forge told ABC News.
With a career spanning over seven decades, Gilberto has been constantly reinterpreted by each generation.
He was a unique character, and would refuse to play in certain clubs because he thought customers talked too much and failed to pay attention to the music, according to his biographer Ruy Castro, in his book, "Chega de Saudade: History of Bossa Nova."
"He was quiet and withdrawn. His character reflected his music. He needed silence to initiate his swing," Forge said.
The turning point of Gilberto's international career was the release of the album "Jazz Samba" with saxophonist Stan Getz. Released in 1963, it included the hit "The Girl from Ipanema," for which Gilberto won a Grammy in 1965.
"This is where the world got for the first time a taste of Brazilian music at an equal level [to] jazz," Forge told ABC News. "Gilberto influenced generations of musicians."
Today, in a public ceremony, Gilberto's open casket will be displayed at the Municipal Theater of Rio de Janeiro, one of the most important music venues in Brazil.
His daughter, singer, Bebel Gilberto, was touring the U.S. when his death was announced. She posted on her Facebook account to say Gilberto was the "Greatest Dad" anyone could dream of.
"I always hummed his songs," Suely Bispo, a mourner who attended the musician's public funeral in Rio, told ABC News. "You can be full of joy, or completely sad. You always have a Joao Gilberto song ready for you."