Although the couple owns a condo two miles away from the hospital, Kelly prefers to be as close as possible to his wife's bedside, where he tells her he loves her and reads out loud some of the thousands of supportive letters and e-mails that have flooded in since the attack.
So far, doctors say, Giffords' recovery has been nothing short of miraculous. She was removed from a ventilator Saturday and her condition was upgraded Monday to serious from critical.
Doctors have inserted a tracheotomy tube in her throat to help her breathe and put a feeding tube in her stomach.
While his wife has yet to speak or hint at how much she understands, Kelly has witnessed Giffords following instructions, like a command to hold up two fingers.
The hopeful emotions were a world away from what was going through his mind a week and half ago when Kelly was in Houston and got a phone call from one of his wife's staffers, telling him his "Gabby" had been shot. He had spoken to his wife 30 minutes before.
"I picked up the phone and she says, 'I don't know how to tell you about this, but I just received a call ... and Gabby's been shot,'" Kelly told Sawyer. "I said, 'Well, that's, you know, that's not possible. Are you sure?'"
With no other information, Kelly ended the conversation and hung up his cell phone. He had to look at the phone's call history to make sure he hadn't imagined the news. He told his children -- Giffords' stepdaughters -- and then called Giffords' parents and his own. Then, there was one thought in his head.
"[I] quickly had to figure out how I'm going to get there very, you know, very fast," Kelly said.
On board a friend's private plane rushing him and his family to his wife's side, Kelly was desperate for updates from the ground. He watched live television reports which erroneously declared that Giffords had lost her life, mourning the loss for 20 crushing minutes.
"The kids ... Claudia and Claire start crying. My mother, you know ... I think she almost screamed. And I just, you know, walked into the bathroom, and you know, broke down," Kelly, an astronaut, told ABC's Diane Sawyer in an exclusive interview. "To hear that she died is just, it's devastating for me."
For Kelly, it was the worst moment in a nightmarish day. Calling from the plane, Kelly learned the truth from people who were with Giffords at the hospital -- that his wife had not died, but she was fighting for her life.
By the time Kelly arrived in Tucson's University Hospital, the initial shock had worn off and another emotion had set in -- anger.
"I was really angry for two to three days. Very, very angry," Kelly said. "The first call I received after I arrived at the hospital was President Obama, and I expressed to him ... how angry I was."
"Whom did you blame?" Sawyer asked.
"Initially, I was upset because she'd, you know, received death threats before," Kelly said, adding that he felt it was "just part of what we've been dealing with for the last year."
He said they had talked "dozens of times" about how "risky" Giffords' job was.
"She says, you know, 'Someday I'm really worried that somebody's going to come up to me at one of these events with a gun,'" Kelly said.
Sawyer asked, "Do you still think the climate in this country had anything to do with it?"